BigData Quality Engineer

Job Description

What's the opportunity?
As part of the Global Functions Technology (GFT) within RBC's Technology and Operations division, you'll be at the heart of a team that extends its services across the organization, offering IT solutions that drive transformation and efficiency. Our collaborative efforts span various domains including Risk, Finance, HR, CAO, Audit, Legal, Compliance, Financial Crime, Capital Markets, Personal and Commercial Banking, and Wealth Management. Moreover, we're at the forefront of creating digital tools and platforms that foster better collaboration across the board.

In this dynamic environment, you'll have the opportunity to work closely with leadership that values the recognition of achievements and the sharing of insights across teams to fuel continuous improvement.

Joining our Finance IT Data As a Service team, you'll play a supporting role in Test Implementation and Test Automation on our cutting-edge Big Data Platform. This role involves working with the latest technologies and programming languages such as Cloudera, Spark, Scala, Unix, SQL, Python, Databricks, and leveraging GenAI and Agentic AI.

Your primary responsibility will be to support QE Automation for our diverse client groups - executing tests, building automation scripts, and validating data pipelines under the guidance of senior engineers. This position offers a unique chance to develop your technical expertise in big data testing, finance data, and AI-driven quality engineering, all while delivering IT solutions in a fast-paced and ever-changing business landscape.

What you will do?

• Execute test cases for data pipelines from ingestion to consumption, validating transformations, aggregations, and business rules across HDFS, Hive, Spark, and Databricks.

• Validate ETL/ELT logic using Python (PySpark) and SQL - joins, filters, lookups, derived columns, and data mappings; perform source-to-target reconciliation.

• Develop and maintain automated test scripts using AI, Python (Pandas, PySpark) and SQL

• Contribute to automated test suites with CI/CD integration (GitHub Actions, Ansible Automation Platform; automate schema validation, row count checks, and data comparisons across environments.

• Execute data quality checks - completeness, accuracy, consistency, and timeliness - on finance datasets

• Test Spark jobs, Hive queries, and Data Lake pipelines; validate data landing, partition management, file integrity, and data freshness.

• Leverage Generative AI tools (GitHub Copilot, Windsurf) to accelerate test case generation, SQL writing, and script development; support building Agentic AI workflows for autonomous test execution and intelligent triage.

• Validate data feeding into ML models and test AI-generated outputs for correctness; support feature store and training data quality validation.

• Understand and test finance data attributes: Trades, booking entities, currencies, GL accounts, product types, settlement dates; support regulatory and management reporting feed validation.

• Partner with data engineers, SRE, and product teams to support testability and observability; log defects with clear root cause analysis in JIRA; maintain test documentation in Confluence.

• Participate in Agile/Scrum ceremonies and contribute to shift-left testing practices and continuous testing workflows.

What youll impact

• Reliability and accuracy of critical data processing that powers Finance, Risk, Audit, Compliance, and enterprise reporting.

• Faster, predictable releases through standardized automation, early validation, and AI-enabled quality acceleration.

• Reduced operational risk via proactive defect prevention, audit-ready artifacts, and governance-aligned controls.

What youll need to thrive

Must-Haves

• 1-2 years hands-on experience in QE, data testing, or data engineering in Big Data environments.

• Strong foundation: Bachelor's degree in Computer Science, Engineering, Finance, or equivalent experience.

• SQL proficiency: Spark SQL, Databricks SQL, Hive, Trino - writing and debugging queries for data validation and reconciliation.

• Code & scripting: Python (PySpark, Pandas), Shell scripting, SQL; ability to build and maintain automated test scripts.

• Big Data fundamentals: HDFS, Hive, Parquet, Spark; understanding of data pipelines

• Databricks: Experience with notebooks, Delta Lake, Databricks SQL, and basic workflow execution.

• Testing fundamentals: Test case design, defect lifecycle, regression testing, integration testing, source-to-target validation.

• Generative AI: Practical use of GitHub Copilot or similar AI assistants; prompt engineering for test generation and SQL writing; awareness of Agentic AI concepts.

• Finance data awareness: Basic understanding of positions, valuations, products, currencies, booking entities, and reporting feeds (or strong willingness to learn quickly).

• DevOps/toolchain: JIRA, Git, Confluence, Linux command line; familiarity with CI/CD concepts.

Nice to have

• Experience with data governance, audit readiness, and privacy-preserving testing.

• Exposure to test automation frameworks (Robot Framework, pytest, Selenium) and CI/CD pipelines (GitHub Actions, Jenkins, UrbanCode Deploy).

• AI/ML ops exposure: dataset curation, validation, synthetic data generation for tests.

Whats in it for you?

• We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.

• A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable

• Leaders who support your development through coaching and managing opportunities

• Ability to make a difference and lasting impact

• Work in a dynamic, collaborative, progressive, and high-performing team

• Opportunities to do challenging work

• Opportunities to take on progressively greater accountabilities

#LI-ASPOST

Job Skills

Additional Job Details

Address:

City:

Country:

Work hours/week:

Employment Type:

Platform:

Job Type:

Pay Type:

Posted Date:

Application Deadline:

Note : Applications will be accepted until 11:59 PM on the day prior to the application deadline date above

Our Employment Opportunities

At RBC, we are guided by living shared values of Client First, Integrity, Collaboration, Respect and Excellence and winning together as One RBC. We believe an inclusive workplace that has diverse perspectives is core to our continued growth as one of the largest and most successful banks in the world. Maintaining a workplace where our employees feel supported to perform at their best, effectively collaborate, drive innovation, and grow professionally helps to bring our Purpose to life and create value for our clients and communities. RBC strives to deliver this through policies and programs intended to foster a workplace based on respect, belonging and opportunity for all.

Join our Talent Community

Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you.

Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com .

RBC is presently inviting candidates to apply for this existing vacancy. Applying to this posting allows you to express your interest in this current career opportunity at RBC. Qualified applicants may be contacted to review their resume in more detail.