Senior Data Engineer, (Global Security)
RBC
Job Description
What is the opportunity?
For a Senior Data Engineer to join our collaborative cyber security team, focusing on building scalable data solutions that directly enhance our application ecosystem. This role will work together with our application developers to integrate a robust data capability into our application to transform how our products leverage data for better design and business value. You’ll be instrumental in developing a comprehensive data strategy that seamlessly bridges the gaps between data and application functionality.
What will you do?
Design, develop, and maintain end-to-end data pipelines in Azure Databricks using Spark (SQL, PySpark).
Implement and optimize ELT/ELT workflows using Databricks Workflows or Apache Airflow ensuring data integrity, quality, and reliability.
Manage Delta Lake solutions for data versioning, incremental loads, and efficient application data access. Apply best practices in data governance, ensuring compliance using Unity Catalog for access management and data lineage tracking.
Monitor, troubleshoot, and optimize Spark jobs for performance, addressing data pipelines bottlenecks that impact application responsiveness.
Build automated monitoring, alerting, and incident managements solution to ensure data reliability, availability, and performance.
Work directly with Python application developers to integrate data capabilities into existing and new applications.
Collaborate with our developers and cross-functional teams to integrate data capabilities into existing and new applications.
Build APIs and data services that applications can consume for real-time and batch data processing.
Develop and maintain comprehensive documentation for data pipelines, transformations and data models. Foster knowledge and collaborative functions
What do you need to succeed?
Bachelor’s or master’s degree in computer science, Data Engineering a, or a related field.
5+ years of proven experience in data engineering, delivering business-critical software solutions for large enterprises with a consistent track record of success.
Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Delta Live Pipelines, Databricks Runtime, Cluster management etc.)
Strong experience in Spark and PySpark for big data processing.
Strong experience in application development in python, building and maintaining microservices.
Knowledge of SCM, Infrastructure-as-code, and CI/CD pipelines.
Nice to Have
Databricks certifications (e.g., Databricks Certified Data Engineer, Spark Engineer).
Exposure to Kubernetes, Docker, and Terraform.
Strong understanding of business intelligence and reporting tools.
Familiarity with Cyber Security Concepts.
What’s in it for you?
We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.
A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
Leaders who support your development through coaching and managing opportunities
Ability to make a difference and lasting impact
Work in a dynamic, collaborative, progressive, and high-performing team
A world-class training program in financial services
Opportunities to do challenging work. Opportunities to take on progressively greater accountabilities. Opportunities to building close relationships with clients
Access to a variety of job opportunities across business and geographies.
#LI-Post
#LI-PK
#TechPJ
Job Skills
Big Data Management, Cloud Computing, Database Development, Databricks Platform, Data Engineering, Data Mining, Data Pipelines, Data Warehousing (DW), ETL Development, ETL Processing, Group Problem Solving, Microservice Framework, Microsoft Azure Databricks, Python (Programming Language), Quality Management, Requirements AnalysisAdditional Job Details
Address:
City:
Country:
Work hours/week:
Employment Type:
Platform:
Job Type:
Pay Type:
Posted Date:
Application Deadline:
Note: Applications will be accepted until 11:59 PM on the day prior to the application deadline date above
Inclusion and Equal Opportunity Employment
At RBC, we believe an inclusive workplace that has diverse perspectives is core to our continued growth as one of the largest and most successful banks in the world. Maintaining a workplace where our employees feel supported to perform at their best, effectively collaborate, drive innovation, and grow professionally helps to bring our Purpose to life and create value for our clients and communities. RBC strives to deliver this through policies and programs intended to foster a workplace based on respect, belonging and opportunity for all.
Join our Talent Community
Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you.
Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com.