Your responsibilities

Extract, combine, transform, load and cleanse data from various sources to deliver high-quality datasets for analysis, including migration from legacy systems
Work with data modellers and business analysts to understand the data model and source-to-staging mapping
Create analysis-ready data structures, including DDLs for MS Cloud SQL Server tables and views in the Pond database (a subset of the Data Lake)
Design and develop Databricks notebooks that extract data from input files and join related entities into Shared File Formats (SFFs)
Configure Ingestion Framework tables to load staging and Data Lake Core (DLC) tables
Build large-scale batch and real-time data pipelines in cloud or on-premises environments and develop performance-efficient algorithms
Apply CI/CD best practices and introduce automation in existing and future integration flows
Track data consumption patterns and work with data governance and information stewards on the curated data catalogue
Drive improvements in data acquisition, testing and tooling to raise data quality
Monitor performance and optimise SQL queries
Prepare detailed design documents based on business and system requirement documents
Work in cross-functional agile teams and take part in requirement analysis and estimates

Our requirements

5+ years of experience in data engineering or a related field, with a strong focus on ETL and cloud data tools
Bachelor's degree in Computer Science, Engineering or a related field (Master's preferred)
Certifications: Microsoft Certified Azure Data Engineer and Databricks Certified Data Engineer
Proficiency in Python, Java or Scala
Hands-on experience with MS Cloud SQL Server, Azure Data Factory, Azure Databricks, PySpark, Spark SQL, ADLS, Power BI, SSIS and SSRS
Strong SQL skills and experience with relational databases
Familiarity with big data technologies and data processing frameworks
Experience with Agile/Scrum
Strong problem-solving and communication skills and a collaborative mindset
English at C1 level

Experience in financial services, ideally in a captive finance company, including customer, dealer and supplier data
Experience delivering global data marts used across regions
Java