Job Description: Data Engineer, LDWH Migration Programme Level: Senior / Mid-Senior Main Responsibilities: Develop and maintain data pipelines using Azure Databricks. Implement ETL processes, ensuring data quality and integrity. Read and comprehend complex SQL stored procedures. Translate architecture direction into engineering delivery. Conduct data profiling and quality assessments. Define acceptance criteria for data transformation. Collaborate with the programme team to support overall migration efforts. Participate in CI/CD processes for efficient deployment. Key Requirements: Senior hands-on experience with Azure Databricks (jobs, clusters, Unity Catalog, SQL warehouses). Expertise in PySpark / Python for transformation logic and ETL pipelines. Solid working knowledge of Delta Lake (schema evolution, MERGE, time travel). Advanced SQL skills (complex queries, window functions); ability to comprehend legacy ETL. Familiarity with Databricks Asset Bundles (DAB) and CI/CD practices (Azure DevOps, Git). Working knowledge of Azure Data Lake Storage Gen2. Comfortable understanding legacy ETL processes and source data profiling. Experience with data reconciliation (defining criteria, interpreting discrepancies). Ability to articulate requirements for junior engineers. Fundamental knowledge of Power BI / Tableau. Nice to Have: Experience in platform engineering and configuration refactoring using YAML. Ability to maintain environment governance across DEV, QAS, and PRD. Skills in source feasibility analysis for new data sources. Capability in transformation development across various data zones using PySpark and Delta Lake. Familiar with implementing acceptance criteria and conducting quality checks. Other Details: This position is part of a complex and dynamic migration programme and is pivotal for ensuring the successful transition of data systems. Remote engagement options are available, and collaboration with an internal team is essential.
... merit. • Competence in developing and maintaining API’s is considered a large merit. We are looking for an experienced Data Engineer professional to join our Data Product Team within the MDM unit. You will be responsible for provisioning data from our master data customer and product systems. Requirements: Python, Databricks, ...
... and Software Engineering - Experience with access management / administration platforms - Experience in the automotive or other large enterprise environment Data Engineer & .NET 100% remote | Poland We are looking for an experienced Data Engineer with software engineering skills to join a team working on a strategic reporting ...
... performance tuning - Experience with data integration or ETL processes - Knowledge of data quality principles - Experience working in Agile teams - Understanding of data ingestion pipelines What will you do? You will join a team responsible for onboarding and integrating new data feeds from upstream systems into internal source ...
... for data or ML services. - Strong testing practices, including TDD and unit/integration tests for Spark and Airflow. - Knowledge of Scala for high-performance data processing. Data Engineer We’re looking for an experienced Data Engineer to build, scale, optimize, and maintain reliable data platforms. You’ll work on real-time ...
Senior Data Engineer Miejsce pracy: Warszawa Technologies we use Expected - Google Cloud Platform - Terraform - Git - GitHub Actions - SQL - Python Optional - Docker - Kubernetes About the project We are looking for a Senior Data Engineer who will act as a technical mentor, define our architecture, and bring best practices ...
Data Engineer – Senior Miejsce pracy: Warszawa Technologies we use Expected - Google Cloud Platform Data and Analytics - Python - dbt - Airflow - Terraform - GitHub Operating system - Windows About the project We're looking for a senior Data Engineer who can step into an environment someone else built, work out how it really ...
... level). - Bachelor’s degree in technical or mathematical studies. Nice to have: - Experience with an MLOps framework such as Kubeflow or MLFlow. - Familiarity with Databricks, and/or dbt. Addepto is a leading AI consulting (https://jobeax.com/link/rOiEvBn3dZ9KkGj7 and data engineering (https://jobeax.com/link/cLdQJ7azj7GOWKYv ...
... we use Expected - SQL - Python About the project As a Senior Database Engineer, you will be responsible for designing, optimizing and scaling high-performance database infrastructure supporting millions of transactions and real-time data flows. You will work with transactional databases, document databases, Kafka-based streaming ...
... technical mentor for the team through pair programming, setting technical standards, and conducting code reviews. Minimum 5 years of commercial experience in data engineering with Google Cloud., Google Cloud Professional Data Engineer certification (expected or to be obtained within the first month of work)., Advanced knowledge ...
... implementing scalable, production-grade data pipelines, data models, and data processing frameworks in cloud environments., Leading the technical design of cloud data architectures, contributing to standards, guidelines, and long-term platform strategy., Driving best practices in data engineering: quality assurance, lineage, ...
... building ETL processes and data pipelines for integrating data across various systems., Enabling reliable and secure data sharing between applications., Ensuring data quality, consistency, and high availability., Monitoring, troubleshooting, and optimizing pipeline performance., Collaborating closely with engineering and business ...
Senior Data Engineer Miejsce pracy: Gdańsk Technologies we use Expected - Databricks - Apache Spark - Python - PyTorch - Microsoft Azure Optional - Pytest About the project We are looking for a skilled Data Engineer to join an international technology and data team on a 6-month engagement, starting in the first or second ...
... Databricks, Delta Lake, Azure Data Factory, C#, Microsoft Azure Optional, Microsoft Power BI About the project, Senior Data Engineer role focused on end-to-end data engineering in modern cloud architectures, bridging business needs with technical solutions. Join agile product teams to manage MDM data products, optimize data flows, ...
Wymagane, Databricks, Python, PySpark Mile widziane, Microsoft Fabric, Azure Synapse, Microsoft Power BI, Snowflake Data Cloud, BigQuery, Microsoft Azure O projekcie, Jako Senior Data Engineer dołączysz do jednego z naszych zespołów rozwijających produkty cyfrowe dla klientów z Europy Północnej, stawiając na jakość oraz ...
... projekcie, Wspólnie z naszym Partnerem, jednym z europejskich liderów w branży bankowej, poszukujemy doświadczonego/ej specjalisty/ki do realizacji projektów jako Big Data Engineer., , Projekt realizowany jest w obszarze bankowości B2B i B2C, w domenie C-Profit odpowiedzialnej za konsolidację i udostępnianie zespołom sprzedaży transakcji ...
... requirements, •Build analytic and aggregation layers that feed APIs, downstream systems, and AI/ML workflows, , Data Engineering & Integration, •Expose kdb+ data to front-end applications, AI engines, and reporting systems via APIs, •Collaborate with AI/data engineering teams to deliver structured, high-quality datasets ...
Senior Data Engineer Location: Wrocław - 2 days in office / 3 days remote Let us introduce you the job offer by EY GDS Poland – a member of the global integrated service delivery center network by EY. The opportunity We are building an innovative team responsible for analyzing ITIL process data and supporting business decision-making ...
... level). - Bachelor’s degree in technical or mathematical studies. Nice to have: - Experience with an MLOps framework such as Kubeflow or MLFlow. - Familiarity with Databricks, and/or dbt. Addepto is a leading AI consulting (https://jobeax.com/link/rOiEvBn3dZ9KkGj7 and data engineering (https://jobeax.com/link/cLdQJ7azj7GOWKYv ...
... from architecture to deployment, collaborating with a modern, high‑performing engineering team. What makes you a great fit: - 5+ years of professional data engineering experience - 2+ years working with programmatic advertising or ad-tech data - Strong proficiency in Python and SQL - Experience with streaming data platforms ...