Senior Data Engineer (Azure, Databricks) w Wrocław, Polska
OPTIVEUM SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ
ZdalniePraca z dowolnego miejsca
HybrydowoPołączenie biura i pracy zdalnej
StacjonarniePraca z biura
KontraktUmowa czasowa lub freelancing
165 zł
Wrocław, Polska
Your responsibilities
Extract, combine, transform, load and cleanse data from various sources to deliver high-quality datasets for analysis, including migration from legacy systems
Work with data modellers and business analysts to understand the data model and source-to-staging mapping
Create analysis-ready data structures, including DDLs for MS Cloud SQL Server tables and views in the Pond database (a subset of the Data Lake)
Design and develop Databricks notebooks that extract data from input files and join related entities into Shared File Formats (SFFs)
Configure Ingestion Framework tables to load staging and Data Lake Core (DLC) tables
Build large-scale batch and real-time data pipelines in cloud or on-premises environments and develop performance-efficient algorithms
Apply CI/CD best practices and introduce automation in existing and future integration flows
Track data consumption patterns and work with data governance and information stewards on the curated data catalogue
Drive improvements in data acquisition, testing and tooling to raise data quality
Monitor performance and optimise SQL queries
Prepare detailed design documents based on business and system requirement documents
Work in cross-functional agile teams and take part in requirement analysis and estimates
Our requirements
5+ years of experience in data engineering or a related field, with a strong focus on ETL and cloud data tools
Bachelor's degree in Computer Science, Engineering or a related field (Master's preferred)
Certifications: Microsoft Certified Azure Data Engineer and Databricks Certified Data Engineer
Proficiency in Python, Java or Scala
Hands-on experience with MS Cloud SQL Server, Azure Data Factory, Azure Databricks, PySpark, Spark SQL, ADLS, Power BI, SSIS and SSRS
Strong SQL skills and experience with relational databases
Familiarity with big data technologies and data processing frameworks
Experience with Agile/Scrum
Strong problem-solving and communication skills and a collaborative mindset
English at C1 level
Optional
Experience in financial services, ideally in a captive finance company, including customer, dealer and supplier data
Experience delivering global data marts used across regions
... cloud data tools, Bachelor's degree in Computer Science, Engineering or a related field (Master's preferred), Certifications: Microsoft Certified Azure Data Engineer and Databricks Certified Data Engineer, Proficiency in Python, Java or Scala, Hands-on experience with MS Cloud SQL Server, Azure Data Factory, Azure Databricks, ...
... for a highly experienced Senior Data Engineer / Technical Lead to take ownership of the architecture, design, and technical direction of a modern enterprise data platform built on Azure Databricks. This role will serve as the technical right-hand to the engineering leadership team, providing technical guidance to a team ...
Expected, Python, Databricks, SQL, Microsoft Azure, DevOps Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
... Volotea, Schmitz Cargobull, and many, many more. We work with leading technologies including AWS, Azure, GCP, Databricks, and Snowflake, and combine strong engineering culture with a consulting mindset and a continuous focus on growth and knowledge sharing. ,[Designing and implementing scalable data pipelines on Databricks., ...
... experience with Databricks (including Azure Databricks and Databricks Workflows) in production environments. - Expert-level proficiency in PySpark, Python, Spark SQL, Data Modelling, and Data Pipeline Design. - Experience implementing CI/CD pipelines for data engineering workloads, Git-based development, and version control. - Solid ...
... professional experience in Data Engineering, including ownership of production-grade data pipelines, - strong, demonstrable hands-on experience with Microsoft Azure and Databricks, - advanced SQL and Python skills, - strong experience designing, building, and operating data pipelines and analytical data products, - a good ...
... compliance with data governance, security, and regulatory standards 3-5 YoE in Data Engineering with Databricks, Very strong Python, SQL, PySpark programming skills, Data Platforms: Databricks (Delta Lake, DLT), cloud data lakes / lakehouse architectures, Experience with Azure (ADLS, Data Factory, Synapse), or equivalent cloud ...
Databricks Data Engineer Miejsce pracy: Wrocław Technologies we use Expected - Python - SQL - PySpark - Microsoft Azure - Git - Azure DevOps - Azure Databricks - Azure Data Factory Optional - Kafka - Event Hub - Microsoft Power BI - Tableau Operating system - Windows About the project In Cyclad we work with top international ...
... implementing data governance and data management processes. - Deep expertise in Big Data technologies, including Databricks, Spark, Apache Airflow and other modern data orchestration and transformation tools. - Experience implementing and deploying solutions in cloud environments (with a preference for Azure ). - Knowledge of ...
... Volotea, Schmitz Cargobull, and many, many more. We work with leading technologies including AWS, Azure, GCP, Databricks, and Snowflake, and combine strong engineering culture with a consulting mindset and a continuous focus on growth and knowledge sharing. ,[- Designing, developing, and maintainig scalable data ingestion ...