Senior Data Engineer (Azure, Databricks) w Wrocław,… - Jobeax
Opis oferty
Senior Data Engineer (Azure, Databricks) w Wrocław, Polska
OPTIVEUM SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ
ZdalniePraca z dowolnego miejsca
HybrydowoPołączenie biura i pracy zdalnej
StacjonarniePraca z biura
KontraktUmowa czasowa lub freelancing
165 zł
Wrocław, Polska
Your responsibilities
Extract, combine, transform, load and cleanse data from various sources to deliver high-quality datasets for analysis, including migration from legacy systems
Work with data modellers and business analysts to understand the data model and source-to-staging mapping
Create analysis-ready data structures, including DDLs for MS Cloud SQL Server tables and views in the Pond database (a subset of the Data Lake)
Design and develop Databricks notebooks that extract data from input files and join related entities into Shared File Formats (SFFs)
Configure Ingestion Framework tables to load staging and Data Lake Core (DLC) tables
Build large-scale batch and real-time data pipelines in cloud or on-premises environments and develop performance-efficient algorithms
Apply CI/CD best practices and introduce automation in existing and future integration flows
Track data consumption patterns and work with data governance and information stewards on the curated data catalogue
Drive improvements in data acquisition, testing and tooling to raise data quality
Monitor performance and optimise SQL queries
Prepare detailed design documents based on business and system requirement documents
Work in cross-functional agile teams and take part in requirement analysis and estimates
Our requirements
5+ years of experience in data engineering or a related field, with a strong focus on ETL and cloud data tools
Bachelor's degree in Computer Science, Engineering or a related field (Master's preferred)
Certifications: Microsoft Certified Azure Data Engineer and Databricks Certified Data Engineer
Proficiency in Python, Java or Scala
Hands-on experience with MS Cloud SQL Server, Azure Data Factory, Azure Databricks, PySpark, Spark SQL, ADLS, Power BI, SSIS and SSRS
Strong SQL skills and experience with relational databases
Familiarity with big data technologies and data processing frameworks
Experience with Agile/Scrum
Strong problem-solving and communication skills and a collaborative mindset
English at C1 level
Optional
Experience in financial services, ideally in a captive finance company, including customer, dealer and supplier data
Experience delivering global data marts used across regions
Expected, Python, Databricks, SQL, Microsoft Azure Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
Expected, Python, Databricks, SQL, Microsoft Azure Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
... practices in data engineering., Ensure data consistency, security, and scalability within cloud-based environments. Experience in data engineering, with hands-on Databricks expertise., Strong experience in Python for automation and data transformation., Experience working with at least one major cloud platform (AWS, Azure, ...
... Współpraca z zespołem przy projektowaniu technicznym, konfiguracji środowisk oraz testach integracyjnych i regresyjnych. Nasze wymagania - Doświadczenie w pracy z Azure Cloud. - Doświadczenie w wdrażaniu Databricks w środowisku Azure Cloud. - Znajomość zagadnień związanych z konfiguracją środowisk, deploymentem i automatyzacją ...
... Python, SQL, Microsoft Azure, Azure Data Factory, ETL/ELT, CI/CD, Databricks, Streaming Optional, dbt Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the opportunity to support and further develop an Azure-based data integration solution built primarily around Azure Data Factory ...
... implementing data governance and data management processes. - Deep expertise in Big Data technologies, including Databricks, Spark, Apache Airflow and other modern data orchestration and transformation tools. - Experience implementing and deploying solutions in cloud environments (with a preference for Azure ). - Knowledge of ...
Databricks Data Engineer Miejsce pracy: Wrocław Technologies we use Expected - Python - SQL - PySpark - Microsoft Azure - Git - Azure DevOps - Azure Databricks - Azure Data Factory Optional - Kafka - Event Hub - Microsoft Power BI - Tableau Operating system - Windows About the project In Cyclad we work with top international ...
... Research and evaluate new technologies in the data engineering space Minimum 3 years of experience in a Data Engineer role or similar position, Strong knowledge of Azure data services (Azure Data Factory, Azure Synapse Analytics, Azure Data Lake), Extensive experience with Databricks and Apache Spark, Proficiency in SQL and experience ...
... experience as a Data Engineer or in a similar role - Advanced knowledge of SQL and Python - Hands-on experience with public cloud platforms such as AWS and/or Azure - Experience with DBMS and ETL/ELT tools - Hands-on experience with Databricks and/or Snowflake - Practical experience with data acquisition using APIs, FTP ...
As a Senior Data Engineer – Azure, you will: - Design, develop, and maintain scalable and reliable data pipelines and enterprise data products. - Build and support data engineering solutions across cloud and on-premises environments. - Develop solutions using Azure Data Factory, Azure Synapse Analytics, Azure Data Lake ...