Expected, Python, Databricks, SQL, Microsoft Azure Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
Expected, Python, Databricks, SQL, Microsoft Azure Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
... delivering advanced data-driven solutions. Our expertise lies in Cloud & Big Data engineering, where we build scalable architectures for processing large and complex datasets across AWS, Azure, and GCP. We manage the entire Software Development Life Cycle (SDLC), leveraging modern data frameworks, programming methodologies, and ...
... requirements - 5+ years of experience in data engineering or a related field, with a strong focus on ETL and cloud data tools - Bachelor's degree in Computer Science, Engineering or a related field (Master's preferred) - Certifications: Microsoft Certified Azure Data Engineer and Databricks Certified Data Engineer - Proficiency in ...
... diverse sources, ensuring efficiency and scalability., Collaborate with Data Science teams on Machine Learning projects, including text/image analysis, feature engineering, and predictive model deployment., Design and manage complex data transformations using Databricks, DBT, and Apache Airflow, ensuring data integrity and ...
... compliance with data governance, security, and regulatory standards Our requirements - 3-5 YoE in Data Engineering with Databricks - Very strong Python, SQL, PySpark programming skills - Data Platforms: Databricks (Delta Lake, DLT), cloud data lakes / lakehouse architectures - Experience with Azure (ADLS, Data Factory, Synapse), ...
... evaluate new technologies in the data engineering space Minimum 3 years of experience in a Data Engineer role or similar position, Strong knowledge of Azure data services (Azure Data Factory, Azure Synapse Analytics, Azure Data Lake), Extensive experience with Databricks and Apache Spark, Proficiency in SQL and experience ...
... with Databricks and/or Snowflake - Practical experience with data acquisition using APIs, FTP downloads, and other data sources - Experience with ETL processes, data transformation, and normalization from raw data to structured database schemas - Experience with data storage solutions including raw files and database systems ...
... for a highly experienced Senior Data Engineer / Technical Lead to take ownership of the architecture, design, and technical direction of a modern enterprise data platform built on Azure Databricks. This role will serve as the technical right-hand to the engineering leadership team, providing technical guidance to a team ...
... data ingestion from diverse sources utilizing technologies including RDBMS, REST HTTP API, flat files, Streams, Time series data, SAP - Research and utilize Data Software Engineering technologies for effective data ingestion - Process and transform data using tools such as Spark, Databricks and various Cloud Services - ...