Expected, Python, Databricks, SQL, Microsoft Azure Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
Expected, Python, Databricks, SQL, Microsoft Azure Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
... delivering advanced data-driven solutions. Our expertise lies in Cloud & Big Data engineering, where we build scalable architectures for processing large and complex datasets across AWS, Azure, and GCP. We manage the entire Software Development Life Cycle (SDLC), leveraging modern data frameworks, programming methodologies, and ...
... inclusive world. Your Tasks - Design and implement data processing solutions using Databricks for large-scale and diverse datasets. - Design, build, and enhance data pipelines with Python and cloud-native tools. - Work closely with solution architects to define and uphold best practices in data engineering. - Ensure data consistency, ...
... compliance with data governance, security, and regulatory standards Our requirements - 3-5 YoE in Data Engineering with Databricks - Very strong Python, SQL, PySpark programming skills - Data Platforms: Databricks (Delta Lake, DLT), cloud data lakes / lakehouse architectures - Experience with Azure (ADLS, Data Factory, Synapse), ...
... evaluate new technologies in the data engineering space Minimum 3 years of experience in a Data Engineer role or similar position, Strong knowledge of Azure data services (Azure Data Factory, Azure Synapse Analytics, Azure Data Lake), Extensive experience with Databricks and Apache Spark, Proficiency in SQL and experience ...
... with Databricks and/or Snowflake - Practical experience with data acquisition using APIs, FTP downloads, and other data sources - Experience with ETL processes, data transformation, and normalization from raw data to structured database schemas - Experience with data storage solutions including raw files and database systems ...
... procesów danych, - współpraca z międzynarodowymi zespołami projektowymi i klientami. Nasze wymagania - minimum kilka lat doświadczenia komercyjnego w obszarze Data Engineering, - praktyczne doświadczenie z Databricks lub Snowflake, - bardzo dobra znajomość PySpark / Apache Spark, - dobra znajomość Python i SQL, - doświadczenie ...
... data ingestion from diverse sources utilizing technologies including RDBMS, REST HTTP API, flat files, Streams, Time series data, SAP - Research and utilize Data Software Engineering technologies for effective data ingestion - Process and transform data using tools such as Spark, Databricks and various Cloud Services - ...
... equivalent senior individual contributor leadership role, with a track record of shaping technical strategy and architecture across multiple teams., Strong software engineering background with experience designing and operating large-scale distributed systems, data platforms or cloud-native solutions., Deep expertise in ...