Expected, Python, Databricks, SQL, Microsoft Azure Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
... (Terraform, Bicep). - Experience with real-time data streaming technologies such as Kafka or Spark Streaming. - Knowledge of cloud storage solutions like Data Lake, Snowflake, or Synapse. - Hands-on experience with PySpark for distributed data processing. - Relevant certifications such as Databricks Certified Data Engineer Associate ...
Senior Data Engineer (Databricks & Python) Miejsce pracy: Warszawa Technologies we use Expected - Python - PySpark - Spark SQL - Delta Lake About the project We are looking for an experienced Senior Data Engineer to join our Labeling Data Platform team. The platform supports the collection, processing, transformation, governance, ...
... with Databricks and/or Snowflake - Practical experience with data acquisition using APIs, FTP downloads, and other data sources - Experience with ETL processes, data transformation, and normalization from raw data to structured database schemas - Experience with data storage solutions including raw files and database systems ...
Your responsibilities - Formulate and execute innovative analytical solutions using Cloud Native, Big Data, and NoSQL-centric technologies - Develop and implement Cloud, On-Premise, and Hybrid solutions utilizing superior data frameworks - Collaborate with product and engineering teams to understand and drive decision-making ...
Expected, Databricks, Python Optional, AWS, Microsoft Azure, Google Cloud Platform, PySpark, Kafka, SQL About the project, Insights & Data is a thriving team of over 400 professionals focused on delivering advanced data-driven solutions. Our expertise lies in Cloud & Big Data engineering, where we build scalable architectures ...
B2B Project: Data Engineer - Databricks Hays Contracting Services operates under a B2B service delivery model. We are currently looking for a B2B contractor to provide Data Engineering services on the Azure Databricks platform, with responsibility for designing and building scalable data solutions. Location, remote within ...
Skills & requirements: - Strong hands-on experience with Databricks and Apache Spark. - Advanced knowledge of PySpark, Python, and SQL. - Knowledge of ETL tools. - Experience working with Azure, AWS, or GCP. - Familiarity with Git, CI/CD, testing, and modern software engineering practices. - Ability to read code and understand ...
... with Azure DevOps. - Experience with SQL Server. - English proficiency at a level suitable for direct client conversations. Optional - Experience with Azure Databricks or Microsoft Fabric. - Experience with data quality frameworks, including Great Expectations, dbt tests, Databricks Expectations, DLT quality rules, Soda, ...