Expected, Python, Databricks, SQL, Microsoft Azure Optional, Airflow, Dagster, Apache Spark, Microsoft Power BI, Azure Event Hub, Kafka, Terraform Operating system, Windows, macOS About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging ...
... delivering advanced data-driven solutions. Our expertise lies in Cloud & Big Data engineering, where we build scalable architectures for processing large and complex datasets across AWS, Azure, and GCP. We manage the entire Software Development Life Cycle (SDLC), leveraging modern data frameworks, programming methodologies, and ...
... inclusive world. Your Tasks - Design and implement data processing solutions using Databricks for large-scale and diverse datasets. - Design, build, and enhance data pipelines with Python and cloud-native tools. - Work closely with solution architects to define and uphold best practices in data engineering. - Ensure data consistency, ...
Senior Data Engineer (Databricks & Python) Miejsce pracy: Warszawa Technologies we use Expected - Python - PySpark - Spark SQL - Delta Lake About the project We are looking for an experienced Senior Data Engineer to join our Labeling Data Platform team. The platform supports the collection, processing, transformation, governance, ...
... diverse sources, ensuring efficiency and scalability., Collaborate with Data Science teams on Machine Learning projects, including text/image analysis, feature engineering, and predictive model deployment., Design and manage complex data transformations using Databricks, DBT, and Apache Airflow, ensuring data integrity and ...
... with Databricks and/or Snowflake - Practical experience with data acquisition using APIs, FTP downloads, and other data sources - Experience with ETL processes, data transformation, and normalization from raw data to structured database schemas - Experience with data storage solutions including raw files and database systems ...
... procesów danych, - współpraca z międzynarodowymi zespołami projektowymi i klientami. Nasze wymagania - minimum kilka lat doświadczenia komercyjnego w obszarze Data Engineering, - praktyczne doświadczenie z Databricks lub Snowflake, - bardzo dobra znajomość PySpark / Apache Spark, - dobra znajomość Python i SQL, - doświadczenie ...
... on Data & AI projects - Build modern data platforms that enable analytics, machine learning, and AI. - Grow your Databricks expertise - Work with the latest Databricks technologies, cloud platforms, and engineering best practices. - Learn from experienced specialists - Collaborate with Architects, Engineers, and Data Scientists ...
B2B Project: Data Engineer - Databricks Hays Contracting Services operates under a B2B service delivery model. We are currently looking for a B2B contractor to provide Data Engineering services on the Azure Databricks platform, with responsibility for designing and building scalable data solutions. Location, remote within ...
Skills & requirements: - Strong hands-on experience with Databricks and Apache Spark. - Advanced knowledge of PySpark, Python, and SQL. - Knowledge of ETL tools. - Experience working with Azure, AWS, or GCP. - Familiarity with Git, CI/CD, testing, and modern software engineering practices. - Ability to read code and understand ...