Senior Data Engineer - Python, Spark, AWS w Warszawa, Polska
Link Group
35000 zł
Warszawa, Polska
5+ years of experience in Python and strong software engineering skills
Solid knowledge of AWS cloud services and best practices
Experience building scalable Spark pipelines in PySpark or Scala
Practical experience with Spark Streaming for low-latency pipelines
Familiarity with Delta Lake and modern data lakehouse architectures
Hands-on experience with Kubernetes and container orchestration
Experience with MongoDB and other NoSQL data stores
Understanding of microservices and event-driven architecture
Excellent communication skills and ability to work in a collaborative, global team
Commitment to high standards of ethics, quality, and delivery
Nice to Have:
Familiarity with Machine Learning , AI models , or Data Science workflows
We’re looking for a Senior Data Engineer to join a highly skilled team responsible for building a scalable, multitenant data platform that powers data ingestion, transformation, cataloging, and distribution for internal and external stakeholders. This role is central to designing robust data pipelines, integrating modern data lakehouse components, and supporting the company’s AWS-based infrastructure.
,[Design and build scalable data ingestion pipelines in AWS, Work extensively with Databricks, including Unity Catalog, streaming, Delta Lake, and data transformation tools, Lead technical delivery and implementation of core platform components across raw, normalized, operational, and historical layers, Collaborate with engineering managers and cross-functional teams across the US and Poland, Drive technical excellence by improving team processes, architecture standards, and engineering best practices, Support and consult with stakeholders to ensure successful delivery of data-driven solutions, Contribute to the growth and maturity of the team’s cloud and data engineering capabilities] Requirements: Python, AWS, Cloud, Spark, PySpark, Scala, Kubernetes, MongoDB, NoSQL, Microservices, Communication skills, Machine learning, AI, Data science Additionally: Sport subscription, Private healthcare.
... engineers, Implement best practices for code quality and data security, Participate in code reviews and knowledge sharing sessions, Optimize costs associated with data storage and processing Proficiency in Python and Spark with at least 3 years in data engineering roles, Strong experience with Azure Databricks and PySpark, Proven ...
Senior/Lead Data Software Engineer (Python, Spark, Azure) Miejsce pracy: Wrocław Technologies we use Expected - Pytest - Spark - Microsoft Azure - PySpark - Docker - Kubernetes - Terraform Cloud - Gtilab About the project We are seeking a Senior/Lead Data Software Engineer to join our team working on a scalable, ML-ready ...
Expected, Python, SQL, Spark, Airflow, AWS, Iceberg Optional, Kubernetes, Kafka, NiFi, Trino, Hudi, Java, Scala, Docker, Databricks About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging projects across various industries, leveraging ...
Expected, Python, SQL, Spark, Airflow, AWS, Iceberg Optional, Kubernetes, Kafka, NiFi, Trino, Hudi, Java, Scala, Docker, Databricks About the project, As a Senior Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging projects across various industries, leveraging ...
... dzielenie się wiedzą oraz wspieranie rozwoju technicznego zespołu. Minimum 5 lata praktycznego doświadczenia jak Data Engineer lub Big Data, Bardzo dobra znajomość PySpark, Spark SQL, DataFrames oraz Structured Streaming., Doświadczenie w odczycie i zapisie danych w MongoDB z wykorzystaniem MongoDB Spark Connector., Praktyczne ...
Data Engineer (Spark) Miejsce pracy: Wrocław Technologie, których używamy Wymagane - PyTorch Mile widziane - Kubernetes - Docker - Prometheus - Grafana - Jenkins - MongoDB - Apache Spark - PyTorch O projekcie Do zespołu Data Platform poszukujemy doświadczonego Data Engineer ( Spark), który będzie odpowiedzialny za projektowanie, ...
Expected, Python, SQL, Spark, Airflow, AWS, Iceberg Optional, Kubernetes, Kafka, NiFi, Trino, Hudi, Java, Scala, Docker, Databricks, dbt About the project, As a Senior Data Engineer (Spark), you will join a complex data engineering project focused on building and developing scalable data pipelines and modern data processing ...
Expected, Python, SQL, Spark, Airflow, AWS, Iceberg Optional, Kubernetes, Kafka, NiFi, Trino, Hudi, Java, Scala, Docker, Databricks, dbt About the project, As a Senior Data Engineer (Spark), you will join a complex data engineering project focused on building and developing scalable data pipelines and modern data processing ...
... pracujemy, u klienta Twój zakres obowiązków, Rozwój procesów przetwarzania danych w chmurze (GCP) i środowisku Big Data., Projektowanie i optymalizacja zadań Spark / PySpark uruchamianych na klastrach GCP Dataproc oraz w ekosystemie Hadoop., Budowa frameworków walidacyjnych w PySpark/Python do weryfikacji schematów, transformacji ...
Data Engineer (Python, Apache Spark, Hadoop) (m/n/k) Miejsce pracy: Kraków Technologie, których używamy Wymagane - Jenkins - Python - PySpark - Hadoop - Spark - BigQuery - Google Cloud Platform - SQL O projekcie Poszukujemy inżyniera danych z silnym zapleczem programistycznym (Python + PySpark), który dołączy do międzynarodowego ...
Wymagane, Apache Kafka, Apache Spark, PySpark, Python, Java, MongoDB, CI/CD, Prometheus, Grafana System operacyjny, Linux O projekcie, W Ideamotive szukamy doświadczonej osoby na stanowisko Senior Data Engineer. Dołączysz do nas, aby realizować projekty dla klientów z sektora bankowego. Skupiamy się na budowie i rozwoju ...