We’re looking for a Mid Data Engineer to take ownership of our data lake — the system of record for millions of financial events every day, including bets , wallet transactions, and live odds, serving 12M+ active users. You’ll be responsible for designing and maintaining reliable data pipelines and defining how data is ingested, stored, retained, reconciled, and governed across AWS S3 and Snowflake/Databricks. Your work will ensure that Analytics, Finance, and Regulatory teams have access to accurate, consistent, and fully traceable data — with zero drift from source systems.
Location: Kraków, Poland. Hybrid — 2 days per week from the office .
What you will do:
Own the lakehouse architecture: bronze/silver/gold layers, Iceberg/Delta tables, schema evolution.
Land operational data via CDC streaming (Kafka, Debezium), handling late and duplicate events.
Design data layout for speed and cost: partitioning, compaction, file sizing, query performance on Trino/Athena/Snowflake.
Own retention and archival: storage tiering, regulatory retention, immutability, GDPR deletion.
Guarantee correctness: freshness SLAs, drift detection, reconciliation against the source wallet and ledger systems.
Own governance: catalog and lineage, row/column access control, PII masking, encryption, audit trails.
Monitor ingestion health, data anomalies, and cloud storage/compute spend.
REQUIREMENTS
Must-have:
+ years of hands -on in a production lakehouse environment.
Lakehouse architecture — bronze/silver/gold layering, an open table format (Iceberg, Delta, or Hudi), schema evolution.
Data layout & query optimization at TB+ scale — partitioning, compaction, file sizing, query performance on Trino/Athena/Snowflake.
Cloud lakehouse/DWH in production — Snowflake, Databricks, or BigQuery.
CDC & streaming ingestion — Kafka + Debezium or equivalent; late, duplicate, and out-of-order events.
Strong SQL and data modeling — enough relational grounding to reason about the OLTP systems you capture from. Critical for financial ledgers.
Expected, Python, SQL, pyspark, Databricks, BigData, GCP About the project, As a Data Engineer, you will play an important role in designing, building and maintaining reliable data solutions that support Kingfisher’s data platforms and products., , Working as part of a collaborative engineering team, you will contribute ...
... Windows About the project, We're looking for a talented Data Engineer to join our client's team and take a central role in building and optimizing high-performance data workflows. If you love working with large datasets, PySpark, and solving tough data challenges, this opportunity is for you! This is how we work, at the client's ...
... project information will be shared during the recruitment process. Bachelor’s degree in Computer Science, Engineering, or related field, Proven experience as a Data Engineer or similar role, Strong knowledge of data processing technologies and database systems (SQL, NoSQL), Experience with data pipeline and workflow management ...
Wymagane, Google Cloud Platform, BigQuery, Datastream, Dataplex, Google Cloud Storage, Cloud Dataflow (Apache Beam), Cloud Composer (Apache Airflow), Pub/Sub, Dataform, Python, SQL, MySQL, PostgreSQL, MS SQL, Git, CI/CD O projekcie, Poszukujemy Data Engineera, który dołączy do zespołu projektowego i utrzymaniowego, wspierając ...
About you - 3+ years of hands-on experience in Data Engineering or Software Engineering. - Degree in computer science, engineering or equivalent technical field experience. - Proficient in cloud-based data processing and storage technologies like Databricks, Spark, Airflow. - Fluent in SQL and proficient in at least one ...
Data Engineer Miejsce pracy: Wrocław Technologie, których używamy Wymagane - Python - SQL - Spark - Polars O projekcie Budujemy „mózg” biddera RTB — systemu, który w imieniu klientów e-commerce kupuje powierzchnię reklamową w aukcjach czasu rzeczywistego. Nasze modele ML decydują, jaką kampanię wyświetlić, jak powinna wyglądać ...
- Minimum 3 lata komercyjnego doświadczenia jako Data Engineer - Praktyczna znajomość Pythona w tworzeniu i utrzymaniu produkcyjnych pipeline'ów ETL/ELT - Doświadczenie w integracjach z REST API i przetwarzaniu danych JSON/XML - Dobra znajomość SQL - Doświadczenie w walidacji danych, obsłudze schema drift, logowaniu błędów ...
... visualization tools (e.g., Power BI, Tableau). - Knowledge of machine learning, predictive analytics, and statistical modeling techniques. - Experience with cloud-based data and analytics platforms (Azure, AWS). - Microsoft Data & AI, Azure Data Engineer, Power BI, or similar analytics certifications are an advantage. What we offer ...
... visualization tools (e.g., Power BI, Tableau)., Knowledge of machine learning, predictive analytics, and statistical modeling techniques., Experience with cloud-based data and analytics platforms (Azure, AWS)., Microsoft Data & AI, Azure Data Engineer, Power BI, or similar analytics certifications are an advantage. What we offer, ...
... data ingestion and transformation processes., Work with cloud-based data platforms and Databricks., Integrate data from multiple operational systems., Build data products that support aviation operations and decision-making., Work with APIs and data exchange services., Collaborate with data providers, engineers and business ...