BigData Engineer (Python or Scala + Spark) w Warszawa, Polska
Poland, Warszawa
BigData Engineer (Python or Scala + Spark)
We are looking for a skilled BigData Engineer to join an exciting project for a leading banking client , one of the top players in the Polish market. You will work on building and improving Big Data solutions that support key business processes, using modern technologies and best engineering practices.
It's a Warsaw-based opportunity. The team visit the office once in two weeks.
Responsibilities:
Write and maintain data pipelines and applications using Python or Scala
Work with Apache Spark and other Big Data tools to process large amounts of data
Use data warehouse principles to keep data well-structured and reliable
Follow good engineering practices: clear code, proper testing, documentation, and clean design
Handle different data formats like JSON, Parquet, ORC, and Avro
Write and improve SQL queries to work with data efficiently
Combine data from different sources into a single system
You will be perfect candidate if you have:
Minimum 3 years of experience in Python or Scala programming
Commercial experience and exposure to BigData technologies, Spark is a must
Familiarity with Data Warehouse principles
Knowledge of good engineering practices, Big Data processing, including Design standards, Data Modelling techniques, coding, documentation, testing and implementation
Experience with various data formats: JSON, PARQUET, ORC, AVRO
Understanding database types and usage scenarios, e.g. Hive, Kudu, HBase, Iceberg, etc.
Advanced knowledge of SQL Experience in integrating data from multiple data sources
Knowledge of tools for building projects/applications, e.g. Maven
Advanced knowledge of Polish and good knowledge of English
It will be a strong plus to have:
Practical knowledge of Agile processes and tools: Jira, Confluence, Kanban, Scrum, etc.
Knowledge of the Kubeflow platform
Knowledge of streaming technologies and tools such as Kafka, Apache Nifi
DCG Poland is looking for experienced professionals to develop distributed BigData processing pipelines. Candidates should have 7+ years of expertise in Spark and Scala, along with substantial experience in Python and Robot Framework. You will collaborate with DevOps and QA teams in an Agile environment, maintaining a focus ...
... scalable REST APIs for data or ML services. - Strong testing practices, including TDD and unit/integration tests for Spark and Airflow. - Knowledge of Scala for high-performance data processing. Data Engineer We’re looking for an experienced Data Engineer to build, scale, optimize, and maintain reliable data platforms. ...
... large-scale data processing systems. You will be responsible for designing systems that handle billions of requests daily, improving application efficiency and scalability. With a minimum of 8 years’ experience in software development, proficiency in Java, Scala, or Python, and expertise in Big Data systems, the role is pivotal ...
... global organizations. Your Responsibilities Designing and building scalable, production-grade backend services and REST APIs using Python and FastAPI. Owning your work end-to-end - from initial spec and design through deployment and monitoring. Working with PostgreSQL to model, query, and optimize data storage for reliability ...
... Projektowanie, rozwijanie i utrzymywanie rozwiązań Data Engineering w środowisku Hadoop i Apache Spark., Tworzenie skalowalnych procesów przetwarzania dużych zbiorów danych., Analizowanie złożonych wymagań biznesowych i przekładanie ich na rozwiązania techniczne., Tworzenie wydajnych zapytań SQL oraz optymalizacja procesów ...
Senior Data Engineer (Java/Scala, Spark, AWS) Miejsce pracy: Warszawa Technologies we use Expected - Spark - Java - Scala - SQL Optional - Databricks - Snowflake About the project We are seeking a Senior Data Engineer to design, build, and maintain large-scale data pipelines and distributed processing systems that power ...
Expected, Spark, Java, Scala, SQL Optional, Databricks, Snowflake About the project, We are seeking a Senior Data Engineer to design, build, and maintain large-scale data pipelines and distributed processing systems that power AI-driven initiatives. In this role, you will leverage your expertise in AWS, Spark, and modern ...
We are seeking a Senior Data Engineer to design, build, and maintain large-scale data pipelines and distributed processing systems that power AI-driven initiatives. In this role, you will leverage your expertise in AWS, Spark, and modern data architectures to deliver secure, scalable, and highly available data solutions ...
... backend and data-intensive solutions using Python and PostgreSQL, working with complex systems, large datasets, and performance-critical applications. You will work within an international engineering environment and contribute to the development, optimization, and maintenance of scalable software solutions. Key Responsibilities ...
... Doświadczenie w pracy jako fullstack engineer. Szukasz miejsca, w którym Twoja praca naprawdę ma znaczenie? W CloudFerro tworzymy technologie, z których na co dzień korzystają europejscy liderzy nauki i sektora kosmicznego - m.in. ESA, EUMETSAT, ECMWF, Mercator Ocean International, DLR czy EGI. Pracując blisko sektora Earth Observation ...
We are looking for an experienced Senior Python Engineer with expertise in Cloud technologies to contribute to a strategic project for one of our key clients. As a Senior Engineer, you will create, enhance, and implement scalable, secure solutions utilizing Python and modern cloud platforms. Collaboration with cross-functional ...
We are looking for an experienced Senior Python Engineer with expertise in Cloud technologies to contribute to a strategic project for one of our key clients. As a Senior Engineer, you will create, enhance, and implement scalable, secure solutions utilizing Python and modern cloud platforms. Collaboration with cross-functional ...
We are looking for an experienced Senior Python Engineer with expertise in Cloud technologies to contribute to a strategic project for one of our key clients. As a Senior Engineer, you will create, enhance, and implement scalable, secure solutions utilizing Python and modern cloud platforms. Collaboration with cross-functional ...
... automation, and Agile best practices Requirements Bachelor’s degree in Computer Science, Engineering, or related field At least 5 years of experience in Data Engineering with a strong ETL background Hands-on experience with Azure Data Factory, Azure Databricks, Azure SQL, ADLS Proficiency in Python, Scala, or Java and strong ...
... move away from legacy solutions. What we expect Must-have: at least 3 years of experience in data engineering; practical experience with at least one cloud platform: GCP or Azure; hands-on experience with at least one of the following: BigQuery, Snowflake or Databricks; strong Python or Java/Scala skills; strong SQL skills; ...
... move away from legacy solutions. Our requirements - at least 3-4 years of experience in data engineering; - practical experience with at least one cloud platform: GCP or Azure; - hands-on experience with at least one of the following: BigQuery, Snowflake or Databricks; - strong Python or Java/Scala skills; - strong SQL ...
... Support testing, logging, and monitoring efforts; - Collaborate with frontend, hardware, and energy engineering teams; - Learn and apply best practices under senior engineer guidance. Qualifications: - Completed degree in computer science, energy engineering, or a comparable field; - Early hands on experience with Python (e.g. ...
... and resilience through proactive engineering practices. - Work independently with strong ownership while collaborating across teams. - Note: Detailed project information will be shared during the recruitment process. Our requirements - Strong Python skills (Python 3.11+) and solid object-oriented programming fundamentals. ...
... of your design decisions. You make smart trade-offs to unblock shipping - and you stand by those decisions. You keep raising the bar on engineering excellence for yourself and everyone else. What We Are Looking For Strong senior-level backend engineering experience in production systems Strong Python skills and the ability ...
... currently seeking candidates for the position of:, , Python Software Engineer - IT Payroll Integrations Your responsibilities, Design and implement solutions to support our payroll systems that adhere to technical standards., Drive integration and software development activities around ADP Global View and SuccessFactors., Participate ...