Big Data Engineer Scala,spark w Polkowice, Polska

DCG Poland is looking for experienced professionals to develop distributed BigData processing pipelines. Candidates should have 7+ years of expertise in Spark and Scala, along with substantial experience in Python and Robot Framework. You will collaborate with DevOps and QA teams in an Agile environment, maintaining a focus ...

• Kontrakt

$2k USD

We are seeking a Senior Data Engineer to design, build, and maintain large-scale data pipelines and distributed processing systems that power AI-driven initiatives. In this role, you will leverage your expertise in AWS, Spark, and modern data architectures to deliver secure, scalable, and highly available data solutions ...

... will develop and maintain a data platform and reporting processes operating in a complex Big Data environment. Responsibilities: Design, develop and maintain Data Engineering solutions using Hadoop and Apache Spark. Build scalable solutions for processing large volumes of data. Analyse complex business requirements and ...

• Zdalnie

21000 - 28560 zł

... experience implementing, developing, or maintaining Big Data systems, data governance and data management processes. - Strong programming skills in Python (or Java/Scala): writing a clean code, OOP design. - Hands-on with Big Data technologies like Spark , Cloudera, Data Platform, Airflow, Iceberg, Kafka, and CI/CD. - Excellent ...

Join Unit8 SA in Warsaw, Poland, as a proficient Software Engineer. You will design, build, and maintain data pipelines and contribute to critical data & AI projects. This role involves collaborating with engineers and clients, requiring strong Python skills and a problem-solving mindset. Enjoy a flexible work environment, ...

• Zdalnie

... large-scale data processing systems. You will be responsible for designing systems that handle billions of requests daily, improving application efficiency and scalability. With a minimum of 8 years’ experience in software development, proficiency in Java, Scala, or Python, and expertise in Big Data systems, the role is pivotal ...

... Computer Science, Engineering, or related field At least 5 years of experience in Data Engineering with a strong ETL background Hands-on experience with Azure Data Factory, Azure Databricks, Azure SQL, ADLS Proficiency in Python, Scala, or Java and strong SQL skills Experience with big data frameworks, CI/CD, and Agile methodologies, ...

• Kontrakt

$2k USD

... technical field 5+ years experience in software development with Big Data technologies Advanced knowledge of Python, Scala, or Java Proficient with Spark and Databricks and/or Snowflake Experience with cloud platforms such as Azure, Google Cloud Platform, Amazon Web Services Demonstrated experience with big data models, ...

• Hybrydowo

... our hiring reflects that. You might be a Python expert with an unparalleled depth of knowledge about financial derivatives risk and pricing. Or you might be a big data guru who writes beautiful Scala (or PySpark) on massive and complex datasets. Or you may be somewhere in-between. Either way you are the kind of person we ...

• Zdalnie
• Kontrakt

20000 - 30000 zł

... Rearchitecture of Spark applications to Cloud Dataproc or BigQuery SQL. Automation with Cloud Composer. Data analytics platform for an investing company Azure-based data platform with Databricks as the central component. Complex ETL processes enriching investment data with Spark jobs in Scala. External data integrations, data ...

• Kontrakt

18000 - 25000 zł

Addepto is a leading AI consulting ( ) and data engineering ( ) company that builds scalable, ROI-focused AI solutions for some of the world's largest enterprises and pioneering startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. With an exclusive focus on Artificial Intelligence and Big Data, Addepto helps ...

• Kontrakt

100 zł PLN - 140 zł PLN

... and maintain scalable and efficient data pipelines in Azure and Databricks Develop and optimize ETL processes using Azure Data Factory and Databricks Build Big Data solutions aligned with business requirements Collaborate with Data Science, BI, and development teams to ensure high-quality data Monitor performance of data ...

... doświadczeniaw pracy z narzędziami i technologiami Big Data oraz w realizacji projektów analitycznych dla danych. Szczegółowe wymagania: - ekspertyza w technologii Big Data i ekosystemach takich jak Hadoop oraz Apache Spark; - projektowanie i wdrażanie skalowalnych rozwiązań Big Data dla dużych zbiorów danych; - dobór technologii ...

DCG seeks an experienced ML Engineer to manage the lifecycle of ML models from development to deployment and monitoring. You will implement MLOps, work with Spark and Python for data ingestion, and build distributed pipelines for massive datasets. The role involves collaboration with data scientists, DevOps, and IT to deliver ...

• Zdalnie
• Kontrakt

... Azure, Google Cloud Platform, AWS, Databricks, Apache Spark, PySpark, Apache Airflow, GitHub Actions, Terraform, Bicep About the project, At Cloudfide, we engineer solutions where data truly becomes an accelerator for innovation. We build and evolve large-scale Big Data and cloud-native platforms for global clients, including ...

Capgemini Poland seeks a data platform engineer to design and build scalable cloud-based data platforms. You will architect data pipelines, integrate diverse data sources, and collaborate with data scientists to operationalize ML models. Role requires hands-on cloud experience (AWS/Azure/GCP), strong SQL and programming ...

• Zdalnie
• Hybrydowo
• Stacjonarnie
• Pełny etat
• Kontrakt

... Experience in data engineering, data science, or related field. Proficiency with data architecture and data processing tools. Certifications: Azure Associate Data Engineer (DP-203), Databricks Professional Data Engineer, Apache Spark Developer Associate, Professional Data Engineer. Experience with Power BI, DAX, and Tabular ...

• Zdalnie

Job Description: Data Engineer, LDWH Migration Programme Level: Senior / Mid-Senior Main Responsibilities: Develop and maintain data pipelines using Azure Databricks. Implement ETL processes, ensuring data quality and integrity. Read and comprehend complex SQL stored procedures. Translate architecture direction into engineering ...

• Zdalnie
• Hybrydowo

... genuinely complex data pipelines, event streaming, and laying the pipeline foundation for future ML & AI workloads, this is for you. What you’ll be doing: Pipeline Engineering: Building & optimizing batch and real-time streams on GCP (BigQuery, Cloud Composer/Airflow, Cloud Run) and Kafka. Modern Transformation & Architecture: ...

• Kontrakt

... połączeń z systemami poza ekosystem CRM • implementacja mechanizmów kontroli jakości oraz walidacji danych Wymagania • Minimum 4-6 lat doświadczenia na stanowisku Data Engineer, preferencyjnie w sektorze o dużej skali danych • doświadczenie w projektowaniu i modelowaniu struktur danych (hurtownie danych, DataLake-ów) na potrzeby ...