Data Engineer - Batch and Streaming Pipelines w… - Jobeax
Opis oferty
Data Engineer - Batch and Streaming Pipelines w Polkowice, Polska
Pełny etatStandardowy tygodniowy wymiar godzin
KontraktUmowa czasowa lub freelancing
Poland, Polkowice
Data Engineer - Batch and Streaming Pipelines w Polkowice, Polska is listed on Jobeax. Browse 120,000+ vacancies available.
If you have 8-10 years of experience building and operating distributed data pipelines and are looking to advance your career working with modern data technologies, this contract role offers the opportunity to make a direct impact on their data infrastructure. Contract Type: Contract. Competitive contract rate reflecting your experience and expertise. Contract flexibility with potential for extension based on performance
SQL
Pełny etat
... cloud consolidation onto a modern GCP & Medallion (Bronze/Silver/Gold) architecture. If you thrive on handling genuinely complex data pipelines, event streaming, and laying the pipeline foundation for future ML & AI workloads, this is for you. What you’ll be doing: Pipeline Engineering: Building & optimizing batch and real-time ...
ComeOn Group is seeking a Data Engineer to design, build, and maintain near real-time and batch data flows powering production ML models. You will work with Data Scientists to create scalable pipelines in a hybrid work environment and help ensure data quality for model training and serving. The role requires 5+ years in ...
... and autonomous culture. ,[Develop and maintain a high-performance data processing platform for automotive data, ensuring scalability and reliability., Design and implement data pipelines that process large volumes of data in both streaming and batch modes., Optimize data workflows to ensure efficient data ingestion, processing, ...
... clients. Depending on the project, your work may include: designing, building and optimizing data pipelines in GCP and/or Azure; working with BigQuery, Snowflake or Databricks; using Python or JVM languages such as Java or Scala; working with SQL in complex data environments; developing scalable batch and streaming data processing ...
... Science, Economics, or engineering or equivalent experience · Results oriented with strong analytical and problem-solving skills, with demonstrated intellectual and analytical rigor · Hands-on experience with one or more data analytics/programming tools such as R/Python · Hands-on experience with SQL databases · Experience ...
... audience recommendations using the Teads Data Suite. - Training: Work with Global Data Teams to create and deliver training programs for all new data products and solutions built by Teads. - Product Feedback: Provide actionable feedback to the Teads Product Management and Engineering departments regarding future data needs ...
... into effective data modelling https://jobeax.com/link/nGdefD578X04jN8a data profiling and analysis on relational and hierarchical data sources, including JSON and XML https://jobeax.com/link/d879mD0gLNBrNwo5 closely with data architects and engineers to ensure models support data transformation and high-quality data https://jobeax.com/link/y4uaV3zu22twrDv0 ...
... Kernel, and LlamaIndexAI / ML Operations: Proven experience with MLOps / LLMOps pipelines, evaluation frameworks, feedback loops, vector databases, and embedding pipelinesData Platforms: Hands-on familiarity with Databricks and Snowflake, ETL/ELT pipelines (both batch and streaming processing), data modeling, and semantic layersAnalytics ...
... Kernel, and LlamaIndexAI / ML Operations: Proven experience with MLOps / LLMOps pipelines, evaluation frameworks, feedback loops, vector databases, and embedding pipelinesData Platforms: Hands-on familiarity with Databricks and Snowflake, ETL/ELT pipelines (both batch and streaming processing), data modeling, and semantic layersAnalytics ...
... Kernel, and LlamaIndexAI / ML Operations: Proven experience with MLOps / LLMOps pipelines, evaluation frameworks, feedback loops, vector databases, and embedding pipelinesData Platforms: Hands-on familiarity with Databricks and Snowflake, ETL/ELT pipelines (both batch and streaming processing), data modeling, and semantic layersAnalytics ...
... DevOps About the project The project migrates data and reporting from a legacy estate onto a modern platform built on Databricks and Microsoft Fabric. Quality engineering is central to the project. The business needs confidence that migrated data reconciles with the source, that pipelines behave as specified, and that reports ...
... depth in backend engineering. - Hands-on experience with microservices and micro-database architecture. - Experience with event-driven systems . - Expert in Java and Spring Framework (Boot/Data/REST/Security/Profiles). - Front-end experience with ReactJS and TypeScript (able to contribute meaningfully; backend-led). - API ...
... in ORM frameworks such as JPA and iBatisProficiency in security frameworks including Azure AD and OAuthKnowledge of Kafka, CosmosDB, PostgreSQL, and Redis for data streaming and storage architecturesFamiliarity with REST, Azure, Docker, Kubernetes, Maven, and GradleUnderstanding of microservices and data streaming architecturesBackground ...
... become fully data-driven. The project focuses on developing and maintaining Data Lake solutions, integrating multiple data sources, and delivering high-quality datasets for global analytics use cases. Your tasks Design, build, and optimize scalable data pipelines (batch & real-time) Develop and maintain ETL processes and ...
Twój zakres obowiązków Praca przy tworzeniu globalnej platformy analitycznej w firmie, z myślą o skalowalności i kosztach. Projektowanie i rozwój data pipeline'ów (batch, streaming) Tworzenie i utrzymywanie kontraktów danych, testów i metryk jakości (freshness, completeness, accuracy) Projektowanie algorytmów (deduplikacja, ...
... for high-volume accounting and data processing workflows. - Build a multi-tenant, cloud-based accounting platform capable of ingesting data in both real-time and batch modes. - Design, build, test, deploy, and operate highly scalable API-driven services, primarily using Java, with SQL and NoSQL databases, hosted in Microsoft ...
... feature engineering, unstructured data, and GenAI use cases. Your Responsibilities Would Include Data Platform & Services Engineering - contributing to the design and implementation of scalable data pipelines, ingestion frameworks, and processing engines for batch, streaming, and event-driven data - architecting and maintaining ...
... processes for data pipelines and applications. Proficiency in Python and PySpark for data processing, transformation, and analysis. Demonstrated experience in using Databricks for data engineering tasks, including working with Delta Lake and Spark SQL. We offer P&G-sized projects and access to world leading IT partners and technologies ...
... Programme Level: Senior / Mid-Senior Main Responsibilities: Develop and maintain data pipelines using Azure Databricks. Implement ETL processes, ensuring data quality and integrity. Read and comprehend complex SQL stored procedures. Translate architecture direction into engineering delivery. Conduct data profiling and quality assessments. ...
... połączeń z systemami poza ekosystem CRM • implementacja mechanizmów kontroli jakości oraz walidacji danych Wymagania • Minimum 4-6 lat doświadczenia na stanowisku Data Engineer, preferencyjnie w sektorze o dużej skali danych • doświadczenie w projektowaniu i modelowaniu struktur danych (hurtownie danych, DataLake-ów) na potrzeby ...