Remote Data Engineer — AWS Data Lake & ETL Specialist… - Jobeax
Opis oferty
Remote Data Engineer — AWS Data Lake & ETL Specialist w Warszawa, Polska
ASTEK Polska Sp. z o.o.
ZdalniePraca z dowolnego miejsca
Warszawa, Polska
Astek Polska sp. z o.o. poszukuje Data Engineera do pracy nad projektami dla sektora farmaceutycznego w Warszawie. Rola obejmuje projektowanie i rozwijanie skalowalnych architektur danych, tworzenie hurtowni danych i Data Lake, a także implementację złożonych procesów ETL/ELT w środowisku AWS.
Wymagana jest praktyczna wiedza z zakresu Big Data, baz danych oraz optymalizacji wydajności systemów. Oferujemy długoterminową współpracę i pakiet benefitów.
... Prefect). - Exposure to CI/CD pipelines and DevOps practices. - Knowledge of streaming technologies (e.g., Kafka, Spark Streaming). - Experience working with Snowflake or Databricks in a production or development environment. - Relevant certifications in AWS, data engineering, or big data technologies. WHAT YOU’LL LOVE ABOUT ...
... typically touch:, •Data Warehouse / Data Lakehouse architectures on AWS alongside streaming and batch pipelines,, •Apache Iceberg as the open table format,, •Data Warehouses and query engines: DuckDB, Amazon Redshift, Amazon Athena, Amazon S3 Tables, Snowflake, ClickHouse, and other AWS-native data services,, •Airflow (AWS ...
... with real-time streaming data solutions (e.g., Kafka Streams, Kinesis). - Knowledge of machine learning pipelines and data science workflows. - Familiarity with data governance and compliance frameworks (e.g., GDPR, CCPA). - Certification in AWS, GCP, or Azure. - Academic background in Computer Science, Data Engineering, or ...
... wydajność i jakość inżynierską. - Współpraca z architektami, Product Ownerami, Business Analyst. Nasze wymagania - Min. 4-5 lat doświadczenia komercyjnego w roli Data Engineer, Cloud Data Engineer lub Data Platform Engineer. - Zaawansowana znajomość ekosystemu AWS, w szczególności: Lambda, DynamoDB, RDS, S3, Glue, Kinesis, ...
- 5+ years of experience in Python and strong software engineering skills - Solid knowledge of AWS cloud services and best practices - Experience building scalable Spark pipelines in PySpark or Scala - Practical experience with Spark Streaming for low-latency pipelines - Familiarity with Delta Lake and modern data lakehouse ...
... within AWS cloud environments using cloud-native patterns - Build and optimize ETL/ELT workflows to support analytical and operational use cases - Implement data modeling strategies aligned with modern data lakehouse architectures - Ensure data quality, governance, and monitoring across all data assets - Deliver secure, ...
... store, including historical backfill – Experience building pipelines feeding an LLM / retrieval system — strong plus – Experience handling sensitive personal data in a regulated or security-sensitive environment – Comfortable being the only data engineer on a small (2.5-FTE) pod, at part-time allocation, without hand-holding ...
Expected, Python, SQL, Microsoft Azure, ETL/ELT, CI/CD, Data Engineering Optional, LLM, RAG, Generative AI, Databricks Operating system, Windows, macOS About the project, As a Data Engineer, you will play a key role in the development of our AI-powered chatbot. Working primarily from a data engineering perspective, you ...
... your annual performance and company results. We are looking for a Junior Data Engineer to join our Conversational Analytics team. You will help us develop the engineering foundations around Looker Conversational Analytics (LCA) and Allegro Data Intelligence (ADI) our in-house conversational analytics solution. These tools ...
... Infrastructure as Code solutions., Integrating the data platform with other systems and applications., Participating in architectural decision-making and defining engineering standards., Conducting code reviews, documenting solutions, and sharing knowledge with team members.] Requirements: Python, ETL, ELT, Apache Spark, PySpark, ...
About you - 3+ years of hands-on experience in Data Engineering or Software Engineering. - Degree in computer science, engineering or equivalent technical field experience. - Proficient in cloud-based data processing and storage technologies like Databricks, Spark, Airflow. - Fluent in SQL and proficient in at least one ...