Opis oferty
$2k USD
Poland, Warszawa
Senior QA / ML Tester w Warszawa, Polska is listed on Jobeax. Browse 110,000+ vacancies available.
We are looking for a Senior QA / ML Tester to join the AI Platform team and take ownership of quality assurance for the Agent Evaluation Framework built on AWS AgentCore. This role involves designing, implementing, and maintaining a functional test suite that validates the correctness of AI agent evaluation pipelines — covering on-demand integration testing, online sampling accuracy, and multi-evaluator execution — and delivering a feasibility assessment for non-AgentCore runtime evaluation scenarios. This is a hands-on, production-focused role at the intersection of software quality engineering and AI/ML system testing, operating within an Agile delivery team and contributing to the reliability of enterprise-grade agentic AI infrastructure. Responsibilities Design and implement a functional test suite for the AWS AgentCore Evaluation API using pytest, covering known-good / known-bad session pair validation, multi-evaluator execution correctness, and edge case handlingDevelop and maintain integration tests for on-demand evaluation mode, integrated into the CI/CD pipeline with automated execution on each buildValidate online mode sampling accuracy, design test scenarios, define acceptance criteria, and report deviations with reproducible evidenceConduct and document a feasibility assessment for non-AgentCore runtime evaluation: analyze alternative runtimes, define evaluation methodology, and deliver a structured findings reportTest OpenTelemetry trace-based evaluation inputs and validate ADOT trace ingestion, trace structure correctness, and evaluator input integrityCollaborate with platform engineers to clarify evaluation contracts, reproduce defects, and align on quality gatesMaintain test documentation, including test plans, test reports, defect logs, and evaluation feasibility artifacts in Confluence/JiraParticipate in Agile ceremonies, including sprint planning, daily standups, demos, and retrospectivesContribute to EngX practices such as code review of test scripts, CI/CD pipeline integration, and test coverage reporting Requirements 5+ years of production experience in QA automation or ML/AI system testingProven experience testing AI/LLM systems, including evaluation pipelines, model outputs, or agent behavior validationProficiency in Python test automation, including pytest, fixtures, parametrize, and mocking (https://jobeax.com/link/Vhs9R0o6sLJV4mN3, moto)Knowledge of AWS AgentCore Evaluation API, covering on-demand and online evaluation modesFamiliarity with OpenTelemetry / ADOT for trace-based evaluation input testing and trace structure validationSkills in REST API testing, including request/response validation and authentication (SigV4, bearer tokens)Experience with CI/CD integration using GitHub Actions, Jenkins, or equivalent, including test pipeline configurationBackground in test data management, including known-good / known-bad session pair design and synthetic trace generationExpertise in functional and integration test design for AI/ML evaluation pipelinesCompetency in defect lifecycle management, including Jira, reproducible bug reports, and root cause analysisUnderstanding of LLM/agent evaluation concepts, such as correctness scoring, sampling strategies, and evaluator chainingAbility to work independently after onboarding, manage own tasks, report status, and escalate blockers proactivelyStrong analytical skills to define test scenarios from ambiguous or evolving specificationsEnglish B2+ level, written and verbal, for daily collaboration with distributed teams Nice to have Hands-on experience with AWS AgentCore Evaluation API or AWS Bedrock testingExperience testing OpenTelemetry / distributed tracing pipelinesFamiliarity with multi-evaluator execution patterns and correctness validation strategiesExperience writing feasibility assessments or technical reports for stakeholdersKnowledge of agentic AI frameworks (LangGraph, Strands Agents) sufficient to understand evaluation contractsISTQB CT-AI certification or equivalent AI testing qualificationExperience with AI Ready / AI Practitioner practices at EPAM (prompt engineering, AI-assisted test design) We offer We gather like-minded people:Top tech minds driving innovation in AI, cloud and digital platform modernizationSupportive team and agile, startup-like cultureHybrid by design mode and opportunity to work remotely within PolandChance to work abroad for up to 60 days annuallyBusiness-driven relocation opportunitiesWe provide growth opportunities:Career development programsThought leadership, mentoring, soft skills and well-being programsCertification (Anthropic, Gemini, GCP, Azure, AWS)English classesWe cover it all:Stable payParticipation in the Employee Stock Purchase Plan with a 15% discountBenefits package (health insurance, multisport, shopping vouchers)Referral bonuses up to $2,000Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and moreCorporate, social and well-being eventsPlease, note:Benefits listed above are available to employees onlyWe are open for working with Contractors. Terms of B2B cooperation agreements are agreed individuallyWe will reach out to selected candidates exclusively EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.
Podobne oferty
$2k USD
We are looking for a Senior QA / ML Tester to join the AI Platform team and take ownership of quality assurance for the Agent Evaluation Framework built on AWS AgentCore. This role involves designing, implementing, and maintaining a functional test suite that validates the correctness of AI agent evaluation pipelines — ...
Poland, Wrocław
SCALO
13600 - 16800 zł
W Scalo zajmujemy się dostarczaniem projektów software'owych i wspieraniem naszych partnerów w rozwijaniu ich biznesu. Tworzymy oprogramowanie, które umożliwia ludziom dokonywanie zmian, działanie w szybszym tempie oraz osiąganie lepszych rezultatów.
Poland, Warszawa
20160 - 21840 zł
W Scalo zajmujemy się dostarczaniem projektów software'owych i wspieraniem naszych partnerów w rozwijaniu ich biznesu. Tworzymy oprogramowanie, które umożliwia ludziom dokonywanie zmian, działanie w szybszym tempie oraz osiąganie lepszych rezultatów.
Poland, Warszawa
Internetwork Expert
... are probably familiar with our best-known product, the hCaptcha security suite. Our approach is simple: low overhead, small teams, and rapid iteration. As a Senior ML Ops Engineer, you will help shape and expand the pipelines that power our products and research efforts. You’ll work across teams to design, maintain, and ...
Poland, Warszawa
Introduction & Summary We are seeking a Senior QA tester who has a strong focus on functional and automated testing. Main Responsibilities The primary responsibility involves quality assurance across legacy applications, ensuring both functional and performance metrics align with expectations. Key tasks include: Monitoring ...
Poland
120 zł PLN - 150 zł PLN
Select how often (in days) to receive an alert: Senior QA Consultant Date: Aug 20, 2026 Location: Poznań, WP, PL, 61-569 Łódź, PL, 90-118 Working place: Hybrid Working Location: Type of contract: B2Bcontract Salary range: 120-150 PLN/H What will you do? As a Senior QA Consultant, you will be responsible for designing and ...
Poland, Poznań
Capgemini
420k
... what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world. Your role Senior Penetration Tester plays a key role in securing complex client environments by leading high-impact security testing activities and guiding small delivery teams. ...
Poland, Gdańsk
18480 - 19320 zł
... provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for: Senior Manual QA Engineer Responsibilities: Ensure QA activities are conducted efficiently and documented according to the latest Quality Standards Analyze and design ...
Poland, Gdańsk
$2k USD
We are looking for a Senior QA Automation Engineer to join an AI-enabled test automation platform initiative for the client. In this role, you will design and develop a scalable test automation platform for web services and APIs, build reusable automation solutions, integrate CI/CD pipelines, and implement AI-powered capabilities ...
Poland, Kraków
30333 - 31083 zł
Operating system, Windows About the project, Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search.
Poland, Warszawa
Diverse CG Sp. z o.o. Sp.k.
DCG seeks an experienced ML Engineer to manage the lifecycle of ML models from development to deployment and monitoring. You will implement MLOps, work with Spark and Python for data ingestion, and build distributed pipelines for massive datasets. The role involves collaboration with data scientists, DevOps, and IT to deliver ...
Poland, Warszawa
$2k USD
We are looking for a Senior QA / ML Tester to join the AI Platform team and take ownership of quality assurance for the Agent Evaluation Framework built on AWS AgentCore. This role involves designing, implementing, and maintaining a functional test suite that validates the correctness of AI agent evaluation pipelines — ...
Poland, Katowice
11400 - 18480 zł
... analitycznymi i biznesowymi w celu zapewnienia wysokiej jakości dostarczanych rozwiązań Oczekiwane kompetencje: Minimum 5 lat doświadczenia na stanowisku Manual Testera/QA Engineera Bardzo dobra znajomość testowania aplikacji desktopowych Doświadczenie w testowaniu sprzętu (hardware) oraz integracji pomiędzy oprogramowaniem ...
Poland, Gdańsk
... projekcie Dołączysz do zespołu rozwijającego nowoczesne rozwiązania reklamowe dla jednego z największych marketplace'ów w Europie. Poszukujemy doświadczonego Senior QA Engineera, który wesprze rozwój produktów z obszaru Advertising i będzie miał realny wpływ na jakość rozwiązań wykorzystywanych przez miliony użytkowników. ...
Poland, Warszawa
ERGO Technology & Services
... and strong global partnerships, we foster a dynamic, multicultural environment that promotes diversity and international opportunities. About the role As a Senior QA Engineer, you will be responsible for ensuring the quality, reliability, and performance of Data Hubs and data processing pipelines in Azure & Databricks. ...
Poland, Warszawa
Via Licensing Corporation
Join the leader in entertainment innovation and help us design the future. At Dolby, science meets art, and high tech means more than computer code. As a member of the Dolby team, you’ll see and hear the results of your work everywhere, from movie theaters to smartphones.
Poland, Wrocław
... production Verifying integrations with external systems through REST APIs and messaging interfaces Ensuring high-quality documentation of defects, test scenarios and QA processes Supporting continuous improvement of QA processes and automated testing practices Requirements: Senior-level commercial experience in software testing ...
Poland, Wrocław
ComeOn Group is seeking a Data Engineer to design, build, and maintain near real-time and batch data flows powering production ML models. You will work with Data Scientists to create scalable pipelines in a hybrid work environment and help ensure data quality for model training and serving. The role requires 5+ years in ...
Poland, Katowice
Aviva plc is seeking an experienced Data Scientist to join the Services Excellence Centre in Warsaw. You will develop and deploy Data Science and ML models, work with Python, R, and SQL, and present findings to business stakeholders. We value a 5+ year track record, strong English, and experience in cloud-based environments. ...
Poland, Warszawa