Opis oferty
$2k USD
Poland, Katowice
Senior QA / ML Tester w Katowice, Polska is listed on Jobeax. Browse 110,000+ vacancies available.
We are looking for a Senior QA / ML Tester to join the AI Platform team and take ownership of quality assurance for the Agent Evaluation Framework built on AWS AgentCore. This role involves designing, implementing, and maintaining a functional test suite that validates the correctness of AI agent evaluation pipelines — covering on-demand integration testing, online sampling accuracy, and multi-evaluator execution — and delivering a feasibility assessment for non-AgentCore runtime evaluation scenarios. This is a hands-on, production-focused role at the intersection of software quality engineering and AI/ML system testing, operating within an Agile delivery team and contributing to the reliability of enterprise-grade agentic AI infrastructure. Responsibilities Design and implement a functional test suite for the AWS AgentCore Evaluation API using pytest, covering known-good / known-bad session pair validation, multi-evaluator execution correctness, and edge case handlingDevelop and maintain integration tests for on-demand evaluation mode, integrated into the CI/CD pipeline with automated execution on each buildValidate online mode sampling accuracy, design test scenarios, define acceptance criteria, and report deviations with reproducible evidenceConduct and document a feasibility assessment for non-AgentCore runtime evaluation: analyze alternative runtimes, define evaluation methodology, and deliver a structured findings reportTest OpenTelemetry trace-based evaluation inputs and validate ADOT trace ingestion, trace structure correctness, and evaluator input integrityCollaborate with platform engineers to clarify evaluation contracts, reproduce defects, and align on quality gatesMaintain test documentation, including test plans, test reports, defect logs, and evaluation feasibility artifacts in Confluence/JiraParticipate in Agile ceremonies, including sprint planning, daily standups, demos, and retrospectivesContribute to EngX practices such as code review of test scripts, CI/CD pipeline integration, and test coverage reporting Requirements 5+ years of production experience in QA automation or ML/AI system testingProven experience testing AI/LLM systems, including evaluation pipelines, model outputs, or agent behavior validationProficiency in Python test automation, including pytest, fixtures, parametrize, and mocking (https://jobeax.com/link/Vhs9R0o6sLJV4mN3, moto)Knowledge of AWS AgentCore Evaluation API, covering on-demand and online evaluation modesFamiliarity with OpenTelemetry / ADOT for trace-based evaluation input testing and trace structure validationSkills in REST API testing, including request/response validation and authentication (SigV4, bearer tokens)Experience with CI/CD integration using GitHub Actions, Jenkins, or equivalent, including test pipeline configurationBackground in test data management, including known-good / known-bad session pair design and synthetic trace generationExpertise in functional and integration test design for AI/ML evaluation pipelinesCompetency in defect lifecycle management, including Jira, reproducible bug reports, and root cause analysisUnderstanding of LLM/agent evaluation concepts, such as correctness scoring, sampling strategies, and evaluator chainingAbility to work independently after onboarding, manage own tasks, report status, and escalate blockers proactivelyStrong analytical skills to define test scenarios from ambiguous or evolving specificationsEnglish B2+ level, written and verbal, for daily collaboration with distributed teams Nice to have Hands-on experience with AWS AgentCore Evaluation API or AWS Bedrock testingExperience testing OpenTelemetry / distributed tracing pipelinesFamiliarity with multi-evaluator execution patterns and correctness validation strategiesExperience writing feasibility assessments or technical reports for stakeholdersKnowledge of agentic AI frameworks (LangGraph, Strands Agents) sufficient to understand evaluation contractsISTQB CT-AI certification or equivalent AI testing qualificationExperience with AI Ready / AI Practitioner practices at EPAM (prompt engineering, AI-assisted test design) We offer We gather like-minded people:Top tech minds driving innovation in AI, cloud and digital platform modernizationSupportive team and agile, startup-like cultureHybrid by design mode and opportunity to work remotely within PolandChance to work abroad for up to 60 days annuallyBusiness-driven relocation opportunitiesWe provide growth opportunities:Career development programsThought leadership, mentoring, soft skills and well-being programsCertification (Anthropic, Gemini, GCP, Azure, AWS)English classesWe cover it all:Stable payParticipation in the Employee Stock Purchase Plan with a 15% discountBenefits package (health insurance, multisport, shopping vouchers)Referral bonuses up to $2,000Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and moreCorporate, social and well-being eventsPlease, note:Benefits listed above are available to employees onlyWe are open for working with Contractors. Terms of B2B cooperation agreements are agreed individuallyWe will reach out to selected candidates exclusively EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.
Podobne oferty
$2k USD
We are looking for a Senior QA / ML Tester to join the AI Platform team and take ownership of quality assurance for the Agent Evaluation Framework built on AWS AgentCore. This role involves designing, implementing, and maintaining a functional test suite that validates the correctness of AI agent evaluation pipelines — ...
Poland, Wrocław
SCALO
13600 - 16800 zł
W Scalo zajmujemy się dostarczaniem projektów software'owych i wspieraniem naszych partnerów w rozwijaniu ich biznesu. Tworzymy oprogramowanie, które umożliwia ludziom dokonywanie zmian, działanie w szybszym tempie oraz osiąganie lepszych rezultatów.
Poland, Warszawa
20160 - 21840 zł
W Scalo zajmujemy się dostarczaniem projektów software'owych i wspieraniem naszych partnerów w rozwijaniu ich biznesu. Tworzymy oprogramowanie, które umożliwia ludziom dokonywanie zmian, działanie w szybszym tempie oraz osiąganie lepszych rezultatów.
Poland, Warszawa
Internetwork Expert
... are probably familiar with our best-known product, the hCaptcha security suite. Our approach is simple: low overhead, small teams, and rapid iteration. As a Senior ML Ops Engineer, you will help shape and expand the pipelines that power our products and research efforts. You’ll work across teams to design, maintain, and ...
Poland, Warszawa
120 zł PLN - 150 zł PLN
Select how often (in days) to receive an alert: Senior QA Consultant Date: Aug 20, 2026 Location: Poznań, WP, PL, 61-569 Łódź, PL, 90-118 Working place: Hybrid Working Location: Type of contract: B2Bcontract Salary range: 120-150 PLN/H What will you do? As a Senior QA Consultant, you will be responsible for designing and ...
Poland, Poznań
Introduction & Summary We are seeking a Senior QA tester who has a strong focus on functional and automated testing. Main Responsibilities The primary responsibility involves quality assurance across legacy applications, ensuring both functional and performance metrics align with expectations. Key tasks include: Monitoring ...
Poland
Capgemini
420k
... what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world. Your role Senior Penetration Tester plays a key role in securing complex client environments by leading high-impact security testing activities and guiding small delivery teams. ...
Poland, Gdańsk
18480 - 19320 zł
... provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for: Senior Manual QA Engineer Responsibilities: Ensure QA activities are conducted efficiently and documented according to the latest Quality Standards Analyze and design ...
Poland, Gdańsk
snowflake
At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work.
Poland, Warszawa
13000 - 25000 zł
We’re currently seeking a passionate Senior QA Engineer for our mobile and IoT projects for a freelance cooperation. At the moment, we have a few open positions for our ongoing projects, which are heavily focused on IoT and mobile. As a freelancer working with intent, you'll have the chance to contribute your expertise ...
Poland, Warszawa
30333 - 31083 zł
Operating system, Windows About the project, Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search.
Poland, Warszawa
Diverse CG Sp. z o.o. Sp.k.
DCG seeks an experienced ML Engineer to manage the lifecycle of ML models from development to deployment and monitoring. You will implement MLOps, work with Spark and Python for data ingestion, and build distributed pipelines for massive datasets. The role involves collaboration with data scientists, DevOps, and IT to deliver ...
Poland, Warszawa
11400 - 18480 zł
... analitycznymi i biznesowymi w celu zapewnienia wysokiej jakości dostarczanych rozwiązań Oczekiwane kompetencje: Minimum 5 lat doświadczenia na stanowisku Manual Testera/QA Engineera Bardzo dobra znajomość testowania aplikacji desktopowych Doświadczenie w testowaniu sprzętu (hardware) oraz integracji pomiędzy oprogramowaniem ...
Poland, Gdańsk
... production Verifying integrations with external systems through REST APIs and messaging interfaces Ensuring high-quality documentation of defects, test scenarios and QA processes Supporting continuous improvement of QA processes and automated testing practices Requirements: Senior-level commercial experience in software testing ...
Poland, Wrocław
Via Licensing Corporation
Join the leader in entertainment innovation and help us design the future. At Dolby, science meets art, and high tech means more than computer code. As a member of the Dolby team, you’ll see and hear the results of your work everywhere, from movie theaters to smartphones.
Poland, Wrocław
ERGO Technology & Services
... and strong global partnerships, we foster a dynamic, multicultural environment that promotes diversity and international opportunities. About the role As a Senior QA Engineer, you will be responsible for ensuring the quality, reliability, and performance of Data Hubs and data processing pipelines in Azure & Databricks. ...
Poland, Warszawa
Aviva plc is seeking an experienced Data Scientist to join the Services Excellence Centre in Warsaw. You will develop and deploy Data Science and ML models, work with Python, R, and SQL, and present findings to business stakeholders. We value a 5+ year track record, strong English, and experience in cloud-based environments. ...
Poland, Warszawa
Poszukujemy doświadczonego testera / doświadczonej testerki do projektu w obszarze bankowości. W tej roli będziesz odpowiadać za testowanie rozwiązań IT oraz zapewnienie funkcjonalnej zgodności oprogramowania z wymaganiami technicznymi i biznesowymi. Praca w modelu hybrydowym, 1 dzień w tygodniu z biura w Warszawie. Zadania: ...
Poland, Warszawa
$2k USD
We are seeking a Senior QA Automation Engineer to design and develop a scalable, AI-enabled test automation platform for web services and APIs. In this role, you will leverage modern test automation frameworks, JavaScript, Selenium, SQL, and cutting-edge AI capabilities to build intelligent automation workflows that enhance ...
Poland, Warszawa