Opis oferty
$2k USD
Poland, Katowice
Senior QA / ML Tester w Katowice, Polska is listed on Jobeax. Browse 110,000+ vacancies available.
We are looking for a Senior QA / ML Tester to join the AI Platform team and take ownership of quality assurance for the Agent Evaluation Framework built on AWS AgentCore. This role involves designing, implementing, and maintaining a functional test suite that validates the correctness of AI agent evaluation pipelines — covering on-demand integration testing, online sampling accuracy, and multi-evaluator execution — and delivering a feasibility assessment for non-AgentCore runtime evaluation scenarios. This is a hands-on, production-focused role at the intersection of software quality engineering and AI/ML system testing, operating within an Agile delivery team and contributing to the reliability of enterprise-grade agentic AI infrastructure. Responsibilities Design and implement a functional test suite for the AWS AgentCore Evaluation API using pytest, covering known-good / known-bad session pair validation, multi-evaluator execution correctness, and edge case handlingDevelop and maintain integration tests for on-demand evaluation mode, integrated into the CI/CD pipeline with automated execution on each buildValidate online mode sampling accuracy, design test scenarios, define acceptance criteria, and report deviations with reproducible evidenceConduct and document a feasibility assessment for non-AgentCore runtime evaluation: analyze alternative runtimes, define evaluation methodology, and deliver a structured findings reportTest OpenTelemetry trace-based evaluation inputs and validate ADOT trace ingestion, trace structure correctness, and evaluator input integrityCollaborate with platform engineers to clarify evaluation contracts, reproduce defects, and align on quality gatesMaintain test documentation, including test plans, test reports, defect logs, and evaluation feasibility artifacts in Confluence/JiraParticipate in Agile ceremonies, including sprint planning, daily standups, demos, and retrospectivesContribute to EngX practices such as code review of test scripts, CI/CD pipeline integration, and test coverage reporting Requirements 5+ years of production experience in QA automation or ML/AI system testingProven experience testing AI/LLM systems, including evaluation pipelines, model outputs, or agent behavior validationProficiency in Python test automation, including pytest, fixtures, parametrize, and mocking (https://jobeax.com/link/Vhs9R0o6sLJV4mN3, moto)Knowledge of AWS AgentCore Evaluation API, covering on-demand and online evaluation modesFamiliarity with OpenTelemetry / ADOT for trace-based evaluation input testing and trace structure validationSkills in REST API testing, including request/response validation and authentication (SigV4, bearer tokens)Experience with CI/CD integration using GitHub Actions, Jenkins, or equivalent, including test pipeline configurationBackground in test data management, including known-good / known-bad session pair design and synthetic trace generationExpertise in functional and integration test design for AI/ML evaluation pipelinesCompetency in defect lifecycle management, including Jira, reproducible bug reports, and root cause analysisUnderstanding of LLM/agent evaluation concepts, such as correctness scoring, sampling strategies, and evaluator chainingAbility to work independently after onboarding, manage own tasks, report status, and escalate blockers proactivelyStrong analytical skills to define test scenarios from ambiguous or evolving specificationsEnglish B2+ level, written and verbal, for daily collaboration with distributed teams Nice to have Hands-on experience with AWS AgentCore Evaluation API or AWS Bedrock testingExperience testing OpenTelemetry / distributed tracing pipelinesFamiliarity with multi-evaluator execution patterns and correctness validation strategiesExperience writing feasibility assessments or technical reports for stakeholdersKnowledge of agentic AI frameworks (LangGraph, Strands Agents) sufficient to understand evaluation contractsISTQB CT-AI certification or equivalent AI testing qualificationExperience with AI Ready / AI Practitioner practices at EPAM (prompt engineering, AI-assisted test design) We offer We gather like-minded people:Top tech minds driving innovation in AI, cloud and digital platform modernizationSupportive team and agile, startup-like cultureHybrid by design mode and opportunity to work remotely within PolandChance to work abroad for up to 60 days annuallyBusiness-driven relocation opportunitiesWe provide growth opportunities:Career development programsThought leadership, mentoring, soft skills and well-being programsCertification (Anthropic, Gemini, GCP, Azure, AWS)English classesWe cover it all:Stable payParticipation in the Employee Stock Purchase Plan with a 15% discountBenefits package (health insurance, multisport, shopping vouchers)Referral bonuses up to $2,000Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and moreCorporate, social and well-being eventsPlease, note:Benefits listed above are available to employees onlyWe are open for working with Contractors. Terms of B2B cooperation agreements are agreed individuallyWe will reach out to selected candidates exclusively EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.
Podobne oferty
Poszukujemy doświadczonego testera / doświadczonej testerki do projektu w obszarze bankowości. W tej roli będziesz odpowiadać za testowanie rozwiązań IT oraz zapewnienie funkcjonalnej zgodności oprogramowania z wymaganiami technicznymi i biznesowymi. Praca w modelu hybrydowym, 1 dzień w tygodniu z biura w Warszawie. Zadania: ...
Poland, Warszawa
Aktualnie dla klienta z bankowości poszukujemy Seniora Testów Manualnych, który dołączył by do zespołu zajmującego się testowaniem aplikacji bankowej oraz serwisu internetowego. Zakres obowiązków: Wykonywanie testów funkcjonalnych, regresyjnych oraz warstwy wizualnej manualnych aplikacji mobilnej oraz serwisu internetowego. ...
Poland, Lublin
$2k USD
We are seeking a Senior QA Automation Engineer to design and develop a scalable, AI-enabled test automation platform for web services and APIs. In this role, you will leverage modern test automation frameworks, JavaScript, Selenium, SQL, and cutting-edge AI capabilities to build intelligent automation workflows that enhance ...
Poland, Katowice
$2k USD
We are seeking a Senior QA Automation Engineer to design and develop a scalable, AI-enabled test automation platform for web services and APIs. In this role, you will leverage modern test automation frameworks, JavaScript, Selenium, SQL, and cutting-edge AI capabilities to build intelligent automation workflows that enhance ...
Poland, Warszawa
$2k USD
We are seeking a skilled Senior AI/ML Solution Engineer with expertise in AI Agentic Security Testing to join our client's security team and support key R&D activities. This project aims to enhance software security testing across the client's product ecosystem, ensuring the highest standards of protection for customer ...
Poland, Wrocław
Tenth Revolution Group
Tenth Revolution Group in Warsaw seeks an experienced AI/ML Software Engineer to join its Banking & Markets technology team. You will develop and advance production AI/ML systems and design, experiment with and evaluate ML-based applications. You will build high-quality software, libraries and frameworks, lead cross-team ...
Poland, Warszawa
Digital Forms
180 zł PLN
... CIO, who currently coordinates delivery directly, take over day-to-day technical ownership of the team, and become the primary technical point of contact for senior client stakeholders. What you'll own - Day-to-day technical leadership across multiple engineering squads (backend, frontend, AI/ML, data) plus QA and DevOps ...
Poland, Warszawa
3k
... PydanticAI and LangGraph. Nice to have Experience with information retrieval and search, including hybrid search, reranking, and vector databases. Experience with ML platforms and tools such as MLflow, AWS SageMaker/Bedrock, Azure ML, Snowflake, or Databricks. Experience with infrastructure as code and containers, using tools ...
Poland, Wrocław
We are looking for an experienced Senior Integration Test & Service Virtualization Engineer to join a team responsible for Service Virtualization, test automation, and performance testing within a complex enterprise environment. The role focuses on working with IBM Rational Integration Tester (RIT), testing integrations ...
Poland, Gdynia
21840 - 25200 zł
QA Lead – ETL Testing We are looking for an experienced QA Lead with strong expertise in ETL/Data Testing to lead end-to-end QA activities across data integration, data warehouse and analytics platforms. Key responsibilities: Define and manage test strategies, plans and release governance Lead ETL/ELT, Functional, Integration, ...
Poland, Warszawa
6800 zł PLN
... QA engineers, researching new technologies and QA solutions, creating proof of concepts (PoCs), and sharing findings with the organization, collaborating with QA Engineers and Architects across Xebia teams in the Netherlands, Switzerland, and India, identifying and driving initiatives to improve QA practices across the ...
Poland, Wrocław
... forward to. Send us your CV and… see you at Wakacje.pl! ✈️ About us: We are a growing team of over 100 IT specialists, including Developers, DevOps Engineers, Testers, Project Managers, Architects, and Analysts. We're driven by passion and engagement and prioritize a culture of open communication. A commitment to high code ...
Poland, Gdańsk
We are looking for a QA Engineer to join our Agile engineering team and help deliver high-quality software solutions through manual and automated testing. Key Responsibilities Design and execute test plans, test cases, and test scenarios. Perform manual, regression, exploratory, and end-to-end testing. Develop and maintain ...
Poland, Warszawa
Jobgether in Poland is seeking a Senior QA Engineer (Web) to own testing strategies, ensure reliability of business-critical web features from hand-off to production release, and drive quality across functional, UI/UX, and API scenarios. The role emphasizes collaboration with Product, Front-End, Back-End, and Architecture ...
Poland, Warząchewka Polska
Euroclear is seeking an experienced Test Lead to steer integration, system and end-to-end testing across business, functional and technical domains. You will join a vibrant squad spread across Belgium, France, Poland and India, driving quality in mission-critical services.
Poland, Warząchewka Polska
... podczas współpracy. W związku z ciągłym rozwojem i dużą liczbą projektów rekrutacyjnych, jakie prowadzimy dla naszych Partnerów, szukamy osoby na stanowisko: Tester Manualny / Testerka Manualna ,[Analiza wymagań biznesowych i technicznych oraz przygotowywanie scenariuszy i przypadków testowych, Planowanie i realizacja testów ...
Poland, Warszawa
$2k USD
We are seeking an experienced QA Team Lead to join our team and drive quality across our exchange connectivity and algorithmic trading platforms. This role requires strong proficiency in both manual and automated testing, with an even 50/50 split between the two. This is a hybrid role, with three days per week working remotely. ...
Poland, Wrocław
We are looking for an experienced QA Automation Engineer to ensure the quality and reliability of core English Language Learning platforms. This role is hands‑on and delivery‑driven. You will be responsible for designing, implementing, and maintaining automated test solutions across frontend and backend systems, supporting ...
Poland, Kraków
80 zł PLN
Ta oferta jest dla Ciebie, jeśli: - posiadasz co najmniej 3+ lata doświadczenia na stanowisku Tester Manualny lub podobnym, - masz wykształcenie wyższe (preferowane kierunki: informatyka, kierunki ścisłe), - posiadasz doświadczenie w pracy w sektorze bankowym/ ubezpieczeń/ telekomunikacji, - bardzo dobrze znasz takie technologie ...
Poland, Warszawa
11760 - 15120 zł
Poszukujemy Testera Automatyzującego do projektu związanego z branżą ochrony zdrowia. Lokalizacja: 100% zdalnie Stawka: 70-90 PLN/h netto na B2B Wymagania: Minimum 5 lat doświadczenia w testowaniu oprogramowania; Minimum 2 lata doświadczenia w automatyzacji testów w narzędziu Selenium WebDriver lub pokrewnym; Znajomość ...
Poland, Warszawa