Responsibilities

Evaluate AI-generated coding interactions end to end.
Judge whether outputs are useful, correct at a high level, and aligned with strong engineering practice.
Assess the quality of explanations, reasoning, and guidance provided by coding agents.
Distinguish between different levels of response quality and help define what excellent interactions look like.
Provide clear, opinionated written feedback and record short video explanations.
Complete a take-home evaluation exercise with a recorded Loom walkthrough.

Requirements

Senior, Staff, or Principal-level software engineer, or equivalent experience.
Strong background in TypeScript, JavaScript, or Python.
Hands-on experience with at least one of OpenAI Codex, Claude Code, or Cursor.
Deep familiarity with modern AI-assisted development workflows.
Ability to evaluate code without executing it or reviewing every line.
Strong written and spoken English at B2 level or above.
Comfort giving direct, opinionated feedback and applying a high bar for engineering quality.

Remote worldwide contract engagement.
Flexible schedule of 10–20 hours per week.
Ongoing work with projects lasting approximately two weeks to a few months.
New projects may be offered to successful evaluators.
Start as soon as the take-home exercise is cleared and a project seat is available.
No interview; the process consists of one take-home evaluation with a recorded Loom walkthrough.