Senior Software Engineer ( Quality and AI)

We need a senior engineer who owns the question of whether our AI features actually work. About 70 percent of the job is quality and evaluation in agent-based development: harnesses, datasets, regression gates, and the standards around them. The other 30 percent is raising the quality of everyone else’s product work. You review code across the Node, React, and React Native apps, coach engineers on resilient design, and shape features before they are built. You write production code too, but to keep your judgment current, not as your primary output.

That stack is Node services, React and React Native front ends, PostgreSQL, and AWS. Playwright for web, Maestro for mobile, GitHub Actions for CI.

If you want a pure test automation role, this is not it. If you want to be the person who defines what good means on a team building with coding agents, and who has the credibility to hold that line in review, keep reading.

What you will own

Quality and evals, about 70 percent of the work

• Eval harnesses, golden datasets, and scoring for production agentic workflows.
• Regression gates in CI. Nothing merges without an eval verdict.
• The Playwright web and Maestro mobile end-to-end suites. Fast, and honest about failures.
• Engineering standards: code review practice, architectural patterns, targeted refactors.
• Production instrumentation for AI features, and the loop that turns real failures into new eval cases.

Product craft and resilience, about 30 percent of the work

• Code review across the Node, React, and React Native codebases. Not rubber stamps. Specific feedback that leaves the author better than you found them.
• Coaching on resilient design: error handling, retries and idempotency, failure isolation, and the tests that prove the behavior.
• Quality designed into features up front. You are in feature design early, not reviewing at the end.
• Light hands-on contribution to production code, enough to keep your review credibility and your context current.
• Game day. When something breaks on a Saturday, you are in the troubleshooting: reading traces, reproducing the failure, and driving it down.

Requirements

• 5+ years building and operating production web software.
• Strong JavaScript and TypeScript. You know the runtime, not just the framework.
• Node and React in production. React Native shipped to the app stores is a strong plus.
• PostgreSQL: schema design for read patterns, index design, query plans, migration under load.
• AWS in production: ECS or Lambda, Secrets Manager, CloudWatch, IAM. You can read the bill.
• Playwright or equivalent, used on a suite you personally had to keep green.
• CI/CD work in GitHub Actions, including making a slow pipeline fast.
• A track record of raising code quality: reviews, architectural patterns, refactors that landed.
• You have reviewed and mentored other engineers on production code. You can name specific practices that spread across a team because of you.
• Strong asynchronous written communication.

AI engineering and evaluation

A hard requirement, not a preference. Expect detailed questions.

• Daily work with frontier models and agentic coding tools such as Claude Code, Codex, or Cursor.
• Evaluations you built yourself for an LLM or agent system. What you scored, how you scored it, and what you did when the score moved.
• Dataset construction. You can explain how you sourced cases, handled labeling, and kept the set from going stale.
• Judgment about failure modes. You can name the ways an agentic workflow breaks that a unit test will never catch.
• A repeatable method for giving agents context: instruction files, tool definitions, MCP servers, subagent roles.
• Cost awareness. You track token spend and wall-clock time per unit of shipped work.

Nice to have: eval platforms such as LangSmith, Braintrust, or Langfuse; agent frameworks and SDKs; internal tooling and MCP servers; mobile end-to-end testing; multi-tenant SaaS; sports, scheduling, or field-services domains