QA Automation and AI Lead

We are looking for a hands-on QA Automation & AI Lead to build the next generation of AI-driven quality engineering for our web applications.

This is not a traditional QA Lead role. We are looking for an engineer who deeply understands QA, is strong in automation, and has practical experience building and testing LLM-powered products.

What You’ll Do

• Build AI-driven frontend test automation, primarily using Playwright + LLMs, to test real user journeys.
• Design frameworks where AI can help generate test cases, execute tests, validate outcomes, and analyze failures.
• Build evaluation frameworks for LLM products, including prompt evaluation, hallucination testing, regression testing, and output quality.
• Make intelligent decisions about which model to use where, balancing accuracy, latency, context size, and cost.
• Optimize token consumption for frontend automation instead of sending entire pages/context to expensive models unnecessarily.
• Identify where to use deterministic automation vs. LLM-based automation.
• Explore AI-assisted approaches to make manual and exploratory testing faster and more effective.
• Remain highly hands-on while providing technical direction and guiding engineers implementing the automation platform.

What We’re Looking For

• Strong software engineering and test automation experience.
• Hands-on experience with LLMs and AI-powered applications.
• Strong experience with Playwright or similar browser automation frameworks.
• Experience with prompt/LLM evaluation, hallucination testing, and non-deterministic systems.
• Good understanding of token optimization, context management, model selection, latency, and LLM costs.
• Strong understanding of QA principles, test strategy, regression testing, and edge cases.
• Ability to prototype quickly and turn experiments into scalable automation frameworks.
• Strong technical leadership skills while remaining an individual contributor.

What Success Looks Like

You will help us move from traditional QA toward AI-native quality engineering—where AI assists in creating and validating test scenarios, frontend journeys are intelligently automated, LLM products are continuously evaluated, and model/token usage is optimized for both quality and cost.

We’re looking for someone who thinks beyond “How do I automate this test?” and asks “What is the smartest combination of deterministic automation and AI to test this product?”

Gross salary 14,000 - 21,000 PLN