Test Engineer Automation (Python with RAG testing experience)

We invite an experienced and proactive Test Automation Engineer with strong Python and manual testing skills to join our team. In this role, you’ll design and implement robust testing solutions for cutting-edge AI products, focusing on improving RAG flows and testing LLM integrations to ensure reliability, safety, and performance.

🟩 What's in it for you:

• Work hands-on on test automation for AI-powered solutions that have real-world impact
• Grow your expertise in AI safety and RAG/LLM testing
• Collaborate with a team of experienced engineers and QA professionals in a modern, innovation-focused environment

✅ Is that you?

• 3+ years of experience in automation testing
• Strong programming skills in Python
• Practical experience with pytest or similar Python testing frameworks
• Experience developing and maintaining test automation frameworks
• Solid understanding of test design, strategy, and QA best practices
• Hands-on experience with CI/CD pipelines and tools (Jenkins, GitLab CI)
• Understanding of RAG pipelines and testing AI/ML-based systems
• Familiarity with AI safety testing (prompt validation, hallucination checks, response consistency)
• Familiarity with API (REST, HTTP) testing and data validation
• Good knowledge of Git, test reporting, and bug tracking tools
• Strong analytical and problem-solving skills
• Upper-Intermediate English level

Desirable:

• Experience with major cloud platforms (GCP, AWS, Azure)
• Knowledge of containerization tools (Docker, Kubernetes)

🧩Key responsibilities and your contribution

In this role, you'll develop and execute automated tests to ensure the quality and reliability of AI-driven products.

• Design, develop, and maintain automation frameworks and test strategies for AI products
• Evaluate and improve RAG flows, ensuring accuracy and performance
• Test LLM integrations with prompt validation, hallucination checks, and response consistency checks
• Prepare test data and clarify testing requirements as needed
• Create and run automated test suites for APIs, data pipelines, and model integrations
• Integrate automated tests into CI/CD pipelines to maintain quality and stability
• Analyze test results, report defects, and work with teams to resolve issues
• Participate in code reviews, sprint ceremonies, and QA planning sessions
• Maintain clear test documentation and provide regular updates to stakeholders
• Continuously improve testing tools, processes, and automation efficiency