Senior QA Engineer (AI Focus)

About the Project

We’re building a cutting-edge AI-powered video portal that aggregates and processes videos from various social media platforms. Our platform leverages advanced AI capabilities such as Retrieval-Augmented Generation (RAG), LLM agent orchestration, and contextual AI search.
Quality is mission-critical here — especially in validating AI behavior and results in real-world use cases.

You’ll be the driving force behind designing and executing our QA strategy, with a special focus on testing AI models, data pipelines, and user-facing AI features. You’ll work directly with the founder/CTO, backend engineers, and ML specialists to establish frameworks, processes, and tools to ensure the system is accurate, reliable, and scalable.

Tech Stack You’ll Work With

- Python / Node.js backend services

- AWS (Lambda, S3, Cognito, DynamoDB, OpenSearch)

- AI/ML stack including RAG pipelines, LLM orchestration, and contextual search

- Test automation tools (your choice — Cypress, Playwright, PyTest, Postman, etc.)

- CI/CD pipelines in a cloud-native, serverless environment

Your Responsibilities

- Define and implement the QA strategy for both traditional software components and AI/ML-driven features.

- Develop test plans and testing frameworks tailored to AI functionalities (e.g., search relevance, RAG accuracy, response quality).

- Collaborate with ML engineers/data scientists to design measurable evaluation metrics for AI model outputs.

- Build automated and manual test suites for:
API endpoints

AI query/response validation

Data ingestion & preprocessing pipelines

UI/UX flows (in collaboration with frontend QA)

- Create and maintain QA documentation, processes, and reporting templates.

- Analyze defects, identify root causes, and work with engineers to resolve them quickly.

- Recommend and integrate QA tools for AI and non-AI testing.

- Ensure all releases meet performance, accuracy, and usability benchmarks before going live.

Who We’re Looking For

- 5+ years of professional QA experience in software projects, including at least 1+ year in AI/ML-related testing.

- Strong background in both manual and automated testing.

- Experience defining test strategies for complex systems with AI components.

- Familiarity with ML evaluation metrics (precision, recall, relevance scoring, etc.).

- Proficiency with at least one test automation framework for backend/API testing.

- Understanding of cloud environments, preferably AWS.

- Excellent analytical, communication, and documentation skills.

- Self-starter mindset, able to design QA processes from scratch in a fast-paced startup.

Being a part of 3asoft means having:
- High level of flexibility and freedom
- p2p relationship with worldwide customers
- Competitive compensation paid in USD
- Fully remote working