Lead AI Engineer

Description

We are building an AI-driven SDLC automation platform leveraging AWS Bedrock, knowledge bases, domain agents, and tool-based orchestration.
We are looking for a Lead AI Engineer to design and implement the AI architecture, including RAG pipelines, agent systems, tool-calling mechanisms (MCP-style architectures), model evaluation, and guardrails.
This role requires real production GenAI experience, not experimentation.

Requirements

• 8+ years software engineering experience
• 3+ years hands-on ML / AI engineering experience
• Strong Python (mandatory)
• Proven production experience with LLM systems

Deep understanding of:

– RAG architecture

– Embeddings

– Chunking strategies

– Retrieval optimization

• Experience with AWS Bedrock
• Experience designing tool-calling LLM systems (MCP or equivalent architecture)
• Experience integrating external APIs as agent tools
• Experience building REST services (FastAPI or similar)
• Experience with vector databases (OpenSearch, Pinecone, etc.)
• Experience implementing evaluation frameworks for LLM quality
• Experience mitigating hallucinations and prompt instability

Strong understanding of:

– Token economics

– Context window constraints

– Cost-performance tradeoffs

• Experience integrating with GitHub and JIRA APIs
• Production experience beyond notebooks or PoCs

Job responsibilities

• Architect and implement RAG pipelines on AWS Bedrock
• Design knowledge ingestion pipelines (JIRA, GitHub, Confluence, S3)
• Define chunking, embedding, and retrieval strategies
• Design vector storage and retrieval architecture
• Architect tool-calling agent systems (MCP or equivalent)

Define when to use:

– Direct API invocation

– Lambda-based tools

– MCP-based tool integration

– Design and implement structured output agents

• Implement hallucination mitigation strategies
• Build evaluation pipelines for model quality and regression testing
• Optimize token usage and latency
• Define model routing strategies (cost vs quality)
• Implement guardrails and structured validation
• Work closely with DevOps to productionize AI services
• Mentor AI engineers and define AI engineering standards
• Define long-term GenAI roadmap and architecture patterns