MLOps Engineer (LLM Infrastructure)

Senior MLOps Engineer with 7+ years of experience needed for an AI-powered construction software company.
 

Responsibilities:

• Design, build, and maintain infrastructure for deploying, serving, and monitoring machine learning models at scale
• Deploy and scale large language models (7B-376B parameters) across varied sizes, optimizing for latency, throughput, and cost
• Operate and enhance a multi-provider gateway for routing requests between self-hosted models and external APIs
• Develop and manage automated pipelines for model training, fine-tuning, evaluation, and continuous delivery
• Monitor production ML services, implement observability, and respond to incidents
• Implement evaluation and verification systems to detect quality and performance regressions
• Collaborate with cross-functional teams to integrate ML models into production environments
• Deliver infrastructure-as-code and CI/CD workflows for repeatable, secure deployments
• Optimize model and infrastructure efficiency through quantization, pruning, and distributed inference
• Provide self-serve tooling and documentation for internal engineering teams
   

Must-have:

• Deep expertise in LLM deployment and scaling (7B to 376B parameters)
• Strong Python proficiency, with C/C++ experience for performance-critical components
• Hands-on experience with inference serving engines (vLLM, Triton, TGI)
• Advanced knowledge of distributed training frameworks (DeepSpeed, FSDP, Accelerate)
• Proficiency with MLOps platforms and pipelines (MLflow, Kubeflow, SageMaker Pipelines)
• Cloud infrastructure and orchestration (AWS, Docker, Kubernetes, EKS, EC2, Lambda)
• GPU/cloud resource optimization and cost efficiency management
• Monitoring, logging, alerting, and incident response for ML services
• Experience with quantization, pruning, and multi-GPU/distributed inference
• Experience building or operating multi-provider model gateways

Conditions:

• B2B Freelance contract
• Full Remote
• 12+ months duration
• ASAP start date

Project:
AI-powered construction software startup. Pioneer in artificial intelligence focused on developing secure, enterprise-grade solutions bridging advanced AI research with real-world applications.