Machine Learning Engineer

Responsibilities

Model Fine-Tuning and Deployment:

Fine-tune pre-trained models (e.g., BERT, GPT) for specific tasks and deploy them using Amazon SageMaker and Bedrock.

RAG Workflows:

Establish Retrieval-Augmented Generation (RAG) workflows that leverage knowledge bases built on Kendra or OpenSearch. This includes integrating various data sources, such as corporate documents, inspection checklists, and real-time external data feeds.

MLOps Integration:

The project includes a comprehensive MLOps framework to manage the end-to-end lifecycle of machine learning models. This includes continuous integration and delivery (CI/CD) pipelines for model training, versioning, deployment, and monitoring. Automated workflows ensure that models are kept up-to-date with the latest data and are optimized for performance in production environments.

Scalable and Customizable Solutions:

Ensure that both the template and ingestion pipelines are scalable, allowing for adjustments to meet specific customer needs and environments. This involves setting up RAG workflows, knowledge bases using Kendra/OpenSearch, and seamless integration with customer data sources.

End-to-End Workflow Automation:

Automate the end-to-end process from user input to response generation, ensuring that the solution leverages AWS services like Bedrock Agents, CloudWatch, and QuickSight for real-time monitoring and analytics.

Advanced Monitoring and Analytics:

Integrated with AWS CloudWatch, QuickSight, and other monitoring tools, the accelerator provides real-time insights into performance metrics, user interactions, and system health. This allows for continuous optimization of service delivery and rapid identification of any issues.

Model Monitoring and Maintenance:

Implement model monitoring to track performance metrics and trigger retraining as necessary.

Collaboration:

Work closely with data engineers and DevOps engineers to ensure seamless integration of models into the production pipeline.

Documentation:

Document model development processes, deployment procedures, and monitoring setups for knowledge sharing and future reference.

Must-Have Skills

Machine Learning: Strong experience with machine learning frameworks such as TensorFlow, PyTorch, or Hugging Face Transformers.

MLOps Tools: Proficiency with Amazon SageMaker for model training, deployment, and monitoring.

Document processing: Experience with document processing for Word, PDF, images.

OCR: Experience with OCR tools like Tesseract / AWS Textract (preferred)

Programming: Proficiency in Python, including libraries such as Pandas, NumPy, and Scikit-Learn.

Model Deployment: Experience with deploying and managing machine learning models in production environments.

Version Control: Familiarity with version control systems like Git.

Automation: Experience with automating ML workflows using tools like AWS Step Functions or Apache Airflow.

Agile Methodologies: Experience working in Agile environments using tools like Jira and Confluence.

Nice-to-Have Skills

LLM: Experience with LLM / GenAI models, LLM Services (Bedrock or OpenAI), LLM abstraction like (Dify, Langchain, FlowiseAI), agent frameworks, rag.

Deep Learning: Experience with deep learning models and techniques.

Data Engineering: Basic understanding of data pipelines and ETL processes.

Containerization: Experience with Docker and Kubernetes (EKS).

Serverless Architectures: Experience with AWS Lambda and Step Functions.

Rule engine frameworks: Like Drools or similar

If you are a motivated individual with a passion for ML and a desire to contribute to a dynamic team environment, we encourage you to apply for this exciting opportunity. Join us in shaping the future of infrastructure and driving innovation in software delivery processes.