MeGaDev

About the Role

We are looking for an experienced Lead / Senior Data Engineer to join a high-performing technology team delivering innovative enterprise and AI-driven solutions within a global professional services environment.

The team designs, develops, and deploys modern technology platforms and intelligent tools that support complex business and operational processes. You will work alongside experts in software engineering, data, AI, change management, and project delivery on initiatives ranging from solution design and development to deployment and continuous improvement.

This is an exciting opportunity to work on large-scale, modern cloud and data engineering projects using Azure, Databricks, Spark, Python, and AI-powered technologies.

Technology Stack

• Azure Cloud
• Azure Databricks
• Apache Spark / PySpark
• Microservices Architecture
• .NET 8
• ASP.NET Core
• Python
• MongoDB
• Azure SQL
• Angular
• GitHub and AI-assisted development tools
• LangGraph
• LangChain
• RAG Pipelines
• Multimodal LLMs

Your Responsibilities

• Define and promote engineering best practices and coding standards across the project
• Conduct thorough code reviews to ensure high code quality and adherence to established standards
• Work independently while collaborating closely with cross-functional and distributed teams
• Provide technical guidance and clear direction to team members
• Support the coordination of day-to-day technical activities and delivery priorities
• Communicate regularly with business and project stakeholders
• Design, develop, and maintain robust, scalable, and high-performance Spark applications
• Write clean, maintainable, and efficient code following modern software engineering principles
• Optimize applications and data-processing workflows for performance and scalability
• Ensure efficient and reliable data handling across large-scale data platforms
• Collaborate with engineering, product, data, and business teams to deliver high-quality solutions
• Identify, investigate, and resolve complex technical issues
• Create and maintain comprehensive technical documentation for code, processes, architectures, and workflows

Experience & Technical Skills

• 5+ years of hands-on experience in software development or data engineering
• Practical experience developing SQL stored procedures and migrating legacy SQL logic into Spark SQL or PySpark environments
• Extensive hands-on experience with PySpark and Azure Databricks
• Strong knowledge of Delta Tables, cluster management, and workflow automation
• Proven experience optimizing Apache Spark performance in production environments
• Strong Python programming skills, particularly for complex data processing and manipulation using libraries such as Polars or Pandas
• Solid understanding of columnar data-storage formats, particularly Parquet
• Practical experience working with Delta Lake and modern lakehouse architectures
• Strong expertise in data processing, transformation, and analytical workflows
• Strong analytical and problem-solving skills with a detail-oriented mindset
• Ability to balance engineering standards and structured processes with pragmatic delivery requirements
• Solid understanding of microservices architecture and scalable distributed systems

Nice to Have

• Experience working with Azure Cloud services or other major cloud platforms
• Experience with cloud-based services such as messaging platforms, Data Lake, object storage, caching solutions, and related managed services
• Familiarity with FastAPI
• Experience with containerization and orchestration technologies such as Docker and Kubernetes
• Exposure to AI-driven applications, RAG architectures, LLM orchestration frameworks, or modern generative AI solutions

About the Client

Our client is the world’s largest human resources consulting firm, headquartered in New York City, with major operations across more than 40 countries.

The organisation employs more than 25,000 professionals globally and delivers services across 130+ countries, supporting some of the world's largest enterprises. Its solutions and expertise are trusted by 97% of Fortune 500 companies.

About the Role

We are looking for experienced Forward Deployed Engineers to design, build, and maintain reusable Claude Code Skills that enable AI agents to support complex software-development and business workflows.

You will work directly with business stakeholders and domain experts to understand their objectives, processes, constraints, and operational knowledge, and translate these inputs into reliable, reusable AI capabilities.

This is a highly hands-on engineering role at the intersection of software engineering, AI agent development, prompt engineering, context engineering, and business process automation.

Key Responsibilities

• Work directly with business stakeholders and domain experts to understand their objectives, workflows, constraints, and expected outcomes.
• Analyse business requirements, documentation, examples, templates, and other reference materials to identify opportunities for AI-driven automation.
• Identify business workflows and use cases that can be implemented, streamlined, or enhanced through Claude Code Skills.
• Design, implement, test, and maintain reusable and production-quality Claude Code Skills.
• Develop clear skill instructions, supporting scripts, reference materials, validation logic, and reusable workflow components.
• Translate complex, ambiguous, or domain-specific business processes into structured and executable instructions for AI agents.
• Apply prompt and context engineering techniques to improve the reliability and effectiveness of AI-driven workflows.
• Evaluate skill behaviour using realistic scenarios, test cases, acceptance criteria, and representative business data.
• Iteratively improve the accuracy, reliability, usability, and maintainability of AI skills based on testing results and stakeholder feedback.
• Collaborate with engineering and business teams to ensure developed solutions are practical, scalable, and aligned with real-world requirements.
• Contribute to the development of reusable patterns and best practices for agentic engineering and AI-powered developer workflows.

Required Experience

• Strong hands-on experience using Claude Code in real-world software development or business workflow scenarios.
• Practical experience creating Claude Code Skills or comparable reusable capabilities for AI coding agents.
• Strong understanding of modern AI engineering and agentic development patterns, including:
• Agentic workflows
• Tool use and tool calling
• Prompt engineering
• Context engineering
• Structured instruction design
• AI workflow orchestration
• Proven professional software engineering experience, with the ability to design, build, test, review, and maintain production-quality solutions.
• Ability to translate business requirements, operational processes, and domain knowledge into clear technical designs and executable AI workflows.
• Experience working with non-deterministic AI systems, including evaluating behaviour, identifying failure modes, and iteratively improving solution quality.
• Strong analytical and problem-solving skills, with the ability to work effectively in ambiguous and rapidly evolving environments.
• Strong communication skills and the ability to collaborate directly with both technical and non-technical stakeholders.

Nice to Have

Advanced hands-on experience with Claude Code, particularly authoring:

• Claude Code Skills
• Plugins
• MCP server
  Experience with LLM evaluation and testing frameworks, such as:
• Promptfoo
• LangSmith
• Custom evaluation harnesses

Key Responsibilities  

• Design, build, and maintain scalable, reliable data pipelines (batch and streaming).  
• Develop and optimise data models, warehouses, and lakehouse architectures.  
• Build and automate CI/CD pipelines for data workflows and deployments.  
• Ensure data quality, integrity, lineage, and governance across environments.  
• Implement proactive monitoring and alerting for pipelines and data infrastructure (CloudWatch, Grafana).  
• Manage IaC for consistent, reliable data infrastructure.  
• Troubleshoot and optimise pipeline and query performance across environments.  
• Lead incident resolution and root cause analysis for data systems.  
• Collaborate with analysts, data scientists, and engineering teams, ensuring security, scalability, and operational excellence to understand data requirements and translate them into technical solutions  
• Document data architecture, lineage, and workflows  

Must have  

• 5+ years in data engineering, with solid DevOps/platform engineering exposure.  
• Strong SQL and data modelling skills (dimensional, normalised, lakehouse).  
• Proficient in Python; working knowledge of Scala, Java, or Go.  
• Hands-on experience with data pipeline/orchestration tools (Airflow, DBT).  
• Experience with distributed processing frameworks (Spark, Flink, or similar).  
• Advanced AWS expertise (Glue, EMR, Redshift, Athena, S3, Lambda, Kinesis).  
• CI/CD tools (GitHub Actions, GitLab CI, Jenkins, CircleCI).  
• Infrastructure as Code (CDK, CloudFormation).  
• Experience with streaming/event-driven architecture (Kafka, Kinesis, Apache Flink, or Pub/Sub).  
• Observability tools: Grafana, CloudWatch.  
• Expertise with data warehouse platforms (Redshift, Snowflake).  

Desirable  

• Experience with containerisation and orchestration (Docker, ECS, Kubernetes).  
• Understanding of data governance, cataloguing, and security/compliance (GDPR, lineage tools).  
• Experience with GitHub Enterprise, Atlassian APIs, and Slack bots.  
• Understanding of cost optimisation strategies in multi-cloud and data warehouse environments.  

Key Responsibilities  

• Design, develop & maintain scalable, secure cloud infrastructure.  
• Architect and manage cloud networking, including VPC design, segmentation, and connectivity.  
• Automate CI/CD pipelines and deployments.  
• Implement proactive monitoring and alerting (CloudWatch).  
• Troubleshoot and optimise system and network performance across environments.  
• Manage IaC for consistent, reliable infrastructure.  
• Lead incident resolution and root cause analysis.  
• Collaborate on architecture, ensuring security, scalability, and operational excellence to ensure reliable application deployment and performance.  
• Continuously evaluate and recommend improvements to infrastructure and DevOps processes.  
• Participate in disaster recovery planning and system backup strategies.  
• Ensure compliance with security best practices and organisational policies.  

Must have  

• 5+ years in DevOps, SRE, or platform engineering.  
• Advanced AWS expertise (Lambda, ECS, API Gateway, S3).  
• Strong AWS networking experience, including Transit Gateway, VPC (subnets, route tables, peering), PrivateLink, Route 53, Elastic Load Balancing, NAT Gateway, and Security Groups/NACLs.  
• Proficient in Python, Bash; working knowledge of Node.js or Go.  
• Skilled in CI/CD tools (GitHub Actions, GitLab CI, AWS Codebuild).  
• Infrastructure as Code (CDK, CloudFormation).  
• Observability tools: Grafana, Datadog.  
• GitHub Enterprise, Atlassian APIs, Slack bots.  
• Event-driven architecture & microservices.  
• Significantly important - Data Platforms experience: ETL Processes, Data Injections, SageMaker, Mash Solutions etc.

Desirable  

• Familiarity with event-based architecture and microservices environments.  
• Experience designing multi-account/multi-VPC network topologies and hybrid connectivity.  
• Experience with GitHub Enterprise, Atlassian APIs, and Slack bots.  
• Proficiency with observability tools like Grafana and Datadog.  
• Understanding of cost optimization strategies in multi-cloud environments, including network/data transfer cost optimization.  

Responsibilities:
- Identify, research, and generate new leads among IT companies and digital agencies (10–500 employees).
- Execute outbound campaigns via LinkedIn, email, and other relevant channels.
- Qualify inbound leads from marketing and partners to ensure they match our ICP.

- Design, automate, and execute high-volume, AI-driven outbound email and LinkedIn campaigns.
- Build and maintain lead lists and outreach pipelines.
- Personalize outreach messages to increase response and conversion rates.
- Collaborate closely with sales to ensure smooth lead handoff and pipeline growth.
- Track and report on outreach performance (open rates, reply rates, conversions).
- Maintain accurate data in CRM systems.
- Continuously test and improve outreach strategies and messaging.

Requirements:
- 1+ years of experience in lead generation or a similar role + experience in IT Outstaffing / Staff Augmentation.
- Strong written communication skills (especially for cold outreach).
- Understanding of outbound sales processes and lead qualification principles.
- Experience with LinkedIn, email outreach tools, and CRM systems (e.g., HubSpot, Pipedrive).
- Ability to research companies and identify decision-makers.
- Basic understanding of software development roles and tech stacks to correctly identify decision-makers (CTOs, VPs of Engineering, Product Owners).
- Results-driven mindset with attention to detail.

Additional Qualifications:
- Having several LinkedIn accounts of your own for work.