Commit Offshore

Senior Data Engineer

$$$$

We are looking for a Senior Data Engineer who can design, implement, and evolve scalable data systems in AWS.


Core Responsibilities
 

  • Design and implement batch and streaming data pipelines using Apache Spark.
  • Build and evolve a scalable AWS-based data lake architecture.
  • Develop and maintain real-time data processing systems (event-driven pipelines).
  • Own performance tuning and cost optimization of Spark workloads.
  • Define best practices for data modeling, partitioning, and schema evolution.
  • Implement monitoring, observability, and data quality controls.
  • Contribute to infrastructure automation and CI/CD for data workflows.
  • Participate in architectural decisions and mentor other engineers.


RRequirements

  • 5+ years of commercial experience as a Data Science / AI Engineer
  • Strong Python skills and solid software engineering fundamentals
  • Proven experience building high-load, production-grade systems
  • 2+ years of experience working with LLM and NLP-related tasks
  • Hands-on experience with RAG pipelines and vector databases
  • Strong understanding of modern deep learning architectures
  • Experience with at least one deep learning framework (TensorFlow or PyTorch)
  • Experience with SQL and NoSQL databases
  • Good knowledge of Kubernetes
  • Experience working with modern AI development tools (e.g., Claude Code, Cursor)
  • Upper-Intermediate or higher level of English (spoken and written)
  • Familiarity with Model Context Protocol (MCP)
  • Experience with production LLM frameworks (e.g., vLLM)
  • Experience training or fine-tuning LLMs
  • Experience with cloud platforms (AWS, Google Cloud, Azure)
  • Participation in Kaggle or similar competitions

Required languages

English B2 - Upper Intermediate
Apache Spark, Batch & Streaming, Apache Kafka, was, AWS, Delta Lake, SparkUI, AWS CDK, Athena, Docker
Published 27 March ยท Updated 4 May
32 views
ยท
3 applications
Response activity: High
Last responded 1 week ago
To apply for this and other jobs on Djinni login or signup.
Loading...