Senior Data Engineering / AI Specialist (RAG / Vector Search)

Project Details

• Location: Ukraine / EU
• Work format: Full-time
• Duration: 3+ months
• Rate: Up to $20/hour

• English: B2+

   

Senior Data Engineering / AI Specialist (RAG / Vector Search)

We are looking for a Senior Data Engineering / AI Specialist to build large-scale document processing and retrieval infrastructure for an AI-powered platform used in construction and infrastructure project management.

The platform works with more than 1 million documents and 10+ TB of data per client, enabling intelligent search and analysis across complex project portfolios.

Responsibilities

• Design and implement scalable batch ingestion pipelines for millions of documents.
• Build resumable, partitioned data processing workflows with retries, deduplication, and quality controls.
• Develop document parsing pipelines, including OCR, table extraction, and layout analysis.
• Generate structure-aware chunks, embeddings, metadata, and hierarchical summaries.
• Design and optimize hybrid search infrastructure combining BM25 and vector search.
• Implement reranking, metadata filtering, hierarchical retrieval, and context compression.
• Evaluate retrieval quality using golden datasets, recall, precision, and faithfulness metrics.
• Optimize GPU inference, batch processing throughput, and infrastructure costs.
• Support incremental document updates, versioning, and production operations.

Requirements

• Strong production experience with large-scale data processing, including millions of documents or multi-terabyte datasets.
• Hands-on experience with Spark, Ray, AWS Batch/Glue, Kubernetes Jobs, or equivalent technologies.
• Experience with vector databases and search engines such as OpenSearch, Elasticsearch, pgvector, Milvus, or Qdrant.
• Strong understanding of vector indexing, hybrid search, ANN tuning, and partitioning.
• Experience with document parsing, OCR, and table extraction technologies.
• Practical experience implementing RAG pipelines, embeddings, chunking, and reranking.
• Experience evaluating retrieval quality through benchmarks and measurable metrics.
• Strong Python skills.
• Production experience with AWS, Azure, or GCP.
• Experience estimating and optimizing GPU, compute, and LLM token costs.
• English: B2+.

Nice to Have

• GPU inference experience with vLLM, Triton, or TEI.
• Experience with Apache Iceberg, Delta Lake, or Parquet.
• Knowledge of GraphRAG and entity linking.
• Experience with permission-aware retrieval and audit requirements.
• Background in construction, engineering, legal technology, or other document-intensive domains.
• Experience deploying solutions across multiple cloud environments.

We are looking for an engineer who can combine strong data engineering expertise with practical AI retrieval experience and build reliable, scalable production systems.