Data Engineer Scraping, ETL, LLM Pipelines

$$$$

We're looking for a strong data engineer to help us build and scale the systems behind our data and LLM products. This is hands-on, senior-level work you'll own real pipelines end to end, not just tickets.

What you'll do

  • Build new datasets from scratch: structured scraping of sources like accelerators and Form D filings, parsing content with LLMs, and storing it cleanly in Supabase with well-defined schemas.
  • Stand up eval frameworks for our LLM pipeline so we can benchmark and swap different models across our various LLM calls.
  • Help design more scalable pipeline architecture, likely leveraging GCP-based services.

What we're looking for

  • 4+ years building data pipelines, backend services, and automated data processing systems, with a core focus on web scraping and ETL.
  • Distributed scraping experience (e.g. Scrapy + RabbitMQ) and comfort with dynamic sources via Playwright.
  • Pipeline deployment on AWS and/or GCP.
  • Solid working knowledge of Airflow, BigQuery, PostgreSQL, FastAPI, and Docker.
  • Fluent English communication.

We care more about depth in scraping, ETL, and pipeline deployment than about matching this exact stack  if you've built these systems well with adjacent tools, we want to talk.

 

To apply, send a short note on a scraping or pipeline system you built end to end what broke, and how you fixed it.

Required skills experience

Data Processing 2 years
Web Scraping / Scraping 2 years
Pipeline/CRM hygiene 1.5 years
Scrapy 1.5 years
Playwright 1.5 years
RabbitMQ 1.5 years
PostgreSQL 2 years
Supabase 1.5 years
BigQuery 1.5 years
Airflow 2 years
FastAPI 1.5 years
Docker 1.5 years
AWS 1.5 years
GCP BigQuery 1.5 years

Required languages

English C1 - Advanced
Ukrainian Native
Published 7 July
13 views
ยท
2 applications
To apply for this and other jobs on Djinni login or signup.
Loading...