Python Developer/Data Mining Engineer needed for data centric startup Offline

We're in the process of scaling our current systems and looking for a clever, flexible, and resourceful engineer to help build out our data pipelines.

Our existing pipelines scrape websites for data, clean and transform the data, then make the data available to our front end UI and API.

Our tech stack primarily uses Kubernetes, Docker, Airflow, Python, Selenium, Postgres, S3, and gitlab for CI/CD.

You will be responsible for:

- Developing web scrapers for the Data Acquisition team and optimizing pipeline execution.

- Using pandas to normalize data.

- Creating docker images of scrapers to be deployed to kubernetes.

- Creating airflow dags to schedule pipelines.

- Developing tests to ensure proper pipeline execution.

This position requires experience developing python and writing web scrapers.

NOTE: Please make all e-mails and communications through the djinni website. Thank you.