Middle Data Engineer

The project: Our client is a private investment company based in the United States, specializing in agriculture and food-related investments. The main idea behind the product that we are developing is to boost the agricultural industry by providing farming companies with a comprehensive Business Intelligence tool. This solution will empower them to acquire, analyze, and visualize data related to Agricultural areas, driving informed decision-making and maximizing revenue potential.
 

Requirements:
 

• 3+ years of development experience with Python using FastAPI or similar web frameworks.
• Passion for continuous learning and sharing knowledge with others, great communication skillіs.
• Strong debugging skills.
• Good understanding of all parts of the web application stack.
• Experience with Pandas/Pyarrow.
• Experience with RDBMS (preferably Postgres).
• Experience working with microservice architecture.
• Strong experience with data processing techniques in tools.
• Experience with ETL pipelines.
• Experience with Apache Spark/Delta Lake or similar tools, as well as experience with Airflow orchestrator.
• English – Upper-Intermediate.
   

Nice to have:
 

• AWS EMR/SNS/SQS, Docker
• Experience with Apache Airflow, building out workflow DAGs, and scheduling jobs
• Experience with OAuth
• Some UI development skills might be useful later on in a project (Dash)
• Experience with NoSQL databases (DynamoDB, Redis).
   

Responsibilities:
 

• Build and optimize ETL pipelines and data-processing workflows with Pandas, Spark/Delta Lake.
• Develop and maintain backend services and microservices using Python (FastAPI) for the BI platform.
• Сontribute to overall data architecture and system performance. support workflow orchestration (Dagster) and ensure smooth data operations.
• Debug complex issues across the application stack and collaborate closely with the team to deliver high-quality features.
   

What we offer:
 

• Annual paid vacation of 18 working days.
• Extra vacation days for long-lasting cooperation.
• Annual paid sick leave of 10 days.
• Maternity/Paternity leave.
• The opportunity for sabbatical leave.
• Marriage and Parenthood Package.
• Compensation for sports activities (up to 250$ per year) or health insurance covering (70%) — after the trial period.
• Internal education(corporate library, Udemy courses).
• Career development plan.
• English and Spanish classes.
• Paying taxes and managing PE (Private Entrepreneur).
• Technical equipment.
• Internal Referral program.
• Opportunity to take part in company volunteering activities.
• Sombra is a “Friendly to Veterans” award-holder.