Senior Data Engineer (offline)

About the project:
Our client provides solutions that leverage sophisticated machine learning, artificial intelligence (ontologies, inference engines, and rules), and natural language processing to provide highly actionable insights and recommendations to business users. Today, their solutions impact just about every aspect of the products you buy at your local store – from what is available to how its priced and even where it is placed on the shelf. Their SolaaS (Solution as a Service) solutions are deployed within private clouds – principally on Azure. Our client helps transform market-leading companies into true data-driven digital enterprises.

Stage: ongoing 1 year +.
Timezone requirements: Europe working time is ok, the client is in the EST, and our team is in Europe.
Location requirements: Ukraine, EU; the ability to travel abroad is essential.

Team:
3ppl from the US (EST) - upper management;
10ppl in Europe (technical team):
1 Tech Lead, 2 Systems Analysts,1 Sr. Full-Stack, 3 Sr. Data Engineer, 1 Sr. Python, 1 Sr. React, 1 Sr. DevOps.

English: Upper Intermediate.

Requirements:
- 5+ years of hands-on experience as a Data Engineer or similar position;
- 3+ years of commercial experience with Python;
- 3+ years of SQL and experience working with relational databases (Postgres preferred);
- Experience with at least one of the following – Databricks, Spark, Hadoop, or Kafka;
- Demonstratable knowledge and experience developing data pipelines to automate data processing workflows;
- Demonstratable experience in data modeling;
- Demonstratable knowledge of data warehousing, business intelligence, and application data integration solutions;
- Demonstratable experience in developing applications and services that run on cloud infrastructure: Azure preferred;
- Excellent problem-solving and communication skills;
- Ability to thrive in a fast-paced, remote environment;
- Comfortable with ambiguity with the ability to build structure and take a proactive approach to drive results;
- Attention to detail – quality and accuracy in work are essential.

Would be a plus:
- Knowledge of one or more of the following technologies: Data Science, Machine Learning, Natural Language Processing, Business Intelligence, and Data Visualization;
- Knowledge of statistics and experience using statistical or BI packages for analyzing large datasets (Excel, R, Python, Power BI, Tableau, etc.);
- Experience with container management and deployment, e.g., Docker and Kubernetes.

What you will do:
The ideal candidate must be collaborative, and deadline driven. Because of the nature of our work and our technology, successful candidates must take a growth mindset and be comfortable with ambiguity, with the ability to take a proactive, structured approach to achieve results. Results-orientation and deadline-driven are critical in our fast-paced environment.
The successful candidate will join a diverse team to:
- Build unique high-impact business solutions utilizing advanced technologies for use by world-class clients;
- Create and maintain the underlying data pipeline architecture for the solution offerings from raw client data to final solution output;
- Create, populate, and maintain data structures for machine learning and other analytics;
- Use quantitative and statistical methods to derive insights from data;
- Combine machine learning, artificial intelligence (ontologies, inference engines, and rules), and natural language processing under a holistic vision to scale and transform businesses — across multiple functions and processes.

Responsibilities Include
- Create and maintain optimal data pipeline architecture, incorporating data wrangling and Extract-Transform-Load (ETL) flows;
- Assemble large, complex data sets to meet analytical requirements – analytics tables, feature-engineering etc.;
- Build the infrastructure required for optimal, automated extraction, transformation, and loading of data from a wide variety of data sources using SQL and other ‘big data’ technologies such as Databricks;
- Build automated analytics tools that utilize the data pipeline to derive actionable insights;
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.;
- Design and develop data integrations and data quality framework;
- Develop appropriate testing strategies and reports for the solution as well as data from external sources;
- Evaluate new technology for use within the project;
- Configure the data pipelines to accommodate client-specific requirements to onboard new clients;
- Perform regular operations tasks to ingest new and changing data – implement automation where possible;
- Implement processes and tools to monitor data quality - investigate and remedy any data-related issues in daily solution operations.

Tech stack: a technical stack of the project:
Backend
Programming Language: Python 3.11.2 (latest)
Package Manager: Poetry
Web Framework: FastAPI
Object-Relational Mapping (ORM): SQLAlchemy
Linting: ruff
Code Formatter: isort
Testing Framework: pytest
Property-based Testing: Hypothesis
Database: PostgreSQL

Frontend
Programming Language: TypeScript
Package Manager: PNPM
Framework: React
State Management: Zustand
Routing: React Router
Data Fetching: TanStack Query (successor to react-query)
Build Tool: Vite
Unit Testing: Vitest
Component Testing: React Testing Library
End-to-End Testing: Playwright
UI Kit (to be determined)
Linting: ESLint + typescript-eslint
Code Formatting: Prettier
And:
Docker and Docker Compose
Kubernetes
Airflow
Azure
Snowflake

Why choose us?
- “Family and Friends”. We are no longer a start-up, but we still, have a family atmosphere in our supportive and spirited team, who are all working together on the same goal.
- “Just break down all barriers and find a better way”. Every day you’ll meet with interesting and challenging (international) projects that cover industries from commercial aviation to fintech (different technologies, different products).
- “Hungry for learning”. You will get a lot of chances for career advancement and the development of new skills, mentorship opportunities, or learning from more experienced colleagues.

Benefits from 8allocate:
- You’ll work with a supportive and spirited team of professionals.
- Corporate events, holidays, and team buildings for your joy.
- Training and development: we have a huge library (about 500 books!) and a budget for your professional development.
- Sport or massage compensation according to the general policy.
- Compensation for psychological support or medical insurance compensation according to the general policy.
- People-oriented management without bureaucracy.
- Paid vacation and sick leaves.

About 8allocate

8allocate is a provider of software development solutions to companies all over the globe, from North America to the EU. We deliver market-ready solutions for FinTech/InsurTech, eCommerce, AI, Web, Mobile, and other domains.

Company website:
https://8allocate.com/

DOU company page:
https://jobs.dou.ua/companies/8allocate/