Senior Data Engineer (offline)
About the project:
Our client provides solutions that leverage sophisticated machine learning, artificial intelligence (ontologies, inference engines, and rules), and natural language processing to provide highly actionable insights and recommendations to business users. Today, their solutions impact just about every aspect of the products you buy at your local store – from what is available to how its priced and even where it is placed on the shelf. Their SolaaS (Solution as a Service) solutions are deployed within private clouds – principally on Azure. Our client helps transform market-leading companies into true data-driven digital enterprises.
Stage: ongoing 1 year +.
Timezone requirements: Europe working time is ok, the client is in the EST, and our team is in Europe.
Location requirements: EU; the ability to travel abroad is essential.
Client team:
3ppl from the US (EST) – upper management;
10ppl in Europe (technical team):
1 Tech Lead, 2 Systems Analysts, 1 Sr. Full-Stack, 3 Sr. Data Engineer, 1 Sr. Python, 1 Sr. React, 1 Sr. DevOps.
English: B2.
Requirements:
- 5+ years of experience as a Data Engineer or similar position;
- 3+ years of commercial experience with Python;
- 3+ years with SQL and experience working with relational databases (Postgres preferred, similar OLTP DB is acceptable);
- Experience with Python 3, Airflow, and Snowflake;
- Demonstrated knowledge and experience developing data pipelines to automate data processing workflows;
- Demonstrated experience in data modeling;
- Demonstrated knowledge of data warehousing, business intelligence, and application data integration solutions;
- Demonstratable experience in developing applications and services that run on Azure cloud infrastructure;
- Strong skills with pandas and Pyspark/Spark;
- Experience with container management and deployment, e.g., Docker and Kubernetes;
- Excellent problem-solving and communication skills;
- Ability to thrive in a fast-paced, remote environment;
- Comfortable with ambiguity with the ability to build structure and take a proactive approach to drive results;
- Attention to detail – quality and accuracy in deliverables.
The following additional skills would be beneficial:
- Knowledge of one or more of the following technologies: Data Science, - Machine Learning, Natural Language Processing, Business Intelligence, and Data Visualization;
- Knowledge of statistics and experience using statistical or BI packages for analyzing large datasets (Excel, R, Python, Power BI, Tableau, etc.);
- Experience with at least one of the following: Databricks, Spark, Hadoop, or Kafka;
- FastAPI / Flask, or similar web backend framework experience;
- Experience with some of the following: Celery, RabbitMQ, and ZeroMQ;
Other SQL or NoSQL experience: Redis, ElasticSearch, DuckDB, ClickHouse.
What you will do:
The successful candidate will join a diverse team to:
- Build unique, high-impact business solutions utilizing advanced technologies for use by world class clients;
- Create and maintain the underlying data pipeline architecture for the solution offerings from raw client data to final solution output;
- Create, populate, and maintain data structures for machine learning and other analytics;
- Use quantitative and statistical methods to derive insights from the data;
- Combine machine learning, artificial intelligence (ontologies, inference engines, and rules), and natural language processing under a holistic vision to scale and transform businesses across multiple functions and processes.
Responsibilities Include:
- Create and maintain an optimal data pipeline architecture, incorporating data wrangling and Extract-Transform-Load (ETL) flows;
- Assemble large, complex data sets to meet analytical requirements – analytics tables, feature-engineering, etc.;
- Build the infrastructure required for optimal, automated extraction, transformation, and loading of data from a wide variety of data sources using SQL and other ‘big data’ technologies;
- Build automated analytics tools that utilize the data pipeline to derive actionable insights;
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.;
- Design and develop data integrations and data quality framework;
- Develop appropriate testing strategies and reports for the solution as well as data from external sources;
- Evaluate new technology for use within the projects;
- Work with the team and client personnel to administer and operate client-specific instances of the offerings;
- Configure the data pipelines to accommodate client-specific requirements to onboard new clients;
- Perform regular operations tasks to ingest new and changing data – implement automation where possible;
Implement processes and tools to monitor data quality – investigate and remedy any data-related issues in daily solution operations.
Tech stack: a technical stack of the project:
Back-end:
Programming Language: Python 3.11.2 (latest)
Package Manager: Poetry
Web Framework: FastAPI
Object-Relational Mapping (ORM): SQLAlchemy
Linting: ruff
Code Formatter: isort
Testing Framework: pytest
Property-based Testing: Hypothesis
Database: PostgreSQL
Front-end
Programming Language: TypeScript
Package Manager: PNPM
Framework: React
State Management: Zustand
Routing: React Router
Data Fetching: TanStack Query (successor to react-query)
Build Tool: Vite
Unit Testing: Vitest
Component Testing: React Testing Library
End-to-End Testing: Playwright
UI Kit (to be determined)
Linting: ESLint + typescript-eslint
Code Formatting: Prettier
And:
Docker and Docker Compose
Kubernetes
Airflow
Azure
Snowflake
Why choose us?
- “Family and Friends”. We are no longer a start-up, but we still, have a family atmosphere in our supportive and spirited team, who are all working together on the same goal.
- “Just break down all barriers and find a better way”. Every day, you’ll meet interesting and challenging (international) projects that cover industries from commercial aviation to fintech (different technologies, different products).
- “Hungry for learning”. You will get a lot of chances for career advancement and the development of new skills, mentorship opportunities, or learning from more experienced colleagues.
Benefits from 8allocate:
- You’ll work with a supportive and spirited team of professionals.
- Corporate events, holidays, and team buildings for your joy.
- Training and development: we have a huge library (about 500 books!) and a budget for your professional development.
- Sport or massage compensation according to the general policy.
- Compensation for psychological support or medical insurance compensation according to the general policy.
- People-oriented management without bureaucracy.
- Paid vacation and sick leaves.
Our client provides solutions that leverage sophisticated machine learning, artificial intelligence (ontologies, inference engines, and rules), and natural language processing to provide highly actionable insights and recommendations to business users. Today, their solutions impact just about every aspect of the products you buy at your local store – from what is available to how its priced and even where it is placed on the shelf. Their SolaaS (Solution as a Service) solutions are deployed within private clouds – principally on Azure. Our client helps transform market-leading companies into true data-driven digital enterprises.
Stage: ongoing 1 year +.
Timezone requirements: Europe working time is ok, the client is in the EST, and our team is in Europe.
Location requirements: EU; the ability to travel abroad is essential.
Client team:
3ppl from the US (EST) – upper management;
10ppl in Europe (technical team):
1 Tech Lead, 2 Systems Analysts, 1 Sr. Full-Stack, 3 Sr. Data Engineer, 1 Sr. Python, 1 Sr. React, 1 Sr. DevOps.
English: B2.
Requirements:
- 5+ years of experience as a Data Engineer or similar position;
- 3+ years of commercial experience with Python;
- 3+ years with SQL and experience working with relational databases (Postgres preferred, similar OLTP DB is acceptable);
- Experience with Python 3, Airflow, and Snowflake;
- Demonstrated knowledge and experience developing data pipelines to automate data processing workflows;
- Demonstrated experience in data modeling;
- Demonstrated knowledge of data warehousing, business intelligence, and application data integration solutions;
- Demonstratable experience in developing applications and services that run on Azure cloud infrastructure;
- Strong skills with pandas and Pyspark/Spark;
- Experience with container management and deployment, e.g., Docker and Kubernetes;
- Excellent problem-solving and communication skills;
- Ability to thrive in a fast-paced, remote environment;
- Comfortable with ambiguity with the ability to build structure and take a proactive approach to drive results;
- Attention to detail – quality and accuracy in deliverables.
The following additional skills would be beneficial:
- Knowledge of one or more of the following technologies: Data Science, - Machine Learning, Natural Language Processing, Business Intelligence, and Data Visualization;
- Knowledge of statistics and experience using statistical or BI packages for analyzing large datasets (Excel, R, Python, Power BI, Tableau, etc.);
- Experience with at least one of the following: Databricks, Spark, Hadoop, or Kafka;
- FastAPI / Flask, or similar web backend framework experience;
- Experience with some of the following: Celery, RabbitMQ, and ZeroMQ;
Other SQL or NoSQL experience: Redis, ElasticSearch, DuckDB, ClickHouse.
What you will do:
The successful candidate will join a diverse team to:
- Build unique, high-impact business solutions utilizing advanced technologies for use by world class clients;
- Create and maintain the underlying data pipeline architecture for the solution offerings from raw client data to final solution output;
- Create, populate, and maintain data structures for machine learning and other analytics;
- Use quantitative and statistical methods to derive insights from the data;
- Combine machine learning, artificial intelligence (ontologies, inference engines, and rules), and natural language processing under a holistic vision to scale and transform businesses across multiple functions and processes.
Responsibilities Include:
- Create and maintain an optimal data pipeline architecture, incorporating data wrangling and Extract-Transform-Load (ETL) flows;
- Assemble large, complex data sets to meet analytical requirements – analytics tables, feature-engineering, etc.;
- Build the infrastructure required for optimal, automated extraction, transformation, and loading of data from a wide variety of data sources using SQL and other ‘big data’ technologies;
- Build automated analytics tools that utilize the data pipeline to derive actionable insights;
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.;
- Design and develop data integrations and data quality framework;
- Develop appropriate testing strategies and reports for the solution as well as data from external sources;
- Evaluate new technology for use within the projects;
- Work with the team and client personnel to administer and operate client-specific instances of the offerings;
- Configure the data pipelines to accommodate client-specific requirements to onboard new clients;
- Perform regular operations tasks to ingest new and changing data – implement automation where possible;
Implement processes and tools to monitor data quality – investigate and remedy any data-related issues in daily solution operations.
Tech stack: a technical stack of the project:
Back-end:
Programming Language: Python 3.11.2 (latest)
Package Manager: Poetry
Web Framework: FastAPI
Object-Relational Mapping (ORM): SQLAlchemy
Linting: ruff
Code Formatter: isort
Testing Framework: pytest
Property-based Testing: Hypothesis
Database: PostgreSQL
Front-end
Programming Language: TypeScript
Package Manager: PNPM
Framework: React
State Management: Zustand
Routing: React Router
Data Fetching: TanStack Query (successor to react-query)
Build Tool: Vite
Unit Testing: Vitest
Component Testing: React Testing Library
End-to-End Testing: Playwright
UI Kit (to be determined)
Linting: ESLint + typescript-eslint
Code Formatting: Prettier
And:
Docker and Docker Compose
Kubernetes
Airflow
Azure
Snowflake
Why choose us?
- “Family and Friends”. We are no longer a start-up, but we still, have a family atmosphere in our supportive and spirited team, who are all working together on the same goal.
- “Just break down all barriers and find a better way”. Every day, you’ll meet interesting and challenging (international) projects that cover industries from commercial aviation to fintech (different technologies, different products).
- “Hungry for learning”. You will get a lot of chances for career advancement and the development of new skills, mentorship opportunities, or learning from more experienced colleagues.
Benefits from 8allocate:
- You’ll work with a supportive and spirited team of professionals.
- Corporate events, holidays, and team buildings for your joy.
- Training and development: we have a huge library (about 500 books!) and a budget for your professional development.
- Sport or massage compensation according to the general policy.
- Compensation for psychological support or medical insurance compensation according to the general policy.
- People-oriented management without bureaucracy.
- Paid vacation and sick leaves.
About 8allocate
8allocate is a provider of software development solutions to companies all over the globe, from North America to the EU. We deliver market-ready solutions for FinTech/InsurTech, eCommerce, AI, Web, Mobile, and other domains.Company website:
https://8allocate.com/
DOU company page:
https://jobs.dou.ua/companies/8allocate/
The job ad is no longer active
Look at the current jobs Data Engineer →
$4000-6000
Average salary range of similar jobs in
analytics →
Similar jobs
Senior Data Engineer with Azure Experience at Wondermentapps
Germany, Spain, Poland, Portugal, Ukraine
Data Migration Lead at TrueAutomation.io
Ukraine $3000-6000
Senior Data Management Engineer at TrueAutomation.io
Ukraine $2000-5000
All jobs 8allocate