Jobs Data Engineer

Quantum is a global technology partner delivering high-end software products that address real-world problems. 

We advance emerging technologies for outside-the-box solutions. We focus on Machine Learning, Computer Vision, Deep Learning, GIS, MLOps, Blockchain, and more.

Here at Quantum, we are dedicated to creating state-of-the-art solutions that effectively address the pressing issues faced by businesses and the world. To date, our team of exceptional people has already helped many organizations globally attain technological leadership.

We constantly discover new ways to solve never-ending business challenges by adopting new technologies, even when there isn’t yet a best practice. If you share our passion for problem-solving and making an impact, join us and enjoy getting to know our wealth of experience!

About the position

Quantum is expanding the team and has brilliant opportunities for a proactive Senior Data Engineer who can design, implement, and evolve scalable data systems in AWS. We are building a greenfield analytics platform supporting both batch and real-time data processing. This role is a strategic blend of hands-on development, high-level architectural decision-making, and end-to-end platform ownership.

The client is a technological research company that utilizes proprietary AI-based analysis and language models to provide comprehensive insights into global stocks in all languages. Our mission is to bridge the knowledge gap in the investment world and empower investors of all types to become “super-investors.”

Through our generative AI technology implemented into brokerage platforms and other financial institutions’ infrastructures, we offer instant fundamental analyses of global stocks alongside bespoke investment strategies, enabling informed investment decisions for millions of investors worldwide.

Must have skills:

• 5+ years of experience in Data Engineering
• Strong hands-on experience with Apache Spark (including Structured Streaming)
• Experience building both batch and streaming pipelines in production environments
• Proven experience designing AWS-based data lake architectures (S3, EMR, Glue, Athena)
• Experience with event streaming platforms such as Apache Kafka or Amazon Kinesis
• Experience implementing lakehouse formats such as Delta Lake
• Strong understanding of partitioning strategies and schema evolution
• Experience using SparkUI and AWS CloudWatch for profiling and optimization
• Strong understanding of Spark performance tuning (shuffle, skew, memory, partitioning)
• Proven track record of cost optimization in AWS environments
• Experience with Docker and CI/CD pipelines
• Experience with Infrastructure as Code (Terraform, AWS CDK, or similar)
• Familiarity with monitoring and observability practices
• Upper-Intermediate or higher English proficiency (spoken and written)

Would be a plus:

• Experience in the financial domain
• Experience running Spark workloads on Kubernetes
• Exposure to or interest in Large Language Models (LLMs) and AI integration
• Experience implementing data quality frameworks or metadata/lineage systems

Your tasks will include:

• Design and implement batch and streaming data pipelines using Apache Spark
• Build and evolve a scalable AWS-based data lake architecture
• Develop and maintain real-time data processing systems (event-driven pipelines)
• Own performance tuning and cost optimization of Spark workloads
• Define best practices for data modeling, partitioning, and schema evolution
• Implement monitoring, observability, and data quality controls
• Contribute to infrastructure automation and CI/CD for data workflows

• Participate in architectural decisions and mentor other engineers

   

We offer:

• Delivering high-end software projects that address real-world problems
• Surrounding experts who are ready to move forward professionally
• Professional growth plan and team leader support
• Taking ownership of R&D and socially significant projects
• Participation in worldwide tech conferences and competitions
• Taking part in regular educational activities
• Being a part of a multicultural company with a fun and lighthearted atmosphere
• Working from anywhere with flexible working hours

• Paid vacation and sick leave days

   

Join Quantum and take a step toward your data-driven future.

We are seeking a skilled, hands-on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real-time data streaming solutions.

Responsibilities

Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.

Develop, optimize, and maintain PySpark applications within the Databricks environment.

Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.

Write and maintain unit tests to ensure high code quality and reliability.

Perform data validation and quality checks to ensure compliance with defined business rules

Support and manage production deployments, ensuring system stability and performance.

Troubleshoot and resolve issues across development, testing, and production environments.

Collaborate with Product, QA, DevOps, and healthcare domain teams to ensure accurate implementation.

Maintain technical documentation for code, workflows, and deployment processes.

Pull from and push code to GitHub repositories, following version control and branching best practices.

Skills

Must have

5+ years of experience in data engineering roles

Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).

4+ years of experience with Databricks data pipelines

Strong hands-on experience with PySpark and/or Apache Spark.

Experience working with CI/CD pipelines and DevOps best practices.

Proficiency with CI/CD, GitHub (pull requests, branching, code reviews).

Experience writing unit tests and performing data validation

Experience supporting production systems and deployments

Strong analytical and troubleshooting skills.

Nice to have

Knowledge of FHIR (Fast Healthcare Interoperability Resources) standards.

Experience with healthcare data integration projects.

Familiarity with REST APIs and FHIR server integrations.

Experience in Agile/Scrum development environments.

We are looking for a Senior Data Engineer who is passionate about building scalable, cloud-native data solutions and eager to grow into a broader architecture role. You will lead the technical delivery of modern data platform solutions while engaging with clients and internal stakeholders to design, implement, and optimize enterprise-grade data systems.

Responsibilities

• Lead end-to-end development and implementation of scalable data pipelines, data lakes, and cloud-native data platforms
• Ensure performance, security, and reliability of data solutions through CI/CD, testing, and automation best practices
• Participate in solution design with architects and senior stakeholders - contribute to data architecture, system integrations, and technology selection
• Translate business needs into data models, transformation logic, and orchestration workflows
• Work closely with business and technical stakeholders to clarify requirements, demonstrate value, and refine deliverables
• Participate in pre-sales activities: help shape proposals, provide technical insights, and create solution roadmaps
• Stay on top of modern data platform trends (e.g., Data Mesh, Lakehouse, real-time analytics, observability)

• Explore new tools and techniques - contribute to N-iX’s internal Data & Analytics accelerators and reference architectures

   

Requirements

• 5-7 years of hands-on experience in data engineering
• Strong SQL, Python, and Spark skills
• Strong expertise in Azure and Databricks modern data platforms
• Knowledge of Airflow, dbt, Kafka, or similar tools preferred
• Proven experience building data lakes, lakehouses, data warehouses, and streaming pipelines
• Familiarity with architectural concepts like Data Mesh, Data Fabric, and cloud-native data architectures
• Experience leading delivery teams or mentoring other engineers
• Excellent verbal and written communication in English
• Willingness to travel for business (when needed) and to engage directly with clients
   

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers

Digis is looking for a Senior Data Engineer to join our long-term partnership with a US-based global leader in hospitality analytics.

About the Project

Our client is a premier SaaS provider that transforms front-desk staff into revenue generators for the world’s most iconic hotel brands, including Hilton and Marriott. Their platform operates in over 2,000 properties across 124 countries, managing billions in revenue.

The project is in an active development stage, focusing on processing massive data volumes to deliver real-time business value.
You will join a large, distributed team (approx. 50 people) and collaborate directly with the US-based CTO.

Your Responsibilities
 

• Manage data pipelines from ingestion to ready-to-use analytical models.
• Help scale case detection processes and improve existing services by splitting them into smaller microservices.
• Build, debug, and optimize ETL jobs using AWS Glue, EMR, and Spark.
• Write and improve SQL queries for PostgreSQL/Redshift, and work with DynamoDB where needed.
• Support workflows in AWS ECS Fargate and set up local environments for testing and deployment.
   

Requirements
 

• 5+ years of experience as a Data Engineer.
• 2+ years of hands-on production experience with Spark or PySpark.
• 3+ years of experience with the AWS ecosystem (specifically Glue, EMR, S3, and Redshift).
• English: High level (Upper-Intermediate/Advanced) for frequent technical meetings and direct collaboration with the US CTO.

We Offer
 

• 20 paid vacation days per year
• 5 paid sick leaves per year (no medical documents required)
• Personalized development plan with training compensation
• English courses compensation
• Work equipment if needed (PC / laptop / monitor)
• Opportunity to switch between projects and technologies within Digis
• Full accounting and legal support

Looking for Backend with #Python #Machine learning #AWS stack

We are looking for a Senior Data Engineer to build and scale a modern, AWS-based data platform powered by serverless pipelines, DuckDB, and Delta Lake architecture. You will work on integrating multiple SaaS and operational data sources into a robust data warehouse used for analytics and Tableau reporting. This role requires a hands-on engineer comfortable with Python-based data processing, AWS services, and non-traditional data stack components (DuckDB, Polars, etc.).

Seniority: Senior
English: Upper
Start date: April 4, 2026
Duration: 1-3 month

We are looking for a Data Engineer to design and implement scalable data pipelines that support Generative AI applications. You will work with large-scale datasets, optimize performance, and ensure seamless data integration across AI workflows.

Responsibilities:

• Design and develop scalable data pipelines to support AI/ML models.
• Work with structured and unstructured data for training and inference.
• Optimize ETL workflows for real-time and batch processing.
• Ensure data quality, governance, and security in cloud environments.
• Collaborate with AI/ML engineers to streamline data workflows.

• Deploy and manage data infrastructure on AWS, GCP, or Azure.

   

Requirements:

• 3+ years of experience in Data Engineering and Big Data technologies.
• Expertise in ETL pipelines, data warehousing, and distributed computing.
• Strong knowledge of SQL, NoSQL, and data lakes.
• Experience with cloud-based data services (AWS Glue, BigQuery, Azure Data Factory).
• Proficiency in Python, Spark, Kafka, Airflow.

• Experience with CI/CD, DevOps, and Agile methodologies.

   

  What We Offer: 

  🚀 A role with real ownership and influence on delivery

  🌍 International eCommerce projects with global clients

  📈 Transparent growth path toward Middle+ → Senior PM

  🎓 Company-supported learning & certification programs

  🏠 Fully remote work with a flexible schedule

  🏝 Paid vacation & sick leave

  💰 Competitive compensation

  🔎 Mature processes and open team culture

We are looking for an experienced Data Engineer to design and implement scalable data pipelines, own core storage layers, and help build the foundations of a modern DataOps lifecycle.
 

Job requirements

• 3+ years of professional experience in Data Engineering / DWH development.
• Strong SQL expertise and solid Python skills (pandas, pyarrow).
• Production experience with Apache Airflow (production DAGs, scheduling, operators).
• Hands‑on work with PostgreSQL (schema design, migrations, partitioning) and ClickHouse (MergeTree engines, materialized views, performance tuning). 
• Experience with Data warehouse design — layered architecture, data modelling, denormalisation.
• Experience integrating REST APIs and third-party data sources
• Experience with Docker / Docker Compose — containerised services and multi-service environments
• Fluency in English.

Nice to Have:

• Experience with Apache Superset (RLS, dashboard layer).
• Familiarity with AI/LLM tooling for data (NL-to-SQL, vector databases). 
• Experience in the Fintech field: FX, brokerage, CRM/BO systems.
• Experience with multi-tenant data architectures.

Job responsibilities

• Design, build, and maintain ETL/ELT pipelines rom multiple product datasources.
• Model and optimise data structures in ClickHouse (views, engines, schema evolution).
• Orchestrate workflows using Apache AirflowCollaborate with BI teams (Superset) and backend teams (FastAPI/Python) to build reliable data products.
• Contribute to architectural decisions.
• Collaborate closely with product managers, BI developers, and DevOps to deliver data products end‑to‑end.

Job benefits

● Competitive and Attractive Pay.
● Flexible hours for a better work-life balance.
● Work your way: Hybrid flexibility to fit your life.
● 21 vacation + 7 no-questions-asked sick days per year.
● Career growth: Continuous Development & Performance Reviews.
 

Our customer is one of the world's largest food and beverage companies, operating in 189 countries with over 2000 brands. The company's diverse portfolio includes products such as bottled water, dairy products, breakfast cereals, coffee, confectionery, frozen food, pet foods, snacks, and baby food.

We are looking for a Middle Data Engineer to design, build, and maintain scalable data systems that support analytics, reporting, and machine learning use cases. This role requires strong ownership, solid SQL skills, and experience working with large datasets in production environments.

Responsibilities

• Build and maintain data pipelines (batch and near real-time).
• Develop and optimize data models for analytics and BI tools.
• Write efficient SQL queries for large-scale datasets.
• Support development and maintenance of ETL/ELT processes.
• Ensure data quality, integrity, and consistency across systems.
• Collaborate with data analysts, data scientists, and product teams to understand data requirements.
• Monitor and troubleshoot data pipelines and workflows.
• Contribute to improving the performance of the existing data infrastructure.
• Work with cloud-based data platforms and modern data stack tools.

Requirements

• 2–4 years of experience in Data Engineering or a related role.
• Main technical skills are: Snowflake, dbt, ADF.
• Solid SQL skills (joins, aggregations, window functions).
• Understanding of data governance and data quality concepts.
• Proficiency in Python.
• Experience with ETL/ELT tools.
• Hands-on experience with the Azure cloud platform.
• Azure Ops experience: provisioning and configuring.
• Experience working with data warehouses (e.g., Snowflake, BigQuery, Redshift).
• Understanding of data modeling concepts (e.g., star schema).

Nice to have

• Basic knowledge of Docker or containerization.
• Experience with data processing frameworks (e.g., Spark).

Soft skills

• Strong analytical thinking and problem-solving skills.
• Ability to work independently with some guidance.
• Good communication skills with technical and non-technical stakeholders.
• Willingness to learn and grow in a collaborative environment.

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers

The Digital Health organization is technology team which focused on next generation Digital Health capabilities which deliver on the Medicine mission and vision to deliver Insight Driven Care. This role will operate within the Digital Health Applications & Interoperability subgroup of the broader Digital Health team, focused on patient engagement, care coordination, AI, healthcare analytics & interoperability amongst other advanced technologies which enhance our product portfolio with new services, while improving clinical & patient experiences.

 The project is a cloud-based PaaS Ecosystem built with a privacy by design centric approach to provide a centralized cloud-based platform to store, classify, and control access to federated datasets in a scalable, secure, and efficient manner.

The ecosystem will allow Customer Operating Units (medical device departments) to store federated data sets of varying sizes and formats and control access to those data sets through Data steward(s). Source data sets can be exposed to qualified use cases and workflows through different project types.

The Healthcare Data Platform ecosystem will provide ML/AI project capabilities for streamlined development processes and a ML/AI workbench to enhance data exploration, wrangling, and model training.

In queue: 15+ OU’s. At this moment focused on – Nuero, Cardio, Diabetes is the OU that data platform is working with, but there could be more OU’s coming up with requirements in future.

GL Role: is to work on the enhancement of current capabilities, including taking over the work that AWS proserve team is doing, and develop new requirements that will keep coming from different OU’s in the future.
 

Requirements:
 

• AWS Platform: Working experience with AWS data technologies, including S3, AWS RDS, and Lake Formation
• Programming Languages: Strong programming skills in Python
• Data Formats: Experience with JSON, XML, and other relevant data formats
• CI/CD Tools: Ability to deploy using established CI/CD pipelines using GitLab CI, Jenkins, Terraform or similar tools
• Scripting and automation: experience in scripting languages such as Python, PowerShell, etc.
• Monitoring and Logging: Familiarity with monitoring & logging tools like CloudWatch, Splunk, ELK, Dynatrace, Prometheus
• Source Code Management: Expertise with GitLab

• Documentation: Experience with markdown and in particula,r Antora for creating technical documentation
   

  NICE TO HAVE:
   

• Previous Healthcare or Medical Device experience
• Experience implementing enterprise grade cyber security & privacy by design into software products
• Experience working in Digital Health software
• Experience developing global applications
• Strong understanding of SDLC;
• Experience with Agile methodologies
• Software estimation
• Experience leading software development teams onshore and offshore
• Experience with FHIR
   

Job responsibilities:
 

• Implement data pipelines using AWS services such as AWS Glue, Lambda, Kinesis, etc
• Implement integrations between the data platform and systems such as Atlan, Trino/Starburst, etc
• Complete logging and monitoring tasks through AWS and Splunk toolsets
• Develop and maintain ETL processes to ingest, clean, transform and store healthcare data from various sources
• Optimize data storage solutions using Amazon S3, AWS RDS, Lake Formation and other AWS technologies.
• Document, configure, and maintain systems specifications that conform to defined architecture standards, address business requirements, and processes in the cloud development & engineering.
• Participate in planning of system and development deployment as well as responsible for meeting compliance and security standards.
• Actively identify system functionality or performance deficiencies, execute changes to existing systems, and test functionality of the system to correct deficiencies and maintain more effective data handling, data integrity, conversion, input/output requirements, and storage.
• Document testing and maintenance of system updates, modifications, and configurations.
• Leverage platform process expertise to assess if existing standard platform functionality will solve a business problem or customization solution would be required.
• Test the quality of a product and its ability to perform a task or solve a problems.
• Perform basic maintenance and performance optimization procedures in each of the primary operating systems.
• Ensure system implementation compliance with global & local regulatory and security standards (i.e. HIPAA, SOCII, ISO27001, etc…)

Project Description:

We are seeking a skilled, hands-on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real-time data streaming solutions.

Responsibilities:

- Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.
- Develop, optimize, and maintain PySpark applications within the Databricks environment.
- Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.
- Write and maintain unit tests to ensure high code quality and reliability.
- Perform data validation and quality checks to ensure compliance with defined business rules
- Support and manage production deployments, ensuring system stability and performance.
- Troubleshoot and resolve issues across development, testing, and production environments.
- Collaborate with Product, QA, DevOps, and healthcare domain teams to ensure accurate implementation.
- Maintain technical documentation for code, workflows, and deployment processes.
- Pull from and push code to GitHub repositories, following version control and branching best practices.

Mandatory Skills Description:

- 5+ years of experience in data engineering roles
- Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).
- 4+ years of experience with Databricks data pipelines
- Strong hands-on experience with PySpark and/or Apache Spark.
- Experience working with CI/CD pipelines and DevOps best practices.
- Proficiency with CI/CD, GitHub (pull requests, branching, code reviews).
- Experience writing unit tests and performing data validation
- Experience supporting production systems and deployments
- Strong analytical and troubleshooting skills.

Nice-to-Have Skills Description:

- Knowledge of FHIR (Fast Healthcare Interoperability Resources) standards.
- Experience with healthcare data integration projects.
- Familiarity with REST APIs and FHIR server integrations.
- Experience in Agile/Scrum development environments.

Languages:

English: C1 Advanced

RedCore is an international business group that creates technological solutions for digital markets. Our products and services cover fintech, marketing, e-commerce, customer service, communications and regulatory technologies. 

We are looking for a Data Engineer to join our team!

Requirements:
- 2+ years of experience in the role of data engineer;

- Experience in designing and implementing data processing systems;

- Good knowledge of Python basics (including SOLID design principles);

- HTTP;

- Git (GitLab);
 

Will be plus:

- Columnstore DBs (Clickhouse);

- Airflow;

Responsibilities:
- Communication with clients on the task requirements;

- Write high performance, testable and maintainable code to implement new functionality;

- Take part in technical discussions to come up with solutions for challenging issues;

- Participate in code reviews to ensure code quality and distribute knowledge.

Our benefits to you:
🔸An exciting and challenging job in a fast-growing holding, the opportunity to be part of a multicultural team of top professionals in Development, Architecture, Management, Operations, Marketing, Legal, Finance and more
Great working atmosphere with passionate experts and leaders, sharing a friendly culture and a success-driven mindset is guaranteed
🔸Modern corporate equipment based on macOS or Windows and additional equipment are provided
🔸Paid vacations, sick leave, personal events days, days off
🔸Referral program — enjoy cooperation with your colleagues and get the bonus
🔸Educational programs: regular internal training sessions, compensation for external education, attendance of specialized global conferences
Rewards program for mentoring and coaching colleagues
🔸Free internal English courses
🔸In-house Travel Service
🔸Multiple internal activities: online platform for employees with quests, gamification, presents and news, PIN-UP clubs for movie / book / pets lovers and more
🔸Other benefits could be added based on your location
 

We are looking for a Senior Data Engineer to join and strengthen data engineering practice on a large-scale, high-impact project within the travel and hospitality domain. In this role, you will work hands-on with complex data ecosystems, taking responsibility for the stability, scalability, and performance of mission-critical data integrations.

Requirements:

• 5+ years experience in Python and proven experience working with large-scale datasets.
• Experience with PMS integrations/Hospitality domain
• Solid background in designing, building, and maintaining data processing pipelines.
• Experience with cloud platforms (GCP, AWS, or Azure).
• Hands-on skills with SQL and data storage/querying systems (e.g., BigQuery, BigTable, or similar).
• Knowledge of containerization and orchestration tools (Docker, Kubernetes).
• Ability to troubleshoot and debug complex technical issues in distributed systems.
• Strong communication skills in English, with the ability to explain technical details to both technical and non-technical stakeholders.
• Experience using AI coding assistants (e.g., Cursor, GitHub Copilot, or similar) in day-to-day development tasks.
• Experience with Google Cloud services such as Pub/Sub, Dataflow, and ML-driven data workflows.

Tasks:

• Maintain and enhance existing data integrations, ensuring the reliability, accuracy, and quality of incoming data.
• Lead the investigation and resolution of complex incidents by performing deep technical analysis and debugging.
• Communicate effectively with stakeholders (including customer-facing teams and external partners) by providing transparent and timely updates.
• Collaborate with partners to troubleshoot integration issues and ensure smooth data flow.
• Identify opportunities to improve processes, tooling, and documentation to scale and streamline data operations.

• Contribute to the design and delivery of new data engineering solutions supporting business-critical systems.

   

The company offers:

• Reimbursement of medical expenses
• Compensation for trainings, seminars, conferences
• Free access to the Pluralsight and ACloudGuru knowledge base
• Engagement in our Mentorship Hub program as a mentor or a mentee to foster professional growth and development

The company is a healthcare data analytics provider offering modern data infrastructure, reporting, and AI‑driven insights. Their technology focuses on improving how healthcare organizations manage, analyze, and use clinical data.

Core Solutions

•  Healthcare data platform: Consolidates structured and unstructured patient data in near real time to create a unified data foundation.

•  Longitudinal clinical data ecosystem: Enables comprehensive management and analysis of patient information.

•  Patient engagement platform: Integrates data from multiple sources to support secure communication and holistic patient care.

•  Population health management suite: Aggregates clinical and claims data, identifies at‑risk groups, and supports interventions to improve outcomes and reduce costs.

•  Care coordination and analytics tools: Provide a complete view of patient data to enhance quality, reduce costs, and support risk stratification.

•  AI and analytics services: Transform healthcare data into actionable insights for better care and operational efficiency.

Tech stack includes:

• SQL

• Python

• Data Warehousing

• ETL

• Snowflake and/or BigQuery

• AWS (Glue, Athena, Redshift is a plus)

• GCP

• Git

Conditions:

• Compensation: 5500–6500 USD/month

• Contract: B2B

• Remote‑friendly

• Project Description:

  We are seeking a skilled, hands-on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real-time data streaming solutions.

   

• Responsibilities:

  - Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.
  - Develop, optimize, and maintain PySpark applications within the Databricks environment.
  - Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.
  - Write and maintain unit tests to ensure high code quality and reliability.
  - Perform data validation and quality checks to ensure compliance with defined business rules
  - Support and manage production deployments, ensuring system stability and performance.
  - Troubleshoot and resolve issues across development, testing, and production environments.
  - Collaborate with Product, QA, DevOps, and healthcare domain teams to ensure accurate implementation.
  - Maintain technical documentation for code, workflows, and deployment processes.
  - Pull from and push code to GitHub repositories, following version control and branching best practices.

   

• Mandatory Skills Description:

  - 5+ years of experience in data engineering roles
  - Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).
  - 4+ years of experience with Databricks data pipelines
  - Strong hands-on experience with PySpark and/or Apache Spark.
  - Experience working with CI/CD pipelines and DevOps best practices.
  - Proficiency with CI/CD, GitHub (pull requests, branching, code reviews).
  - Experience writing unit tests and performing data validation
  - Experience supporting production systems and deployments
  - Strong analytical and troubleshooting skills.

Client

Our client is one of the world’s top 20 investment companies headquartered in Great Britain, with branch offices in the US, Asia, and Europe.

Project overview

The company’s IT environment is constantly growing, with around 30 programs and more than 60 active projects. They are building a data marketplace that aggregates and analyzes data from multiple sources such as stock exchanges, news feeds, brokers, and internal quantitative systems.

As the company moves to a new data source, the main goal of this project is to create a golden source of data for all downstream systems and applications. The team is performing classic ELT/ETL: transforming raw data from multiple sources (third-party and internal) and creating a single interface for delivering data to downstream applications.

Position overview

We are looking for a Data Engineer with strong expertise in DBT, Snowflake, and modern data engineering practices. In this role, you will design and implement scalable data models, build robust ETL/ELT pipelines, and ensure high-quality data delivery for critical investment management applications.

Responsibilities

• Design, build, and deploy DBT Cloud models.
• Design, build, and deploy Airflow jobs (Astronomer).
• Identify and test for bugs and bottlenecks in the ELT/ETL solution.

Requirements

• 5+ years of experience in software engineering (GIT, CI/CD, Shell scripting).
• 3+ years of experience building scalable and robust Data Platforms (SQL, DWH, Distributed Data Processing).
• 2+ years of experience developing in DBT Core/Cloud.
• 2+ years of experience with Snowflake.
• 2+ years of experience with Airflow.
• 2+ years of experience with Python.
• Good spoken English.

Nice to have

• Proficiency in message queues (Kafka).
• Experience with cloud services (Azure).
• CI/CD knowledge (Jenkins, Groovy scripting).