Jobs Data Engineer

Are you passionate about building large-scale cloud data infrastructure that makes a real difference? We are looking for a Senior Data Engineer to join our team and work on an impactful healthcare technology project. This role offers a remote work format with the flexibility to collaborate across international teams.

At Sigma Software, we deliver innovative IT solutions to global clients in multiple industries, and we take pride in projects that improve lives. Joining us means working with cutting-edge technologies, contributing to meaningful initiatives, and growing in a supportive environment.

CUSTOMER
Our client is a leading medical technology company. Its portfolio of products, services, and solutions is at the center of clinical decision-making and treatment pathways. Patient-centered innovation has always been, and will always be, at the core of the company. The client is committed to improving patient outcomes and experiences, regardless of where patients live or what they face. The Customer is innovating sustainably to provide healthcare for everyone, everywhere. 

PROJECT
The project focuses on building and maintaining large-scale cloud-based data infrastructure for healthcare applications. It involves designing efficient data pipelines, creating self-service tools, and implementing microservices to simplify complex processes. The work will directly impact how healthcare providers access, process, and analyze critical medical data, ultimately improving patient care.

Responsibilities:

• Collaborate with the Product Owner and team leads to define and design efficient pipelines and data schemas
• Build and maintain infrastructure using Terraform for cloud platforms
• Design and implement large-scale cloud data infrastructure, self-service tooling, and microservices
• Work with large datasets to optimize performance and ensure seamless data integration
• Develop and maintain squad-specific data architectures and pipelines following ETL and Data Lake principles
• Discover, analyze, and organize disparate data sources into clean, understandable schemas

Requirements:

• Possibility to meet clients annually in Europe
• Hands-on experience with cloud computing services in data and analytics
• Experience with data modeling, reporting tools, data governance, and data warehousing
• Proficiency in Python and PySpark for distributed data processing
• Experience with Azure, Snowflake, and Databricks
• Experience with Docker and Kubernetes
• Knowledge of infrastructure as code (Terraform)
• Advanced SQL skills and familiarity with big data databases such as Snowflake, Redshift, etc.
• Experience with stream processing technologies such as Kafka, Spark Structured Streaming
• At least an Upper-Intermediate level of English 

Чи захоплюєтесь ви створенням масштабної хмарної інфраструктури даних, яка має реальний вплив? Ми шукаємо Senior Data Engineer, який приєднається до нашої команди та працюватиме над важливим проєктом у сфері медичних технологій. Ця роль передбачає віддалений формат роботи з можливістю співпрацювати з міжнародними командами.

У Sigma Software ми створюємо інноваційні ІТ-рішення для глобальних клієнтів у різних галузях і пишаємося проєктами, які покращують життя людей. Приєднавшись до нас, ви працюватимете з передовими технологіями, робитимете внесок у значущі ініціативи та розвиватиметеся в підтримуючому середовищі.

Клієнт:
Наш клієнт — провідна компанія у сфері медичних технологій. Її портфель продуктів, сервісів та рішень знаходиться в центрі процесів клінічного прийняття рішень і побудови шляхів лікування. Інновації, орієнтовані на пацієнтів, завжди були і залишаються основою діяльності компанії. Клієнт прагне покращувати результати лікування та досвід пацієнтів незалежно від того, де вони живуть і з якими викликами стикаються. Компанія інноваційно розвивається, щоб забезпечити доступ до медицини для кожного, всюди.

Проєкт:
Проєкт зосереджений на створенні та підтримці масштабної хмарної інфраструктури даних для медичних застосунків. Він включає проєктування ефективних data pipelines, створення self-service інструментів та реалізацію мікросервісів для спрощення складних процесів. Робота безпосередньо впливатиме на те, як медичні установи отримують доступ до критично важливих медичних даних, обробляють їх та аналізують, що зрештою покращує якість медичної допомоги пацієнтам.

 

Обов'язки: 

• Співпрацювати з Product Owner та технічними лідами для визначення та проєктування ефективних pipelines і схем даних
• Будувати та підтримувати інфраструктуру для хмарних платформ за допомогою Terraform
• Проєктувати та впроваджувати масштабну хмарну інфраструктуру даних, self-service інструменти та мікросервіси
• Працювати з великими наборами даних для оптимізації продуктивності та забезпечення безперебійної інтеграції даних
• Розробляти та підтримувати архітектури даних і pipelines для команди відповідно до принципів ETL та Data Lake
• Виявляти, аналізувати та впорядковувати різнорідні джерела даних у зрозумілі та чисті схеми

Вимоги:

• Можливість раз на рік зустрічатися з клієнтами в Європі
• Практичний досвід роботи з хмарними сервісами для даних та аналітики
• Досвід data modeling, reporting tools, data governance та data warehousing
• Впевнене володіння Python і PySpark для розподіленої обробки даних
• Досвід роботи з Azure, Snowflake та Databricks
• Досвід роботи з Docker та Kubernetes
• Знання infrastructure as code (Terraform)
• Просунуті навички SQL та знайомство з big data базами даних, такими як Snowflake, Redshift тощо
• Досвід роботи зі stream processing технологіями, такими як Kafka, Spark Structured Streaming
• Рівень англійської не нижче Upper-Intermediate

Our customer is one of the world's largest food and beverage companies, operating in 189 countries with over 2000 brands. The company's diverse portfolio includes products such as bottled water, dairy products, breakfast cereals, coffee, confectionery, frozen food, pet foods, snacks, and baby food.

We are looking for a Middle Data Engineer to design, build, and maintain scalable data systems that support analytics, reporting, and machine learning use cases. This role requires strong ownership, solid SQL skills, and experience working with large datasets in production environments.

Responsibilities

• Build and maintain data pipelines (batch and near real-time).
• Develop and optimize data models for analytics and BI tools.
• Write efficient SQL queries for large-scale datasets.
• Support development and maintenance of ETL/ELT processes.
• Ensure data quality, integrity, and consistency across systems.
• Collaborate with data analysts, data scientists, and product teams to understand data requirements.
• Monitor and troubleshoot data pipelines and workflows.
• Contribute to improving the performance of the existing data infrastructure.
• Work with cloud-based data platforms and modern data stack tools.

Requirements

• 2–4 years of experience in Data Engineering or a related role.
• Main technical skills are: Snowflake, dbt, ADF.
• Solid SQL skills (joins, aggregations, window functions).
• Understanding of data governance and data quality concepts.
• Proficiency in Python.
• Experience with ETL/ELT tools.
• Hands-on experience with the Azure cloud platform.
• Azure Ops experience: provisioning and configuring.
• Experience working with data warehouses (e.g., Snowflake, BigQuery, Redshift).
• Understanding of data modeling concepts (e.g., star schema).

Nice to have

• Basic knowledge of Docker or containerization.
• Experience with data processing frameworks (e.g., Spark).

Soft skills

• Strong analytical thinking and problem-solving skills.
• Ability to work independently with some guidance.
• Good communication skills with technical and non-technical stakeholders.
• Willingness to learn and grow in a collaborative environment.

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers

The Digital Health organization is technology team which focused on next generation Digital Health capabilities which deliver on the Medicine mission and vision to deliver Insight Driven Care. This role will operate within the Digital Health Applications & Interoperability subgroup of the broader Digital Health team, focused on patient engagement, care coordination, AI, healthcare analytics & interoperability amongst other advanced technologies which enhance our product portfolio with new services, while improving clinical & patient experiences.

 The project is a cloud-based PaaS Ecosystem built with a privacy by design centric approach to provide a centralized cloud-based platform to store, classify, and control access to federated datasets in a scalable, secure, and efficient manner.

The ecosystem will allow Customer Operating Units (medical device departments) to store federated data sets of varying sizes and formats and control access to those data sets through Data steward(s). Source data sets can be exposed to qualified use cases and workflows through different project types.

The Healthcare Data Platform ecosystem will provide ML/AI project capabilities for streamlined development processes and a ML/AI workbench to enhance data exploration, wrangling, and model training.

In queue: 15+ OU’s. At this moment focused on – Nuero, Cardio, Diabetes is the OU that data platform is working with, but there could be more OU’s coming up with requirements in future.

GL Role: is to work on the enhancement of current capabilities, including taking over the work that AWS proserve team is doing, and develop new requirements that will keep coming from different OU’s in the future.
 

Requirements:
 

• AWS Platform: Working experience with AWS data technologies, including S3, AWS RDS, and Lake Formation
• Programming Languages: Strong programming skills in Python
• Data Formats: Experience with JSON, XML, and other relevant data formats
• CI/CD Tools: Ability to deploy using established CI/CD pipelines using GitLab CI, Jenkins, Terraform or similar tools
• Scripting and automation: experience in scripting languages such as Python, PowerShell, etc.
• Monitoring and Logging: Familiarity with monitoring & logging tools like CloudWatch, Splunk, ELK, Dynatrace, Prometheus
• Source Code Management: Expertise with GitLab

• Documentation: Experience with markdown and in particula,r Antora for creating technical documentation
   

  NICE TO HAVE:
   

• Previous Healthcare or Medical Device experience
• Experience implementing enterprise grade cyber security & privacy by design into software products
• Experience working in Digital Health software
• Experience developing global applications
• Strong understanding of SDLC;
• Experience with Agile methodologies
• Software estimation
• Experience leading software development teams onshore and offshore
• Experience with FHIR
   

Job responsibilities:
 

• Implement data pipelines using AWS services such as AWS Glue, Lambda, Kinesis, etc
• Implement integrations between the data platform and systems such as Atlan, Trino/Starburst, etc
• Complete logging and monitoring tasks through AWS and Splunk toolsets
• Develop and maintain ETL processes to ingest, clean, transform and store healthcare data from various sources
• Optimize data storage solutions using Amazon S3, AWS RDS, Lake Formation and other AWS technologies.
• Document, configure, and maintain systems specifications that conform to defined architecture standards, address business requirements, and processes in the cloud development & engineering.
• Participate in planning of system and development deployment as well as responsible for meeting compliance and security standards.
• Actively identify system functionality or performance deficiencies, execute changes to existing systems, and test functionality of the system to correct deficiencies and maintain more effective data handling, data integrity, conversion, input/output requirements, and storage.
• Document testing and maintenance of system updates, modifications, and configurations.
• Leverage platform process expertise to assess if existing standard platform functionality will solve a business problem or customization solution would be required.
• Test the quality of a product and its ability to perform a task or solve a problems.
• Perform basic maintenance and performance optimization procedures in each of the primary operating systems.
• Ensure system implementation compliance with global & local regulatory and security standards (i.e. HIPAA, SOCII, ISO27001, etc…)

Project Overview

UADAMAGE is hiring a Data Engineer to join a cross-functional team working on a national-scale Big Data platform for Ukraine. The project focuses on building a geospatial platform for multifactor analysis of territories to support demining and recovery planning. The system integrates remote sensing, environmental and socio-economic indicators, and includes both Big Data infrastructure and AI/ML modules. The platform is under active development with support from governance and international partners.

Role Summary

Design and maintain scalable and sustainable data pipelines within the platform to ensure timely, accurate, and consistent integration of multi-source data. Develop and support workflows for data ingestion, cleansing, normalization, and transformation from internal and external sources (e.g. government systems, commercial registers, API feeds). Ensure compatibility of all data pipelines with downstream use in GIS systems, land analysis models, dashboards and reports. Execute basic data analysis. Collaborate with product, analytics, and engineering teams to maintain data quality and feasibility for decision-support tools. Support data architecture evolution, implement data processing best practices, and contribute to technical documentation.

Qualifications required

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Data Engineering, or related fields.
• 3+ years of experience in data engineering or data platform development.
• Proven experience in building and maintaining data pipelines (ETL/ELT), performing data cleansing, normalization, and transformation of structured and unstructured datasets.
• Hands-on experience with technologies such as Python, PySpark, Scala, SQL, Pandas, NumPy and tools including Apache Spark, Hadoop, and Amazon EMR.
• Experience working with AWS-based platforms, including S3 and cloud-native data processing services.
• Practical knowledge of streaming frameworks and distributed data processing patterns, such as MapReduce and real-time cloud pipelines.
• Experience integrating data from diverse sources (relational databases, APIs, satellite/geospatial data, file systems) for use in analytical models and data management platforms.
• Understanding of data integration strategies, API/data services, and long-term system maintainability.
• Familiarity with best practices in data documentation, versioning, metadata tracking, and validation.
• Familiarity with geospatial data formats, geo coordinate systems, and spatial preprocessing is an asset.
• Experience working in complex or multi-stakeholder environments is an asset.

• Proficiency in Ukrainian and English is required.

   

We offer

• Competitive salary
• Motivated and engaged team
• Open communication and active knowledge sharing
• Direct support from project leadership
• Clear specifications and well-defined requirements
• Opportunity to contribute to a high-impact, nationally relevant initiative

Project Description:

We are seeking a skilled, hands-on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real-time data streaming solutions.

Responsibilities:

- Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.
- Develop, optimize, and maintain PySpark applications within the Databricks environment.
- Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.
- Write and maintain unit tests to ensure high code quality and reliability.
- Perform data validation and quality checks to ensure compliance with defined business rules
- Support and manage production deployments, ensuring system stability and performance.
- Troubleshoot and resolve issues across development, testing, and production environments.
- Collaborate with Product, QA, DevOps, and healthcare domain teams to ensure accurate implementation.
- Maintain technical documentation for code, workflows, and deployment processes.
- Pull from and push code to GitHub repositories, following version control and branching best practices.

Mandatory Skills Description:

- 5+ years of experience in data engineering roles
- Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).
- 4+ years of experience with Databricks data pipelines
- Strong hands-on experience with PySpark and/or Apache Spark.
- Experience working with CI/CD pipelines and DevOps best practices.
- Proficiency with CI/CD, GitHub (pull requests, branching, code reviews).
- Experience writing unit tests and performing data validation
- Experience supporting production systems and deployments
- Strong analytical and troubleshooting skills.

Nice-to-Have Skills Description:

- Knowledge of FHIR (Fast Healthcare Interoperability Resources) standards.
- Experience with healthcare data integration projects.
- Familiarity with REST APIs and FHIR server integrations.
- Experience in Agile/Scrum development environments.

Languages:

English: C1 Advanced

RedCore is an international business group that creates technological solutions for digital markets. Our products and services cover fintech, marketing, e-commerce, customer service, communications and regulatory technologies. 

We are looking for a Data Engineer to join our team!

Requirements:
- 2+ years of experience in the role of data engineer;

- Experience in designing and implementing data processing systems;

- Good knowledge of Python basics (including SOLID design principles);

- HTTP;

- Git (GitLab);
 

Will be plus:

- Columnstore DBs (Clickhouse);

- Airflow;

Responsibilities:
- Communication with clients on the task requirements;

- Write high performance, testable and maintainable code to implement new functionality;

- Take part in technical discussions to come up with solutions for challenging issues;

- Participate in code reviews to ensure code quality and distribute knowledge.

Our benefits to you:
🔸An exciting and challenging job in a fast-growing holding, the opportunity to be part of a multicultural team of top professionals in Development, Architecture, Management, Operations, Marketing, Legal, Finance and more
Great working atmosphere with passionate experts and leaders, sharing a friendly culture and a success-driven mindset is guaranteed
🔸Modern corporate equipment based on macOS or Windows and additional equipment are provided
🔸Paid vacations, sick leave, personal events days, days off
🔸Referral program — enjoy cooperation with your colleagues and get the bonus
🔸Educational programs: regular internal training sessions, compensation for external education, attendance of specialized global conferences
Rewards program for mentoring and coaching colleagues
🔸Free internal English courses
🔸In-house Travel Service
🔸Multiple internal activities: online platform for employees with quests, gamification, presents and news, PIN-UP clubs for movie / book / pets lovers and more
🔸Other benefits could be added based on your location
 

We are looking for a Senior Data Engineer to join and strengthen data engineering practice on a large-scale, high-impact project within the travel and hospitality domain. In this role, you will work hands-on with complex data ecosystems, taking responsibility for the stability, scalability, and performance of mission-critical data integrations.

Requirements:

• 5+ years experience in Python and proven experience working with large-scale datasets.
• Experience with PMS integrations/Hospitality domain
• Solid background in designing, building, and maintaining data processing pipelines.
• Experience with cloud platforms (GCP, AWS, or Azure).
• Hands-on skills with SQL and data storage/querying systems (e.g., BigQuery, BigTable, or similar).
• Knowledge of containerization and orchestration tools (Docker, Kubernetes).
• Ability to troubleshoot and debug complex technical issues in distributed systems.
• Strong communication skills in English, with the ability to explain technical details to both technical and non-technical stakeholders.
• Experience using AI coding assistants (e.g., Cursor, GitHub Copilot, or similar) in day-to-day development tasks.
• Experience with Google Cloud services such as Pub/Sub, Dataflow, and ML-driven data workflows.

Tasks:

• Maintain and enhance existing data integrations, ensuring the reliability, accuracy, and quality of incoming data.
• Lead the investigation and resolution of complex incidents by performing deep technical analysis and debugging.
• Communicate effectively with stakeholders (including customer-facing teams and external partners) by providing transparent and timely updates.
• Collaborate with partners to troubleshoot integration issues and ensure smooth data flow.
• Identify opportunities to improve processes, tooling, and documentation to scale and streamline data operations.

• Contribute to the design and delivery of new data engineering solutions supporting business-critical systems.

   

The company offers:

• Reimbursement of medical expenses
• Compensation for trainings, seminars, conferences
• Free access to the Pluralsight and ACloudGuru knowledge base
• Engagement in our Mentorship Hub program as a mentor or a mentee to foster professional growth and development

The company is a healthcare data analytics provider offering modern data infrastructure, reporting, and AI‑driven insights. Their technology focuses on improving how healthcare organizations manage, analyze, and use clinical data.

Core Solutions

•  Healthcare data platform: Consolidates structured and unstructured patient data in near real time to create a unified data foundation.

•  Longitudinal clinical data ecosystem: Enables comprehensive management and analysis of patient information.

•  Patient engagement platform: Integrates data from multiple sources to support secure communication and holistic patient care.

•  Population health management suite: Aggregates clinical and claims data, identifies at‑risk groups, and supports interventions to improve outcomes and reduce costs.

•  Care coordination and analytics tools: Provide a complete view of patient data to enhance quality, reduce costs, and support risk stratification.

•  AI and analytics services: Transform healthcare data into actionable insights for better care and operational efficiency.

Tech stack includes:

• SQL

• Python

• Data Warehousing

• ETL

• Snowflake and/or BigQuery

• AWS (Glue, Athena, Redshift is a plus)

• GCP

• Git

Conditions:

• Compensation: 5500–6500 USD/month

• Contract: B2B

• Remote‑friendly

We are seeking a skilled Data Engineer to join our team and contribute to the development of large- scale analytics platforms. The ideal candidate will have strong experience in cloud ecosystems such as Azure and AWS, as well as expertise in AI and machine learning applications. Knowledge of the healthcare industry and life sciences is a plus.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for large-scale analytics platforms.
• Implement cloud-based solutions using Azure and AWS, ensuring reliability and performance.
• Work closely with data scientists and AI/ML teams to optimize data workflows.
• Ensure data quality, governance, and security across platforms.
• Collaborate with cross-functional teams to integrate data solutions into business processes.

Required Qualifications

• Bachelor's degree (or higher) in Computer Science, Engineering, or a related field.
• 3+ years of experience in data engineering, big data processing, and cloud-based architecture.
• Strong proficiency in cloud services (Azure, AWS) and distributed computing frameworks.
• Mandatory hands-on experience with Databricks (UC, DLTs, Delta Sharing, etc.)
• Expertise in SQL and database management systems (SQL Server, MySQL, etc.).
• Experience with data modeling, ETL processes, and data warehousing solutions.
• Knowledge of AI and machine learning concepts and their data requirements.
• Proficiency in Python, Scala, or similar programming languages.
• Basic knowledge of C# and/or Java programming.
• Familiarity with DevOps, CI/CD pipelines.
• High-level proficiency in English (written and spoken).

Preferred Qualifications

• Experience in the healthcare or life sciences industry.
• Understanding of regulatory compliance related to healthcare data (HIPAA, GDPR, etc.).
• Familiarity with interoperability standards such as HL7, FHIR, and EDI.

We are looking for a specialist to design and implement an end-to-end performance testing framework for a healthcare system running on Databricks and Microsoft Azure. You will build a repeatable, automated approach to measure and improve performance across data ingestion, ETL/ELT pipelines, Spark workloads, serving layers, APIs, security/identity flows, integration components, and presentation/UI, while meeting healthcare-grade security and compliance expectations.

This role sits at the intersection of performance engineering, cloud architecture, and test automation, with strong attention to regulated-domain requirements (privacy, auditability, access controls).

Key Responsibilities

• Design and build a performance testing strategy and framework for a Databricks + Azure healthcare platform.
• Define performance KPIs/SLOs (e.g., pipeline latency, throughput, job duration, cluster utilization, cost per run, data freshness).
• Create workload models that reflect production usage (batch, streaming, peak loads, concurrency, backfills).
• Create a test taxonomy: smoke perf, baseline benchmarks, load, stress, soak/endurance, spike tests, and capacity planning.
  
   
• Implement automated performance test suites for:
  • Databricks jobs/workflows (Workflows, Jobs API)
  • Spark/Delta Lake operations (reads/writes, mergers, compaction, Z-Ordering where relevant)
  • Data ingestion (ADF, Event Hubs, ADLS Gen2, Autoloader, etc. as applicable)
• Build test data generation and data anonymization/synthetic data approaches suitable for healthcare contexts.
• Instrument, collect, and analyze metrics from:
  • Spark UI / event logs
  • Databricks metrics and system tables
  • Azure Monitor / Log Analytics
  • Application logs and telemetry (if applicable)
• Produce actionable performance reports and dashboards (trend, regression detection, run-to-run comparability).
• Create performance tests for key user journeys (page load, search, dashboards) using appropriate tooling.
• Measure client-side and network timings and correlate them with API/backend performance.
• Integrate performance tests into CI/CD (Azure DevOps or GitHub Actions), including gating rules and baselines.
• Document framework usage, standards, and provide enablement to engineering teams.

Required Qualifications

• Proven experience building performance testing frameworks (not just executing tests), ideally for data platforms.
• Strong hands-on expertise with Databricks and Apache Spark performance tuning and troubleshooting.
• Strong knowledge of Azure services used in data platforms (commonly ADLS Gen2, ADF, Key Vault, Azure Monitor/Log Analytics; others as relevant).
• Strong programming/scripting ability in Python and/or Java/TypeScript.
• Familiarity with load/performance tools and approaches (e.g., custom harnesses, Locust/JMeter/k6 where appropriate, or Spark-specific benchmarking).
• Ability to design repeatable benchmarking (baseline creation, environment parity, noise reduction, statistical comparison).
• Understanding of data security and compliance needs typical for healthcare (e.g., HIPAA-like controls, access management, auditability; adapt to your jurisdiction).
• High-level proficiency in English

Nice-to-Have / Preferred

• Experience with Delta Lake optimization (OPTIMIZE, ZORDER, liquid clustering where applicable), streaming performance, and structured streaming.
• Experience with Terraform/IaC for reproducible test environments.
• Knowledge of Unity Catalog, data governance, and fine-grained access controls.
• Experience with OpenTelemetry tracing and correlation across UI → API → data workloads.
• FinOps mindset: performance improvements tied to cost efficiency on Databricks/Azure.
• Prior work on regulated domains (healthcare, pharma, insurance).

 Working Model

• Contract
• Remote
• Collaboration with Data Engineering, Platform Engineering, Security/Compliance, and Product teams.

• Project Description:

  We are seeking a skilled, hands-on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real-time data streaming solutions.

   

• Responsibilities:

  - Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.
  - Develop, optimize, and maintain PySpark applications within the Databricks environment.
  - Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.
  - Write and maintain unit tests to ensure high code quality and reliability.
  - Perform data validation and quality checks to ensure compliance with defined business rules
  - Support and manage production deployments, ensuring system stability and performance.
  - Troubleshoot and resolve issues across development, testing, and production environments.
  - Collaborate with Product, QA, DevOps, and healthcare domain teams to ensure accurate implementation.
  - Maintain technical documentation for code, workflows, and deployment processes.
  - Pull from and push code to GitHub repositories, following version control and branching best practices.

   

• Mandatory Skills Description:

  - 5+ years of experience in data engineering roles
  - Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).
  - 4+ years of experience with Databricks data pipelines
  - Strong hands-on experience with PySpark and/or Apache Spark.
  - Experience working with CI/CD pipelines and DevOps best practices.
  - Proficiency with CI/CD, GitHub (pull requests, branching, code reviews).
  - Experience writing unit tests and performing data validation
  - Experience supporting production systems and deployments
  - Strong analytical and troubleshooting skills.

Client

Our client is one of the world’s top 20 investment companies headquartered in Great Britain, with branch offices in the US, Asia, and Europe.

Project overview

The company’s IT environment is constantly growing, with around 30 programs and more than 60 active projects. They are building a data marketplace that aggregates and analyzes data from multiple sources such as stock exchanges, news feeds, brokers, and internal quantitative systems.

As the company moves to a new data source, the main goal of this project is to create a golden source of data for all downstream systems and applications. The team is performing classic ELT/ETL: transforming raw data from multiple sources (third-party and internal) and creating a single interface for delivering data to downstream applications.

Position overview

We are looking for a Data Engineer with strong expertise in DBT, Snowflake, and modern data engineering practices. In this role, you will design and implement scalable data models, build robust ETL/ELT pipelines, and ensure high-quality data delivery for critical investment management applications.

Responsibilities

• Design, build, and deploy DBT Cloud models.
• Design, build, and deploy Airflow jobs (Astronomer).
• Identify and test for bugs and bottlenecks in the ELT/ETL solution.

Requirements

• 5+ years of experience in software engineering (GIT, CI/CD, Shell scripting).
• 3+ years of experience building scalable and robust Data Platforms (SQL, DWH, Distributed Data Processing).
• 2+ years of experience developing in DBT Core/Cloud.
• 2+ years of experience with Snowflake.
• 2+ years of experience with Airflow.
• 2+ years of experience with Python.
• Good spoken English.

Nice to have

• Proficiency in message queues (Kafka).
• Experience with cloud services (Azure).
• CI/CD knowledge (Jenkins, Groovy scripting).

Project description

We are seeking a skilled, hands-on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real-time data streaming solutions.

Responsibilities

Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.

Develop, optimize, and maintain PySpark applications within the Databricks environment.

Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.

Write and maintain unit tests to ensure high code quality and reliability.

Perform data validation and quality checks to ensure compliance with defined business rules

Support and manage production deployments, ensuring system stability and performance.

Troubleshoot and resolve issues across development, testing, and production environments.

Collaborate with Product, QA, DevOps, and healthcare domain teams to ensure accurate implementation.

Maintain technical documentation for code, workflows, and deployment processes.

Pull from and push code to GitHub repositories, following version control and branching best practices.

Skills

Must have

5+ years of experience in data engineering roles

Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).

4+ years of experience with Databricks data pipelines

Strong hands-on experience with PySpark and/or Apache Spark.

Experience working with CI/CD pipelines and DevOps best practices.

Proficiency with CI/CD, GitHub (pull requests, branching, code reviews).

Experience writing unit tests and performing data validation

Experience supporting production systems and deployments

Strong analytical and troubleshooting skills.

Nice to have

Knowledge of FHIR (Fast Healthcare Interoperability Resources) standards.

Experience with healthcare data integration projects.

Familiarity with REST APIs and FHIR server integrations.

Experience in Agile/Scrum development environments.

Languages

English: C1 Advanced

We are seeking a skilled, hands-on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real-time data streaming solutions.

• Responsibilities:

  - Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.
  - Develop, optimize, and maintain PySpark applications within the Databricks environment.
  - Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.
  - Write and maintain unit tests to ensure high code quality and reliability.
  - Perform data validation and quality checks to ensure compliance with defined business rules
  - Support and manage production deployments, ensuring system stability and performance.
  - Troubleshoot and resolve issues across development, testing, and production environments.
  - Collaborate with Product, QA, DevOps, and healthcare domain teams to ensure accurate implementation.
  - Maintain technical documentation for code, workflows, and deployment processes.
  - Pull from and push code to GitHub repositories, following version control and branching best practices.

• Mandatory Skills Description:

  - 5+ years of experience in data engineering roles
  - Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).
  - 4+ years of experience with Databricks data pipelines
  - Strong hands-on experience with PySpark and/or Apache Spark.
  - Experience working with CI/CD pipelines and DevOps best practices.
  - Proficiency with CI/CD, GitHub (pull requests, branching, code reviews).
  - Experience writing unit tests and performing data validation
  - Experience supporting production systems and deployments
  - Strong analytical and troubleshooting skills.

• Nice-to-Have Skills Description:

  - Knowledge of FHIR (Fast Healthcare Interoperability Resources) standards.
  - Experience with healthcare data integration projects.
  - Familiarity with REST APIs and FHIR server integrations.
  - Experience in Agile/Scrum development environments.

Project Description:

We are seeking a skilled, hands-on Data Engineer with demonstrated experience in Databricks, Apache Spark, Python, and real-time data streaming solutions.

Responsibilities:

- Implement resource composition logic provided by the Product team into scalable and efficient PySpark code.

- Develop, optimize, and maintain PySpark applications within the Databricks environment.

- Utilize and maintain existing CI/CD pipelines in Databricks for automated build, testing, and deployment.

- Write and maintain unit tests to ensure high code quality and reliability.

- Perform data validation and quality checks to ensure compliance with defined business rules

- Support and manage production deployments, ensuring system stability and performance.

- Troubleshoot and resolve issues across development, testing, and production environments.

- Collaborate with Product, QA, DevOps, and healthcare domain teams to ensure accurate implementation.

- Maintain technical documentation for code, workflows, and deployment processes.

- Pull from and push code to GitHub repositories, following version control and branching best practices.

Mandatory Skills Description:

- 5+ years of experience in data engineering roles

- Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).

- 4+ years of experience with Databricks data pipelines

- Strong hands-on experience with PySpark and/or Apache Spark.

- Experience working with CI/CD pipelines and DevOps best practice.

 Proficiency with CI/CD, GitHub (pull requests, branching, code reviews).

- Experience writing unit tests and performing data validation

- Experience supporting production systems and deployments

- Strong analytical and troubleshooting skills.

Nice-to-Have Skills Description:

- Knowledge of FHIR (Fast Healthcare Interoperability Resources) standards.

- Experience with healthcare data integration projects.

- Familiarity with REST APIs and FHIR server integrations.

- Experience in Agile/Scrum development environments.