Jobs Data Engineer

Join a Company That Invests in You

Seeking Alpha is the world’s leading community of engaged investors. We’re the go-to destination for investors looking for actionable stock market opinions, real-time market analysis, and unique financial insights. At the same time, we’re also dedicated to creating a workplace where our team thrives. We’re passionate about fostering a flexible, balanced environment with remote work options and an array of perks that make a real difference.

Here, your growth matters. We prioritize your development through ongoing learning and career advancement opportunities, helping you reach new milestones. Join Seeking Alpha to be part of a company that values your unique journey, supports your success, and champions both your personal well-being and professional goals.

What We're Looking For

Seeking Alpha is looking for a Senior Data Engineer responsible for designing, building, and maintaining the infrastructure necessary for analyzing large data sets. This individual should be an expert in data management, ETL (extract, transform, load) processes, and data warehousing and should have experience working with various big data technologies, such as Hadoop, Spark, and NoSQL databases. In addition to technical skills, a Senior Data Engineer should have strong communication and collaboration abilities, as they will be working closely with other members of the data and analytics team, as well as other stakeholders, to identify and prioritize data engineering projects and to ensure that the data infrastructure is aligned with the overall business goals and objectives.

What You'll Do

• Work closely with data scientists/analytics and other stakeholders to identify and prioritize data engineering projects and to ensure that the data infrastructure is aligned with business goals and objectives
• Design, build and maintain optimal data pipeline architecture for extraction, transformation, and loading of data from a wide variety of data sources, including external APIs, data streams, and data stores. 
• Continuously monitor and optimize the performance and reliability of the data infrastructure, and identify and implement solutions to improve scalability, efficiency, and security
• Stay up-to-date with the latest trends and developments in the field of data engineering, and leverage this knowledge to identify opportunities for improvement and innovation within the organization
• Solve challenging problems in a fast-paced and evolving environment while maintaining uncompromising quality.
• Implement data privacy and security requirements to ensure solutions comply with security standards and frameworks.
• Enhance the team's dev-ops capabilities.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• 2+ years of proven experience developing large-scale software using an object-oriented or functional language.
• 5+ years of professional experience in data engineering, focusing on building and maintaining data pipelines and data warehouses
• Strong experience with Spark, Scala, and Python, including the ability to write high-performance, maintainable code
• Experience with AWS services, including EC2, S3, Athena, Kinesis/Firehose Lambda and EMR
• Familiarity with data warehousing concepts and technologies, such as columnar storage, data lakes, and SQL
• Experience with data pipeline orchestration and scheduling using tools such as Airflow
• Strong problem-solving skills and the ability to work independently as well as part of a team
• High-level English - a must. 
• A team player with excellent collaboration skills.

Nice to Have:

• Expertise with Vertica or Redshift, including experience with query optimization and performance tuning
• Experience with machine learning and/or data science projects
• Knowledge of data governance and security best practices, including data privacy regulations such as GDPR and CCPA.
• Knowledge of Spark internals (tuning, query optimization)

Are you passionate about building large-scale cloud data infrastructure that makes a real difference? We are looking for a Senior Data Engineer to join our team and work on an impactful healthcare technology project. This role offers a remote work format with the flexibility to collaborate across international teams.

At Sigma Software, we deliver innovative IT solutions to global clients in multiple industries, and we take pride in projects that improve lives. Joining us means working with cutting-edge technologies, contributing to meaningful initiatives, and growing in a supportive environment.

CUSTOMER
Our client is a leading medical technology company. Its portfolio of products, services, and solutions is at the center of clinical decision-making and treatment pathways. Patient-centered innovation has always been, and will always be, at the core of the company. The client is committed to improving patient outcomes and experiences, regardless of where patients live or what they face. The Customer is innovating sustainably to provide healthcare for everyone, everywhere. 

PROJECT
The project focuses on building and maintaining large-scale cloud-based data infrastructure for healthcare applications. It involves designing efficient data pipelines, creating self-service tools, and implementing microservices to simplify complex processes. The work will directly impact how healthcare providers access, process, and analyze critical medical data, ultimately improving patient care.

Responsibilities:

• Collaborate with the Product Owner and team leads to define and design efficient pipelines and data schemas
• Build and maintain infrastructure using Terraform for cloud platforms
• Design and implement large-scale cloud data infrastructure, self-service tooling, and microservices
• Work with large datasets to optimize performance and ensure seamless data integration
• Develop and maintain squad-specific data architectures and pipelines following ETL and Data Lake principles
• Discover, analyze, and organize disparate data sources into clean, understandable schemas

Requirements:

• Hands-on experience with cloud computing services in data and analytics
• Experience with data modeling, reporting tools, data governance, and data warehousing
• Proficiency in Python and PySpark for distributed data processing
• Experience with Azure, Snowflake, and Databricks
• Experience with Docker and Kubernetes
• Knowledge of infrastructure as code (Terraform)
• Advanced SQL skills and familiarity with big data databases such as Snowflake, Redshift, etc.
• Experience with stream processing technologies such as Kafka, Spark Structured Streaming
• At least an Upper-Intermediate level of English 

The project is for one of the world's famous science and technology companies in pharmaceutical industry, supporting initiatives in AWS, AI and data engineering, with plans to launch over 20 additional initiatives in the future. Modernizing the data infrastructure through the transition to Snowflake as a priority, as it will enhance capabilities for implementing advanced AI solutions and unlock numerous opportunities for innovation and growth.

We are seeking a highly skilled Snowflake Data Engineer to design, build, and optimize scalable data pipelines and cloud-based solutions across AWS, Azure, and GCP. The ideal candidate will have strong expertise in Snowflake, ETL Tools like DBT, Python, visualization tools like Tableau and modern CI/CD practices, with a deep understanding of data governance, security, and role-based access control (RBAC). Knowledge of data modeling methodologies (OLTP, OLAP, Data Vault 2.0), data quality frameworks, Stream lit application development and SAP integration and infrastructure-as-code with Terraform is essential. Experience working with different file formats such as JSON, Parquet, CSV, and XML is highly valued.

• Responsibilities:

  • Design and develop data pipelines using Snowflake and Snow pipe for real-time and batch ingestion.
  • Implement CI/CD pipelines in Azure DevOps for seamless deployment of data solutions.
  • Automate DBT jobs to streamline transformations and ensure reliable data workflows.
  • Apply data modeling techniques including OLTP, OLAP, and Data Vault 2.0 methodologies to design scalable architectures.
  • Document data models, processes, and workflows clearly for future reference and knowledge sharing.
  • Build data tests, unit tests, and mock data frameworks to validate and maintain reliability of data solutions.
  • Develop Streamlit applications integrated with Snowflake to deliver interactive dashboards and self-service analytics.
  • Integrate SAP data sources into Snowflake pipelines for enterprise reporting and analytics.
  • Leverage SQL expertise for complex queries, transformations, and performance optimization.
  • Integrate cloud services across AWS, Azure, and GCP to support multi-cloud data strategies.
  • Develop Python scripts for ETL/ELT processes, automation, and data quality checks.
  • Implement infrastructure-as-code solutions using Terraform for scalable and automated cloud deployments.
  • Manage RBAC and enforce data governance policies to ensure compliance and secure data access.
  • Collaborate with cross-functional teams including business analysts, and business stakeholders to deliver reliable data solutions.

• Mandatory Skills Description:

  • Strong proficiency in Snowflake (Snowpipe, RBAC, performance tuning).
  • Hands-on experience with Python , SQL , Jinja , JavaScript for data engineering tasks.
  • CI/CD expertise using Azure DevOps (build, release, version control).
  • Experience automating DBT jobs for data transformations.
  • Experience building Streamlit applications with Snowflake integration.
  • Cloud services knowledge across AWS (S3, Lambda, Glue), Azure (Data Factory, Synapse), and GCP (BigQuery, Pub/Sub).

• The project is for one of the world's famous science and technology companies in pharmaceutical industry, supporting initiatives in AWS, AI and data engineering, with plans to launch over 20 additional initiatives in the future. Modernizing the data infrastructure through the transition to Snowflake as a priority, as it will enhance capabilities for implementing advanced AI solutions and unlock numerous opportunities for innovation and growth.
  
  We are seeking a highly skilled Snowflake Data Engineer to design, build, and optimize scalable data pipelines and cloud-based solutions across AWS, Azure, and GCP. The ideal candidate will have strong expertise in Snowflake, ETL Tools like DBT, Python, visualization tools like Tableau and modern CI/CD practices, with a deep understanding of data governance, security, and role-based access control (RBAC). Knowledge of data modeling methodologies (OLTP, OLAP, Data Vault 2.0), data quality frameworks, Stream lit application development and SAP integration and infrastructure-as-code with Terraform is essential. Experience working with different file formats such as JSON, Parquet, CSV, and XML is highly valued.

• Responsibilities:

  • Design and develop data pipelines using Snowflake and Snow pipe for real-time and batch ingestion.
  • Implement CI/CD pipelines in Azure DevOps for seamless deployment of data solutions.
  • Automate DBT jobs to streamline transformations and ensure reliable data workflows.
  • Apply data modeling techniques including OLTP, OLAP, and Data Vault 2.0 methodologies to design scalable architectures.
  • Document data models, processes, and workflows clearly for future reference and knowledge sharing.
  • Build data tests, unit tests, and mock data frameworks to validate and maintain reliability of data solutions.
  • Develop Streamlit applications integrated with Snowflake to deliver interactive dashboards and self-service analytics.
  • Integrate SAP data sources into Snowflake pipelines for enterprise reporting and analytics.
  • Leverage SQL expertise for complex queries, transformations, and performance optimization.
  • Integrate cloud services across AWS, Azure, and GCP to support multi-cloud data strategies.
  • Develop Python scripts for ETL/ELT processes, automation, and data quality checks.
  • Implement infrastructure-as-code solutions using Terraform for scalable and automated cloud deployments.
  • Manage RBAC and enforce data governance policies to ensure compliance and secure data access.
  • Collaborate with cross-functional teams including business analysts, and business stakeholders to deliver reliable data solutions.

• Mandatory Skills Description:

  • Strong proficiency in Snowflake (Snowpipe, RBAC, performance tuning).
  • Hands-on experience with Python , SQL , Jinja , JavaScript for data engineering tasks.
  • CI/CD expertise using Azure DevOps (build, release, version control).
  • Experience automating DBT jobs for data transformations.
  • Experience building Streamlit applications with Snowflake integration.
  • Cloud services knowledge across AWS (S3, Lambda, Glue), Azure (Data Factory, Synapse), and GCP (BigQuery, Pub/Sub).

• Nice-to-Have Skills Description:

  - Cloud certifications is a plus

• Languages:
  • English: B2 Upper Intermediate

Description

We are looking for a ETL Developer to join our team and work on data integration for a Pharmaceutical Marketing company. 

You will develop and support ETL processes that run in Docker containers. Your daily work will primarily involve writing complex SQL queries and views, performing data transformations, and ensuring accurate and timely delivery of data by monitoring notifications and logs in AWS CloudWatch. Work also involves scripting in Bash and Python for automation, SFTP data transfers, and connecting to APIs when required. 

We work as a team, care about code and data quality, and like people who want to learn and improve. 

Our teams have daily standups and direct communication with a client on a daily basis. 
The platform processes sensitive data, so development is manual, controlled, and accuracy-driven rather than highly automated. 

Requirements

• 3+ years of experience working with ETL processes or data pipelines
• Strong SQL skills: creating and debugging complex queries, aggregations, and validation logic
• Experience with a relational database (preferably PostgreSQL)
• Basic understanding of data warehouse concepts (facts, dimensions, SCD, star schema)
• Experience building ETL pipelines
• Python knowledge (Pandas, boto3, paramiko), connecting to SFTPs, APIs, and pulling/pushing data
• Understanding of clean code and good coding practices
• Experience using Git and pipelines
• Solid Bash scripting skills for automation and troubleshooting
• Experience with Docker (images, containers, passing data between containers)
• Basic knowledge of AWS, including:
  • Running containers in ECS
  • Mounting EFS volumes
  • Viewing logs in CloudWatch
• English level B2 (can communicate and understand documentation)
• Willingness to learn and improve skills
• Interest in software development and data work

Nice to have

• Experience with Amazon Redshift, Snowflake, Postgres
• Experience using AWS CLI
• Knowledge of AWS services such as:
  • ECR
  • ECS
  • EventBridge
  • CloudWatch
  • Lambda
  • Step Functions
• Experience working with REST APIs
• Knowledge of NoSQL databases
• Experience with CI/CD tools

We offer:

• Possibility to propose solutions on a project
• Dynamic and challenging tasks
• Team of professionals
• Competitive salary
• Low bureaucracy
• Continuous self-improvement
• Long-term employment with paid vacation and other social benefits
• Bunch of perks 😊

This vacancy is exclusively for Ukrainian developers!

QIT Software is looking for a Data Engineer to a hospitality technology company which running an analytics platform that serves 2,500+ hotels and 500+ restaurants. You will own and operate our AWS data infrastructure - building pipelines, fixing what breaks, and making the platform more reliable and scalable.


Project: 
Hospitality Analytics Platform
Requirements:
- 6+ years hands-on data engineering (not architecture diagrams - actual pipelines in production)
- Strong Spark/PySpark and Python
- Advanced SQL
- AWS data stack: EMR, Glue, S3, Redshift (or similar), IAM, CloudWatch
- Terraform

Would be a plus:
- Kafka/Kinesis streaming experience
- Airflow or similar orchestration
- Experience supporting BI tools and analytics teams

Responsibilities:
- Build and operate Spark/PySpark workloads on EMR and Glue
- Design end-to-end pipelines: ingestion from APIs, databases, and files → transformation → delivery to analytics consumers
- Implement data quality checks, validation, and monitoring
- Optimize for performance, cost, and reliability — then keep it running
- Work directly with product and analytics teams to define data contracts and deliver what they need
- Manage infrastructure via Terraform

Work conditions:
- The ability to work remotely from anywhere in the world;
- Flexible work schedule, no micromanagement, no strict deadlines and free overtime work;
- Work in European and American products with a modern technology stack in different industries (Finance, Technology, Health, Construction, Media, etc.);
- Revision of wages every year or on an individual basis;
- Accounting support and full payment of taxes by the company;
- 100% compensation for remote English lessons;
- 15 paid leaves (PTO) and public holidays.

 

Summary

• 5+ years in data science or data engineering roles;
• Proficient in Python, SQL, and common data tools (Pandas, Plotly, Streamlit, Dash);
• Familiarity with large language models (LLMs) and deploying ML in production;
• This role is NOT focused on BI, data platforms, or research work;
• Good fit: Hands-on, Python-first Applied AI / GenAI engineers with real delivery ownership and client-facing experience;
• No fit: Data platform or BI profiles, architecture-heavy or lead-only roles, research-focused profiles, or candidates with only PoC-level GenAI exposure and no ownership.

Role:

This role is ideal for someone comfortable working throughout the entire pre-sales to delivery lifecycle, rolling up their sleeves to solve complex multi-faceted problems, thrives as a technical communicator, and works well as a key member of a team. 

Requirements:

• 5+ years in data science or data engineering roles;
• Proficient in Python, SQL, and common data tools (pandas, Plotly, Streamlit, Dash); 
• Familiarity with large language models (LLMs) and deploying ML in production;
• Experience working with APIs and interpreting technical documentation;
• Client-facing mindset with clear ownership of decisions and outcomes;

• Project Description:

  We are looking for an experienced Data Engineer to join the Shelf Analytics project – a data-driven application designed to analyze how P&G products are positioned on store shelves. The primary objective of the solution is to improve product visibility, optimize in-store execution, and ultimately increase sales by combining shelf layout data with sales insights.
  
  As a Data Engineer, you will play a key role in building, maintaining, and enhancing scalable data pipelines and analytics workflows that power shelf-level insights. You will work closely with analytics and business stakeholders to ensure high-quality, reliable, and performant data solutions.
   

• Responsibilities:

  Design, develop, and maintain data pipelines and workflows using Databricks and PySpark
  Read, understand, and extend existing codebases; independently develop new components for Databricks workflows
  Implement object-oriented Python solutions (classes, inheritance, reusable modules)
  Develop and maintain unit tests to ensure code quality and reliability
  Work with Spark SQL and SQL Server Management Studio to create and optimize complex queries
  Create and manage Databricks workflows, clusters, databases, and tables
  Handle data storage and access management in Azure Data Lake Storage (ADLS), including ACL permissions
  Collaborate using GitHub, following CI/CD best practices and working with GitHub Actions
  Support continuous improvement of data engineering standards, performance, and scalability
   

• Mandatory Skills Description:

  Strong programming skills in Python and PySpark
  Hands-on experience with Databricks (workflows, clusters, tables, databases)
  Solid knowledge of SQL and experience with Spark SQL and SQL Server Management Studio
  Experience with pandas, dbx, and unit testing frameworks
  Practical experience working with Azure Storage (ADLS) and access control (ACLs)
  Proficiency with GitHub, including CI/CD pipelines and GitHub Actions
  Ability to work independently, analyze existing solutions, and propose improvements

We’re currently looking for a Senior Data Engineer for a long-term project, with immediate start.

The role requires:

- Databricks certification (mandatory)

- Solid hands-on experience with Spark

- Strong SQL (Microsoft SQL Server) knowledge

The project involves the migration from Microsoft SQL Server to Databricks, along with data-structure optimization and enhancements.

We are seeking a Senior Data Engineer to deliver data-driven solutions that optimize fleet utilization and operational efficiency across 46,000+ assets in 545+ locations. You will enable decision-making through demand forecasting, asset cascading, contract analysis, and risk detection, partnering with engineering and business stakeholders to take models from concept to production on AWS. 

Requirements 

• 5+ years of experience in data engineering 
• 3+ years of hands-on experience building and supporting production ETL/ELT pipelines 
• Advanced SQL skills (CTEs, window functions, performance optimization) 
• Strong Python skills (pandas, API integrations) 
• Proven experience with Snowflake (schema design, Snowpipe, Streams, Tasks, performance tuning, data quality) 
• Solid knowledge of AWS services: S3, Lambda, EventBridge, IAM, CloudWatch, Step Functions 
• Strong understanding of dimensional data modeling (Kimball methodology, SCDs) 
• Experience working with enterprise systems (ERP, CRM, or similar) 

Nice-to-haves 

• Experience with data quality frameworks (Great Expectations, Deequ) 
• Knowledge of CDC tools and concepts (AWS DMS, Kafka, Debezium) 
• Hands-on experience with data lake technologies (Apache Iceberg, Parquet) 
• Exposure to ML data pipelines and feature stores (SageMaker Feature Store) 
• Experience with document processing tools such as Amazon Textract 

Core Responsibilities 

• Design and develop ETL/ELT pipelines using Snowflake, Snowpipe, internal systems, Salesforce, SharePoint, and DocuSign 
• Build and maintain dimensional data models in Snowflake using dbt, including data quality checks (Great Expectations, Deequ) 
• Implement CDC patterns for near real-time data synchronization 
• Manage and evolve the data platform across S3 Data Lake (Apache Iceberg) and Snowflake data warehouse 
• Build and maintain Medallion architecture data lake in Snowflake 
• Prepare ML features using SageMaker Feature Store 
• Develop analytical dashboards and reports in Power BI 

What we offer   

• Continuous learning and career growth opportunities 
• Professional training and English/Spanish language classes   
• Comprehensive medical insurance 
• Mental health support 
• Specialized benefits program with compensation for fitness activities, hobbies, pet care, and more 
• Flexible working hours 
• Inclusive and supportive culture 

We are looking for an experienced Data Engineer to join the Shelf Analytics project – a data-driven application designed to analyze how P&G products are positioned on store shelves. The primary objective of the solution is to improve product visibility, optimize in-store execution, and ultimately increase sales by combining shelf layout data with sales insights.

As a Data Engineer, you will play a key role in building, maintaining, and enhancing scalable data pipelines and analytics workflows that power shelf-level insights. You will work closely with analytics and business stakeholders to ensure high-quality, reliable, and performant data solutions.

• Responsibilities:

  Design, develop, and maintain data pipelines and workflows using Databricks and PySpark
  Read, understand, and extend existing codebases; independently develop new components for Databricks workflows
  Implement object-oriented Python solutions (classes, inheritance, reusable modules)
  Develop and maintain unit tests to ensure code quality and reliability
  Work with Spark SQL and SQL Server Management Studio to create and optimize complex queries
  Create and manage Databricks workflows, clusters, databases, and tables
  Handle data storage and access management in Azure Data Lake Storage (ADLS), including ACL permissions
  Collaborate using GitHub, following CI/CD best practices and working with GitHub Actions
  Support continuous improvement of data engineering standards, performance, and scalability

• Mandatory Skills Description:

  Strong programming skills in Python and PySpark
  Hands-on experience with Databricks (workflows, clusters, tables, databases)
  Solid knowledge of SQL and experience with Spark SQL and SQL Server Management Studio
  Experience with pandas, dbx, and unit testing frameworks
  Practical experience working with Azure Storage (ADLS) and access control (ACLs)
  Proficiency with GitHub, including CI/CD pipelines and GitHub Actions
  Ability to work independently, analyze existing solutions, and propose improvements

• Nice-to-Have Skills Description:

  Experience with retail, CPG, or shelf analytics–related solutions
  Familiarity with large-scale data processing and analytics platforms
  Strong communication skills and a proactive, problem-solving mindset

Project Description:

We are looking for an experienced Data Engineer to join the Shelf Analytics project – a data-driven application designed to analyze how P&G products are positioned on store shelves. The primary objective of the solution is to improve product visibility, optimize in-store execution, and ultimately increase sales by combining shelf layout data with sales insights.

As a Data Engineer, you will play a key role in building, maintaining, and enhancing scalable data pipelines and analytics workflows that power shelf-level insights. You will work closely with analytics and business stakeholders to ensure high-quality, reliable, and performant data solutions.

Responsibilities:

Design, develop, and maintain data pipelines and workflows using Databricks and PySpark
Read, understand, and extend existing codebases; independently develop new components for Databricks workflows
Implement object-oriented Python solutions (classes, inheritance, reusable modules)
Develop and maintain unit tests to ensure code quality and reliability
Work with Spark SQL and SQL Server Management Studio to create and optimize complex queries
Create and manage Databricks workflows, clusters, databases, and tables
Handle data storage and access management in Azure Data Lake Storage (ADLS), including ACL permissions
Collaborate using GitHub, following CI/CD best practices and working with GitHub Actions
Support continuous improvement of data engineering standards, performance, and scalability

Mandatory Skills Description:

Strong programming skills in Python and PySpark
Hands-on experience with Databricks (workflows, clusters, tables, databases)
Solid knowledge of SQL and experience with Spark SQL and SQL Server Management Studio
Experience with pandas, dbx, and unit testing frameworks
Practical experience working with Azure Storage (ADLS) and access control (ACLs)
Proficiency with GitHub, including CI/CD pipelines and GitHub Actions
Ability to work independently, analyze existing solutions, and propose improvements

Nice-to-Have Skills Description:

Experience with retail, CPG, or shelf analytics–related solutions
Familiarity with large-scale data processing and analytics platforms
Strong communication skills and a proactive, problem-solving mindset

Languages:

English: B2 Upper Intermediate

Project Description
The project focuses on the modernization, maintenance, and development of an eCommerce platform for a large US-based retail company, serving millions of omnichannel customers weekly.

Solutions are delivered by several Product Teams working on different domains: Customer, Loyalty, Search & Browse, Data Integration, and Cart.

Current key priorities:

• New brands onboarding
• Re-architecture
• Database migrations
• Migration of microservices to a unified cloud-native solution without business disruption

Responsibilities

• Design data solutions for a large retail company.
• Support the processing of big data volumes.
• Integrate solutions into the current architecture.

Mandatory Skills

• Microsoft Azure Data Factory / SSIS
• Microsoft Azure Databricks
• Microsoft Azure Synapse Analytics
• PostgreSQL
• PySpark

Mandatory Skills Description

• 3+ years of hands-on expertise with Azure Data Factory and Azure Synapse.
• Strong expertise in designing and implementing data models (conceptual, logical, physical).
• In-depth knowledge of Azure services (Data Lake Storage, Synapse Analytics, Data Factory, Databricks) and PySpark for scalable data solutions.
• Proven experience in building ETL/ELT pipelines to load data into data lakes/warehouses.
• Experience integrating data from disparate sources (databases, APIs, external providers).
• Proficiency in data warehousing solutions (dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, Data Vault).
• Strong SQL skills: complex queries, transformations, performance tuning.
• Experience with metadata and governance in cloud data platforms.
• Certification in Azure/Databricks (advantage).
• Experience with cloud-based analytical databases.
• Hands-on with Azure MI, PostgreSQL on Azure, Cosmos DB, Azure Analysis Services, Informix.
• Experience in Python and Python-based ETL tools.
• Knowledge of Bash/Unix/Windows shell scripting (preferable).

Nice-to-Have Skills

• Experience with Elasticsearch.
• Familiarity with Docker/Kubernetes.
• Skills in troubleshooting and performance tuning for data pipelines.
• Strong collaboration and communication skills.

Languages

• English: B2 (Upper Intermediate)

We are seeking an expert with deep proficiency as a Platform Engineer, possessing experience in data engineering. This individual should have a comprehensive understanding of both data platforms and software engineering, enabling them to integrate the platform effectively within an IT ecosystem.

Responsibilities:

• Manage and optimize data platforms (Databricks, Palantir).
• Ensure high availability, security, and performance of data systems.
• Provide valuable insights about data platform usage.
• Optimize computing and storage for large-scale data processing.
• Design and maintain system libraries (Python) used in ETL pipelines and platform governance.
• Optimize ETL Processes – Enhance and tune existing ETL processes for better performance, scalability, and reliability.

Mandatory Skills Description:

• Minimum 10 Years of experience in IT/Data.
• Minimum 5 years of experience as a Data Platform Engineer/Data Engineer.
• Bachelor's in IT or related field.
• Infrastructure & Cloud: Azure, AWS (expertise in storage, networking, compute).
• Data Platform Tools: Any of Palantir, Databricks, Snowflake.
• Programming: Proficiency in PySpark for distributed computing and Python for ETL development.
• SQL: Expertise in writing and optimizing SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake.
• Data Warehousing: Experience working with data warehousing concepts and platforms, ideally Databricks.
• ETL Tools: Familiarity with ETL tools & processes
• Data Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design.
• Version Control: Proficiency with version control tools like Git to manage codebases and collaborate on development.
• Data Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance.
• Data Quality Tools: Experience implementing data validation, cleaning, and quality frameworks, ideally Monte Carlo.

Nice-to-Have Skills Description:

• Containerization & Orchestration: Docker, Kubernetes.
• Infrastructure as Code (IaC): Terraform.
• Understanding of Investment Data domain (desired).

Languages:

English: B2 Upper Intermediate

We’re hiring a Database Engineer to design, build, and operate reliable data platforms and pipelines. You’ll focus on robust ETL/ELT workflows, scalable big data processing, and cloud-first architectures (Azure preferred) that power analytics and applications.

What You’ll Do

• Design, build, and maintain ETL/ELT pipelines and data workflows (e.g., Azure Data Factory, Databricks, Spark, ClickHouse, Airflow, etc.).
• Develop and optimize data models, data warehouse/lake/lakehouse schema (partitioning, indexing, clustering, cost/performance tuning, etc.).
• Build scalable batch and streaming processing jobs (Spark/Databricks, Delta Lake; Kafka/Event Hubs a plus).
• Ensure data quality, reliability, and observability (tests, monitoring, alerting, SLAs).
• Implement CI/CD and version control for data assets and pipelines.
• Secure data and environments (IAM/Entra ID, Key Vault, strong tenancy guarantees, encryption, least privilege).
• Collaborate with application, analytics, and platform teams to deliver trustworthy, consumable datasets.

Required Qualifications

• ETL or ELT experience required (ADF/Databricks/dbt/Airflow or similar).
• Big data experience required.
• Cloud experience required; Azure preferred (Synapse, Data Factory, Databricks, Azure Storage, Event Hubs, etc.).
• Strong SQL and performance tuning expertise; hands-on with at least one warehouse/lakehouse (Synapse/Snowflake/BigQuery/Redshift or similar).
• Solid data modeling fundamentals (star/snowflake schemas, normalization/denormalization, CDC, etc.).
• Experience with CI/CD, Git, and infrastructure automation basics.

Nice to Have

• Streaming pipelines (Kafka, Event Hubs, Kinesis, Pub/Sub) and exactly-once/at-least-once patterns.
• Orchestration and workflow tools (Airflow, Prefect, Azure Data Factory).
• Python for data engineering.
• Data governance, lineage, and security best practices.
• Infrastructure as Code (Terraform) for data platform provisioning.