Jobs Data Engineer

Description
The ideal candidate brings solid Machine Learning and AI expertise, with hands-on experience building and operating data-driven and intelligent systems in production environments. Candidates must be comfortable defending architectural decisions and writing production-grade code in a high-performance environment.

Key Qualifications

Strong experience with Apache Spark, including optimization techniques for large-scale workloads
Strong experience with Spark Structured Streaming
Strong experience with Spark SQL and query performance optimization
Hands-on experience with Python for data processing, ML, and AI use cases
Practical experience in Machine Learning (ML), including model development and integration
Solid understanding of AI / GenAI concepts, with ability to apply them in real-world scenarios
Experience with Terraform and Databricks Asset Bundles
AI / ML Focus (Added Emphasis)

Proven practical ML experience, not only theoretical knowledge
Experience working with AI solutions before and/or during the GenAI era (classical ML is valuable)
Ability to integrate ML/AI components into scalable data platforms
Understanding of data pipelines supporting AI/ML workflows
Nice to Have

Familiarity with Databricks (good working knowledge; deep expertise is a plus but not required)
Experience with application development and Databricks Apps
Comfort working in Linux environments
Background in financial markets, such as market making, trading, or related domains
Knowledge of German or French is a plus
Working Style

Able to work independently, prioritize effectively, and proactively identify opportunities for improvement
Comfortable operating in a loosely defined scope, contributing not only to execution but also to defining priorities and driving work forward

We're looking for the right person to build that foundation from the ground up.
This is a hands-on leadership role with real ownership: you'll shape how data is collected, structured, and delivered — and grow a team around you as the function matures.

 

WHAT YOU'LL BE WORKING ON
 

• Design the target-state data architecture: Postgres CDC → Kafka → Snowflake → client-facing data products.
• Own tooling decisions across ingestion, orchestration, transformation, and quality layers.
• Implement CDC-based ingestion from PostgreSQL services (RDS, Aurora, EC2, K8s operator) using Debezium or equivalent.
• Build streaming and near-real-time pipelines with defined SLAs.
• Build a data quality control layer: checksums, reconciliation, schema validation, anomaly detection, and alerting.
• Define quality checkpoints across the full pipeline — from source capture through Snowflake to client delivery.
• Define and enforce data contracts with service-owning teams for core entities: transaction, merchant, settlement, and processing status.
• Build the external data delivery layer: financial settlement, transaction status, processor reconciliation, and client analytics.
• Design tenant separation and implement replay/reload mechanisms for failure recovery.
• Start hands-on, then gradually hire and grow a small data engineering team as the function matures.
  Build a pragmatic roadmap with concrete deliverables at 3, 6, and 12 months.
   

WHAT YOU NEED TO SUCCEED IN THIS ROLE
 

• 5+ years in data engineering with end-to-end ownership of production pipelines.
• Hands-on with Snowflake, PostgreSQL CDC (Debezium preferred), and Kafka.
• Solid AWS experience — S3, RDS, Aurora, and cloud data infrastructure.
• Data quality engineering mindset: monitoring, reconciliation, lineage.
• Comfortable defining data contracts and driving requirements with backend engineering teams.
• Technical leadership experience: project ownership, cross-team alignment, delivery under constraints.
• Kubernetes, Airflow/Prefect/Dagster, and dbt are strong pluses.
• Payments domain knowledge — settlement, transaction lifecycle, processor integrations — is a strong plus.
• Familiarity with gRPC, RabbitMQ, and reading Go/Python service code is expected.

• Project Description:

  Owns the technical direction, quality, and delivery enablement of the data platform within the team. Ensures scalable architecture, reliable releases, and strong engineering practices while enabling the team to deliver efficiently within defined engineering and release standards.

• Responsibilities:

  Architecture & Technical Ownership
  • Owns end-to-end technical design across Snowflake, dbt, ADF, and Azure components
  • Acts as the final technical decision authority within the team
  • Ensures alignment with enterprise architecture and data governance standards
  • Maintains architecture documentation and diagrams
  ________________________________________
  Engineering Excellence
  • Defines and enforces coding standards and best practices
  • Reviews critical PRs and ensures maintainability and scalability
  • Drives performance optimization and cost efficiency
  • Continuously improves development efficiency by reducing technical complexity and removing bottlenecks
  ________________________________________
  CI/CD & Release Ownership
  • Owns and improves CI/CD pipelines and deployment processes
  • Defines and enforces the release process, ensuring it is standardized and consistently followed
  • Establishes release criteria based on Definition of Done, testing, and required approvals
  • Ensures all changes meet release criteria before deployment to production
  • Oversees and coordinates production releases, including validation of readiness and risk assessment
  • Has authority to block releases that do not meet defined quality and governance standards
  • Ensures pipelines enforce quality gates (tests, validations, approvals where required)
  • Continuously improves release safety and reduces production incidents
  ________________________________________
  Quality & Definition of Done
  • Ensures adherence to Definition of Done (including testing and documentation)
  • Works with QA and team to ensure test coverage aligns with acceptance criteria
  ________________________________________
  Team Enablement & Mentoring
  • Mentors engineers and supports technical growth
  • Leads knowledge sharing and supports complex problem-solving
  ________________________________________
  Planning & Delivery Support
  • Contributes to sprint planning with technical estimates, risks, and dependencies
  • Owns identification and prioritization of technical debt in collaboration with the Product Owner
  ________________________________________
  Production Ownership
  • Leads incident analysis and root cause resolution
  • Ensures monitoring, alerting, and data quality validation mechanisms are in place across pipelines and in production
  • Maintains runbooks for operational readiness
  ________________________________________
  Ways of Working
  • Collaborates closely with Product Owner and Scrum Master to align technical delivery with business priorities; supports team continuity and helps facilitate meetings when the Scrum Master is absent
  • Proactively works with upstream data source teams and other squads to resolve cross-team dependencies, data issues, and production bugs
  • Drives technical alignment across teams/squads where shared data models, pipelines, or integrations are involved
  • Acts as the primary technical point of contact for external teams and stakeholders

• Mandatory Skills Description:

  7+ experience in Data Engineering and development of DWH solutions
  Hands-on experience with Snowflake, dbt, Azure Data Factory
  Experience with Azure Cloud Solutions, Azure DevOps
  Practical experience and understanding of CI/CD processes
  Experience of technical leadership in Agile SDLC

• Nice-to-Have Skills Description:

  Experience with AWS and/or GCP cloud solutions
  Certifications related to DWH

• Languages:
  • English: C1 Advanced

We are seeking a motivated Junior Big Data Engineer to join our team. In this role, you will contribute to the development and maintenance of data pipelines, data models, and data processing solutions within modern cloud and big data environments. You will work closely with senior engineers to build scalable data solutions and support data-driven business initiatives.

The ideal candidate has hands-on experience with Python, PySpark, and SQL, a solid understanding of data engineering fundamentals, and an interest in working with large-scale data platforms and AI-powered solutions.

Key Responsibilities:

• Support the design, development, and maintenance of data pipelines and ETL/ELT processes.
• Develop and optimize data transformation workflows using Python and PySpark.
• Assist in integrating data from multiple sources while ensuring data quality, consistency, and reliability.
• Collaborate with data engineers, data scientists, and AI/ML engineers to prepare and manage datasets for analytics and machine learning applications.
• Participate in data modeling, schema design, and data warehousing activities.
• Monitor data pipelines, investigate issues, and contribute to performance improvements.
• Follow data governance, security, and privacy best practices.
• Document technical solutions, processes, and data flows.
• Stay current with emerging technologies and industry trends, incorporating innovative solutions into data engineering practices, and effectively document and communicate technical solutions and processes.

Tools and skills you will use in this role:

• Palantir Foundry
• Python
• PySpark
• SQL
• TypeScript

Requirements:

• 1+ year of experience in data engineering
• Strong proficiency in Python and SQL
• Hands-on experience with PySpark
• Familiarity with big data technologies (e.g., Apache Hadoop, Spark, Kafka, BigQuery, etc.)
• Basic experience with cloud services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow)
• Understanding of ETL/ELT pipelines and data integration techniques
• Experience with database systems (e.g., PostgreSQL, MySQL, NoSQL, etc.)
• Effective problem-solving and analytical skills, coupled with excellent communication and collaboration abilities
• Strong communication and teamwork abilities
• Strong mathematical, statistical, and algorithmic skills
• English - Upper-Intermediate or higher

Nice to have:

• Experience with AI/ML
• Previous experience working with JavaScript and TypeScript

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers

Gypsy Collective is looking for a DWH Engineer to join our team and help build scalable, reliable, and high-performance data warehouse solutions that support analytics and business decision-making. We are looking for an experienced and proactive engineer who can work across the full data lifecycle - from ingestion and modeling to optimization, monitoring, and automation - while collaborating closely with analysts, developers, and business stakeholders.

    📅 Your daily adventures include:

• Designing, building, and maintaining scalable Data Warehouse architectures aligned with business needs;
• Developing and optimizing ETL/ELT pipelines using Python, Airflow, and custom solutions;
• Working with DWH/Data Lake technologies including PostgreSQL, Trino, and BigQuery;
• Implementing incremental loads, CDC, backfills, and reprocessing strategies;
• Optimizing query performance, data models, and pipeline execution;
• Ensuring data quality through validation, automated testing, monitoring, and alerting;
• Integrating new data sources (APIs, third-party systems, raw data) without disrupting existing pipelines;
• Collaborating with analysts, engineers, BI teams, and business stakeholders to translate requirements into scalable data solutions;

• Reviewing code, mentoring engineers, and contributing to data engineering standards and best practices.

  
  🧭 What makes you a great match:

• Strong SQL skills: complex queries, CTEs, window functions, analytical queries (5+ years experience);
• Knowledge of Python or other scripting languages for data transformations (4+ years experience);
• Deep understanding of DWH concepts: ETL/ELT, Data Vault, Kimball, Star/Snowflake schemas (5+ years experience);
• Experience with Airflow or other data pipeline orchestrators (5+ years experience);
• Hands-on experience with modern DWH and query engines: BigQuery, Snowflake, Redshift, ClickHouse, Vertica, AWS Athena, Trino (5+ years experience);
• Confident use of Git; experience with team workflows (pull requests, rebasing, merge conflict resolution (5+ years experience);

• Understanding of server and cloud infrastructure: basic skills in configuration, maintenance, monitoring, and load control (3+ years experience).
   

  ⭐ Nice to have:

• Experience with CDC tools and streaming data sources;
• Knowledge of Docker, Kubernetes, and Infrastructure as Code (Terraform);
• Experience with cloud platforms such as AWS, GCP, or Azure;

• Familiarity with data governance, data cataloging, and lineage tools.
   

  Benefits:

• 💸 Flexible payment options: choose the method that works best for you.
• 🎁 Financial perks: Bonuses for holidays, B-day, work milestones and more - just to show we care.
• 📈 Learn & grow: We cover courses and certifications — and offer real opportunities to grow your career with us.
• 🥐 Benefit Сafeteria: Choose what suits you — sports, language courses, therapy sessions, and more.
• 🎉 Stay connected: From team-building events to industry conferences — we bring people together online, offline, and on stage.
• 💻 Modern Equipment: We provide new laptops along with essential peripherals like monitors and headphones for a comfortable workflow.
• 🕘 Your schedule, your rules: Start your day at 9, 10, or even 11 — we care about results, not clock-ins.
  
  If this role sounds like a good fit for you, we’d love to hear from you — apply and tell us a bit about yourself.

Looking for a Data Engineer to join the Dataforest team. If you are looking for a friendly team, a healthy working environment, and a flexible schedule ‒ you have found the right place to send your CV.

Skills requirements:
• 2+ years of experience with Python;
• 2+ years of experience as a Data Engineer;
• Experience with Pandas;
• Experience with SQL DB / NoSQL (Redis, Mongo, Elasticsearch) / BigQuery;
• Familiarity with Amazon Web Services;
• Knowledge of data algorithms and data structures is a MUST;
• Working with high volume tables 10m+.

Optional skills (as a plus):
• Experience with Spark (pyspark);
• Experience with Airflow;
• Experience with Kafka;
• Experience in statistics;
• Knowledge of DS and Machine learning algorithms..

Key responsibilities:
• Create ETL pipelines and data management solutions (API, Integration logic);
• Different data processing algorithms;
• Involvement in creation of forecasting, recommendation, and classification models.

We offer:

• Great networking opportunities with international clients, challenging tasks;

• Building interesting projects from scratch using new technologies;

• Personal and professional development opportunities;

• Competitive salary fixed in USD;

• Paid vacation and sick leaves;

• Flexible work schedule;

• Friendly working environment with minimal hierarchy;

• Team building activities, corporate events.

We are looking for a Senior Data Engineer to join the Identity team at Equals5.  We are building a dynamic data ecosystem in which AI is deeply integrated—both as a productivity multiplier and as a core component of our data-processing logic for identity data enrichment and scoring.

You will own the infrastructure that handles over 10,000 executions per minute, ensuring stability, scalability, and data integrity. You will work with a modern stack on Google Cloud Platform, utilizing Cloud Functions and Kubernetes.

We are looking for an engineer who improves infrastructure, automates everything, and is eager to implement LLM-based logic directly into high-load data flows.
 

Responsibilities
 

• AI-Driven Data Scoring: Design and implement pipelines that utilize LLMs to analyze and score identity data in real-time. You will integrate AI models directly into the decision-making loop, balancing accuracy with latency and cost.
• Own the Data Architecture: Architect scalable data solutions using GCP and Python. You will manage data storage and retrieval using BigQuery and Apache Iceberg to support querying of TBs of data.
• Heavy Data Processing: Utilize Apache Spark for data transformations and batch processing when lightweight cloud functions are not enough.
• Manage High-Load Orchestration: Maintain and optimize our system instances. It involves complex dataflows, custom Python nodes, and performance tuning for 10k+ execs/minute.
• Release Lifecycle (CI/CD): Take ownership of the deployment process, ensuring that updates to pipelines and infrastructure are released safely with proper testing and rollback strategies.
• Database Optimization: Manage PostgreSQL performance under heavy load, optimizing complex queries and indexing strategies.
• Active AI Usage: Use Claude Code and other AI engineering tools to accelerate your own development, refactoring, and testing processes.
• Incident Resolution: Proactively monitor the system. When alerts fire, you investigate the root cause—whether it’s a database lock or an LLM hallucination—and fix it permanently.
   

Requirements
 

• 4-5+ years of experience in Data Engineering or Backend Engineering with a strong data focus.
• Production AI Integration: Experience integrating LLMs (OpenAI, Anthropic, Gemini) into production applications via API. You understand latency, token limits, and how to structure data for AI scoring.
• Expertise in GCP: Understanding of Google Cloud Platform (Cloud Functions, IAM, Networking).
• Strong Python: You write clean, efficient, and testable code. You are comfortable building custom logic where standard tools fall short.
• Big Data Stack: Experience with BigQuery, Apache Spark, and modern table formats like Apache Iceberg.
• Kubernetes (K8s): Experience deploying and scaling services in containerized environments.
• Workflow Automation: Understanding of workflow orchestration tools at a deep technical level. N8N is a big part of our domain, so familiarity with it is highly valuable.
• PostgreSQL Mastery: Proven ability to handle heavy write/read loads and optimize schemas.
• English: B2+ (Upper-Intermediate) or higher.
   

Culture & Mindset
 

• Self-improvement: You are a fast learner. You don't fear AI replacing you; you master it to replace your manual tasks.
• Ownership: You treat the Identity domain as your own business. If a scoring model drifts or a pipeline slows down, you notice it and fix it without being asked.
• Internal Locus of Control: You take responsibility for outcomes. If an external API fails, you build a fallback mechanism instead of just blaming the provider.
• Get It Done: You prioritize shipping value. You know when to use a simple script and when to build a complex architecture.
• Openness: You share knowledge freely. If you find a better way to prompt the AI for scoring, you share it with the team.
   

What We Offer
 

• Fully remote with flexible hours (aligned with EU timezones for syncs).
• AI-Native Environment: We provide licenses for Claude Code and encourage using the bleeding edge of AI tech for both daily coding and product features.
• High-Impact Role: You will directly influence how we identify and score users, impacting the core business logic.
• Cross-functional visibility: Work closely with Product and Tech Leads to shape the future of Identity.
• No Bureaucracy: Fast decisions, no legacy processes, focus on results.

About the Company

The firm is a specialized consultancy in statistics, data science, and AI. We work across industries - including the public sector, insurance, and other data-intensive domains. For this role, we are specifically looking to strengthen our teams working on public-sector projects.

In these projects, we develop solutions that are not only technically challenging but also create a tangible positive impact on society. We are looking for strong Senior Data Engineers who are drawn to complex problems rather than intimidated by them - and who want to build systems that truly make a difference.

Role Overview

Do you want to do more than maintain existing data infrastructures? Are you interested in strategically leading data engineering initiatives and helping shape sustainable data platforms from the ground up? Do you think in terms of data architecture, scalability, quality, security, and operational excellence - and want your work in the public sector to have a visible impact? Then you are in the right place.

As a Senior Data Engineer, you will take technical ownership of the data engineering domain: from architecture and key technical decisions to design, implementation, operations, monitoring, and continuous optimization of data pipelines and platforms. You will develop robust, scalable data solutions that not only work today but remain maintainable and reliable in production over the long term.

What You'll Be Doing

• Lead data engineering topics within projects, drive strategic decisions, and take overall responsibility for this area.
• Design, develop, maintain, and optimize ETL/ELT pipelines.
• Ensure data pipelines remain scalable, maintainable, and efficient as data volumes and the number of data sources grow.
• Define and establish standards and best practices for pipeline design, testing, monitoring, operations, and performance.
• Review changes to data pipelines and data architecture, ensuring compliance with quality standards.
• Work hands-on with the Cloudera/Apache stack as well as modern data stack technologies.
• Ensure data quality, access control, privacy, and security throughout the entire data lifecycle.

Requirements

• Several years of solid experience in data engineering and in a technical leadership role.
• Proven experience designing and implementing scalable ETL/ELT pipelines.
• Experience with both batch and streaming data pipelines.
• Practical experience with modern data stack tools such as dbt and S3/Object Storage.
• Strong understanding of data architecture, scalability, maintainability, and data engineering best practices.
• Excellent SQL skills and solid programming skills, ideally in Python and/or Java.
• Experience with code reviews, establishing engineering standards, and performance optimization.
• A strong sense of ownership and the ability to independently lead and deliver technical initiatives.
• Experience building custom data pipelines from scratch, including without relying on prebuilt frameworks or specialized cloud services.
• Demonstrated experience processing various data formats, such as tabular data, JSON, text, images, and other formats.
• Experience handling confidential, security-sensitive, and personal data.
• Knowledge of Cloudera tooling and the Apache ecosystem is a plus, but not required.

Candidate Requirements

• Ukrainian native speaker
•  English proficiency: B2 or higher

Are you a Senior Backend Engineer passionate about building scalable AI-driven systems? Join us at Sigma Software to work on a cutting-edge platform that transforms how large-scale engineering organizations manage and analyze complex technical data.
This is a remote position with flexible locations across Europe, Ukraine, and LATAM. You will be part of an innovative team leveraging AI, semantic graphs, and distributed processing to deliver impactful solutions.
At Sigma Software, we value expertise, creativity, and collaboration. Why join us? You will work with advanced technologies, contribute to mission-critical projects, and be part of a company recognized for excellence and innovation.

CUSTOMER
Our customer operates in the AdTech industry, delivering advanced technology solutions that optimize advertising performance and audience targeting. While the name is confidential, the organization is known for leveraging cutting-edge AI and data-driven strategies to enhance campaign effectiveness and deliver measurable business impact.

PROJECT
We are developing a next-generation AI-powered Knowledge Base and Gap Analysis platform for SysML-based engineering environments. The system enables large-scale engineering organizations to ingest, structure, analyze, and reason over complex MBSE artifacts and technical documentation. It supports both cloud and secure classified environments, improving traceability, identifying gaps, and enhancing decision-making in mission-critical projects.

Responsibilities

• Design and develop scalable AI-powered backend systems for SysML-based engineering environments
• Build and maintain distributed data ingestion and ETL pipelines for large-scale engineering artifacts and technical documentation
• Develop and optimize LLM-powered workflows for metadata extraction, semantic analysis, and entity resolution
• Implement AI agents and multi-agent orchestration workflows
• Design and improve RAG-based architectures and semantic retrieval pipelines
• Develop graph-based knowledge representation and traceability analysis solutions
• Work with graph databases, graph processing libraries, and semantic relationship modeling
• Build and optimize distributed data processing workflows using Apache Spark
• Collaborate with cross-functional engineering teams to integrate AI capabilities into platform services
• Design scalable and high-performance APIs and backend services
• Improve system reliability, scalability, observability, and performance across distributed environments
• Participate in architecture discussions and technical decision-making processes
• Contribute to cloud-native infrastructure and deployment workflows
• Support deployments in secure, air-gapped, or classified environments when required
• Create and maintain technical documentation and engineering best practices

Qualifications

• At least 5 years of commercial experience in software engineering, Data Engineering, or AI systems development
• Experience with at least one JVM language
• Hands-on experience building distributed and scalable systems
• Practical experience with LLM-based applications and AI integrations
• Experience building AI agents and multi-agent systems
• Strong understanding of RAG architectures and semantic retrieval workflows
• Hands-on experience with graph technologies, graph libraries, or graph databases
• Experience with Apache Spark and distributed data processing
• Strong understanding of ETL pipelines and large-scale data ingestion workflows
• Experience with cloud-native infrastructure and distributed environments
• Practical experience with backend platform development and API integrations
• Good understanding of semantic search, entity resolution, and metadata extraction
• Experience working with highly scalable and high-performance systems
• Strong problem-solving and communication skills
• Upper-Intermediate level of English

WILL BE A PLUS

• Background in Data Engineering
• Experience with  distributed data processing, Apache Spark, or Apache Beam
• Experience with Knowledge Graphs and graph-based semantic modeling
• Familiarity with MBSE or SysML environments
• Experience supporting air-gapped or classified environments
• Experience with vector databases and embedding pipelines
• Experience with Kubernetes and cloud platforms such as AWS, GCP, or Azure

Additional information

PERSONAL PROFILE

• Analytical mindset with strong problem-solving skills
• Ability to work independently and in a distributed team
• Adaptability to secure and classified environments
• Strong communication and collaboration abilities
• Passion for AI-driven engineering solutions

Are you a Senior Backend Engineer passionate about building scalable AI-driven systems? 

Join us at Sigma Software to work on a cutting-edge platform that transforms how large-scale engineering organizations manage and analyze complex technical data.
This is a remote position with flexible locations across Europe, Ukraine, and LATAM. You will be part of an innovative team leveraging AI, semantic graphs, and distributed processing to deliver impactful solutions.
At Sigma Software, we value expertise, creativity, and collaboration. Why join us? You will work with advanced technologies, contribute to mission-critical projects, and be part of a company recognized for excellence and innovation.

CUSTOMER
Our customer operates in the AdTech industry, delivering advanced technology solutions that optimize advertising performance and audience targeting. While the name is confidential, the organization is known for leveraging cutting-edge AI and data-driven strategies to enhance campaign effectiveness and deliver measurable business impact.

PROJECT
We are developing a next-generation AI-powered Knowledge Base and Gap Analysis platform for SysML-based engineering environments. The system enables large-scale engineering organizations to ingest, structure, analyze, and reason over complex MBSE artifacts and technical documentation. It supports both cloud and secure classified environments, improving traceability, identifying gaps, and enhancing decision-making in mission-critical projects.

Responsibilities

• Design and develop scalable AI-powered backend systems for SysML-based engineering environments
• Build and maintain distributed data ingestion and ETL pipelines for large-scale engineering artifacts and technical documentation
• Develop and optimize LLM-powered workflows for metadata extraction, semantic analysis, and entity resolution
• Implement AI agents and multi-agent orchestration workflows
• Design and improve RAG-based architectures and semantic retrieval pipelines
• Develop graph-based knowledge representation and traceability analysis solutions
• Work with graph databases, graph processing libraries, and semantic relationship modeling
• Build and optimize distributed data processing workflows using Apache Spark
• Collaborate with cross-functional engineering teams to integrate AI capabilities into platform services
• Design scalable and high-performance APIs and backend services
• Improve system reliability, scalability, observability, and performance across distributed environments
• Participate in architecture discussions and technical decision-making processes
• Contribute to cloud-native infrastructure and deployment workflows
• Support deployments in secure, air-gapped, or classified environments when required
• Create and maintain technical documentation and engineering best practices

Qualifications

• At least 5 years of commercial experience in software engineering, Data Engineering, or AI systems development
• Experience with Python
• Hands-on experience building distributed and scalable systems
• Practical experience with LLM-based applications and AI integrations
• Experience building AI agents and multi-agent systems
• Strong understanding of RAG architectures and semantic retrieval workflows
• Hands-on experience with graph technologies, graph libraries, or graph databases
• Experience with Apache Spark and distributed data processing
• Strong understanding of ETL pipelines and large-scale data ingestion workflows
• Experience with cloud-native infrastructure and distributed environments
• Practical experience with backend platform development and API integrations
• Good understanding of semantic search, entity resolution, and metadata extraction
• Experience working with highly scalable and high-performance systems
• Strong problem-solving and communication skills
• Upper-Intermediate level of English

WILL BE A PLUS

• Background in Data Engineering
• Experience with  distributed data processing, Apache Spark, or Apache Beam
• Experience with Knowledge Graphs and graph-based semantic modeling
• Familiarity with MBSE or SysML environments
• Experience supporting air-gapped or classified environments
• Experience with vector databases and embedding pipelines
• Experience with Kubernetes and cloud platforms such as AWS, GCP, or Azure

Additional information

PERSONAL PROFILE

• Analytical mindset with strong problem-solving skills
• Ability to work independently and in a distributed team
• Adaptability to secure and classified environments
• Strong communication and collaboration abilities
• Passion for AI-driven engineering solutions

Are you passionate about building scalable, reliable data systems that power real-world solutions? We are looking for a Senior Data Engineer to join our team and work remotely from Europe, Ukraine, or LATAM.

At Sigma Software, we partner with innovative companies to deliver cutting-edge technology solutions. This role offers the opportunity to design and operate a canonical data platform that integrates diverse external data sources, ensuring high quality and flexibility.

Why join us? You’ll work with modern technologies, collaborate with talented professionals across multiple regions, and have the freedom to experiment and innovate while contributing to impactful projects in the AdTech domain.

Customer

Our client is a fast-growing AdTech company focused on solving real-world challenges in digital advertising and data-driven marketing. They specialize in building highly resilient and scalable systems that encourage experimentation and innovation. Their data platform is designed to be both powerful and flexible, enabling engineers to introduce new ideas and leverage the latest technologies to optimize advertising performance and audience insights.

Project

The project involves developing and maintaining a canonical data platform that integrates multiple external data sources. It supports both full data loads and incremental updates, ensures high data quality, and leverages AI-assisted tools for monitoring and issue detection.

Key Technologies: Java, Scala, Apache Beam/Apache Spark, SQL, GCP Dataflow, REST APIs, AI-assisted tools, event-driven architectures

Responsibilities

• Design, build, and maintain reliable data pipelines to ingest and process data from external systems
• Implement integrations using Java or Scala, ensuring scalability and resilience
• Handle both full data loads and incremental updates efficiently
• Collaborate with engineering and product teams to align on integration and data needs
• Develop and maintain a unified internal data model
• Ensure data quality, detect and resolve incorrect or incomplete data
• Utilize AI-assisted tools to improve data quality and operational efficiency
• Monitor and optimize data pipelines for production readiness

Requirements

• At least 5 years of experience as a Data Engineer or Backend/Data Engineer
• Strong production experience with Java or Scala
• Experience with Apache Beam or Apache Spark
• Proven experience integrating external systems via REST APIs (pagination, rate limiting, token-based authentication)
• Strong data modelling skills with structured, relational data
• Solid SQL knowledge
• Experience with GCP Dataflow
• Experience operating data pipelines in production environments
• Strong communication skills and ability to collaborate across teams
• At least an Upper Intermediate level of English

WILL BE A PLUS:

• Experience with event-driven architectures or messaging systems
• Designing canonical schemas across multiple data sources
• Experience with LLM (Large Language Models)

Personal Profile

• Proactive and detail-oriented
• Thrives in collaborative environments
• Enjoys solving complex integration challenges
• Comfortable experimenting with new technologies
• Committed to improving data systems

We are looking for a Big Data Engineer to join our growing team. In this role, you will be responsible for the design and development of our big data processing engine — a heavy ETL pipeline handling complex logic including geo-location analysis, running across both cloud and on-prem environments.

Requirements:

• 4+ years of experience in Data Engineering
• 2+ years of hands-on experience with Scala (must) or Java (optional)
• 2+ years of experience with SQL and NoSQL DBs
• Hands-on experience with Spark & PySpark (must)
• Experience with Airflow (must)
• Experience with AWS and/or GCP
• Experience with Kubernetes (k8s) and on-prem deployments
• Solid understanding of distributed systems design, pipeline architecture, and partitioning strategies
• Deep understanding of Agile principles, practices and values
• Solid understanding of design patterns, software development techniques, and clean code practices
   

Advantage:

• Experience with Apache Iceberg, Helm Charts, or data catalog tools
• Experience with Spring Framework
• Experience with microservice architecture
• Experience working with DevOps and CI/CD practices
• Customer-facing experience or comfort working directly with clients
• Bachelor's, BSc or Master's degree in Computer Science or Engineering from a leading academic institution

N-iX is a global software development service company that helps businesses across the globe create next-generation software products. Founded in 2002, we unite 2,400+ tech-savvy professionals across 40+ countries, working on impactful projects for industry leaders and Fortune 500 companies. Our expertise spans cloud, data, AI/ML, embedded software, IoT, and more, driving digital transformation across finance, manufacturing, telecom, healthcare, and other industries. Join N-iX and become part of a team where your ideas make a real impact.

We are looking for a Senior Machine Learning Engineer to lead the development and deployment of advanced AI models. In this role, you will be responsible for the end-to-end lifecycle of our machine learning systems, from architectural design and data preprocessing to model training, optimization, and production deployment. 

You will work at the intersection of generative AI and traditional machine learning, building the engines that power two strategic AI initiatives spanning NLP-based document intelligence and predictive analytics. Operating within a structured agile delivery model with formal review gates, you will ensure our models are not just accurate, but also robust, explainable, and deployable within highly secure, on-premise environments . 

Key Responsibilities:

1. LLM & NLP Pipelines 

• Design and fine-tune Large Language Model (LLM) pipelines to interpret complex regulatory texts (e.g. standards, building codes) and extract structured rules. 
• Convert natural language requirements into computer-processable formats (e.g., logic tuples) that can be executed by downstream compliance engines. 
• Implement RAG (Retrieval-Augmented Generation) architectures to enable semantic querying of technical documentation and historical project data. 
• Optimize prompt strategies (few-shot learning, chain-of-thought) to improve model performance on domain-specific tasks without extensive retraining.

1. Predictive & Analytical Models (Supply Chain) 

• Develop time-series forecasting models to predict material demand and spend categories, integrating internal ERP data with external market signals. 
• Build classification and anomaly detection models to assess supplier risk profiles based on financial health, delivery performance, and geopolitical factors. 
• Design algorithms for multi-objective optimization(e.g., balancing cost vs. lead time vs. risk) to support procurement decision-making. 

1. MLOps & Productionization 

• Containerize models using Docker/Kubernetes and deploy them into secure, on-premise inference environments. 
• Build automated training and inference pipelines using tools like Kubeflow or MLflow to ensure reproducibility and scalability. 
• Optimize model inference latency and resource usage (e.g., quantization, distillation) to run efficiently on available hardware. 
• Implement monitoring systems to track model drift and performance in production, establishing feedback loops for continuous improvement. 

Requirements:

•  5+ years of experience in Machine Learning Engineering, with a proven track record of deploying models into production environments. 
• Expert proficiency in Python and standard ML libraries (PyTorch, TensorFlow, Scikit-learn, Pandas, NumPy). 
• Strong experience with transformer architectures (BERT, GPT, Llama) and NLP frameworks (Hugging Face, LangChain). 
• Proficiency with MLOps tools and practices, including containerization (Docker), orchestration (Kubernetes), and experiment tracking (MLflow). 
• Ability to design data preprocessing pipelines for both structured (SQL, tabular) and unstructured (text, PDF) data. 
• Strong grasp of algorithmic principles for implementing custom logic, such as graph traversal or geometric computations.
• Ability to quickly learn and apply ML techniques to specialized domains like engineering, supply chain, or other highly regulated industries. 
• Experience working in agile environments (Sprints) while adhering to rigorous engineering standards and documentation requirements.
• Strong communication skills to work effectively with Data Scientists, Backend Engineers, and Domain Experts to align technical solutions with business needs. 

Why This Role? 

You will be building the intelligence that drives critical enterprise infrastructure. Your models will not just generate text or predictions; they will directly influence the design of complex industrial systems and the resilience of supply chains. If you are ready to apply advanced ML to tangible, high-stakes problems in a rigorous engineering environment, join us.

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers

About the Role

We are seeking a highly analytical and detail-oriented Data QA Engineer to ensure the quality, accuracy, and reliability of data platforms, pipelines, and reporting systems. The ideal candidate has strong SQL expertise, a solid understanding of data engineering concepts, and the ability to independently investigate data issues, validate complex data flows, and support data quality initiatives across the organization.

Key Responsibilities

• Validate data accuracy, completeness, and consistency across multiple systems and environments.
• Perform data quality testing for ETL/ELT pipelines, data warehouses, and reporting solutions.
• Design and execute test plans, test cases, and validation strategies for data-centric applications.
• Write complex SQL queries for data validation, reconciliation, and root cause analysis.
• Analyze and troubleshoot data discrepancies across source and target systems.
• Review, understand, and debug Python-based data pipeline code when necessary.
• Collaborate with data engineers, software engineers, analysts, and business stakeholders to ensure data integrity.
• Utilize AI-powered tools to improve testing efficiency, investigation, analysis, and productivity.
• Work independently and take ownership of data quality initiatives throughout the project lifecycle.

Required Skills

Database Engineering & SQL

• Strong knowledge of relational databases and database concepts.
• Advanced SQL skills with the ability to write complex queries for:
  • Data validation
  • Data reconciliation
  • Root cause analysis
  • Data quality investigations
• Experience working with large datasets and optimizing data validation processes.

Data Infrastructure & Concepts

• Solid understanding of data engineering principles and best practices.
• Experience validating data pipelines and data processing workflows.
• Understanding of ETL/ELT architectures and data movement processes.
• Knowledge of data warehouse concepts and dimensional modeling.
• Familiarity with OLAP technologies and modern cloud data platforms such as Redshift, Snowflake, BigQuery, or equivalent.

Programming Skills

• Strong proficiency in Python.
• Ability to read, understand, analyze, and debug data pipeline code.
• Experience using Python for data validation, testing, or data analysis activities.

Additional Requirements

• Experience utilizing AI tools to improve testing, analysis, and productivity.
• Strong analytical thinking and problem-solving skills.
• Excellent attention to detail.
• Ability to work independently with minimal supervision.
• Strong communication and collaboration skills.

Nice to Have

• Experience with data quality frameworks and monitoring tools.
• Experience in testing cloud-based data platforms.
• Knowledge of Apache Airflow, dbt, Spark, or similar data technologies.
• Experience working in Agile/Scrum environments.
• Familiarity with CI/CD practices for data engineering projects.

About PredictSpring

PredictSpring is a market-leading company shaping the future of omni-channel retail and modern POS technology. We help global retail and lifestyle brands deliver seamless, modern, and data-driven customer experiences across digital and in-store channels.

We are looking for a highly skilled Senior AI/ML Engineer – GenAI to design, develop, and deploy enterprise-grade AI solutions with a strong focus on Generative AI, Large Language Models, Retrieval-Augmented Generation, and Agentic AI systems.

In this role, you will build production AI applications, multi-agent workflows, scalable ML pipelines, and cloud-native AI solutions that create business value across our platform and operations.
 

Responsibilities

As a Senior AI/ML Engineer – GenAI, you will:

• Design, develop, and deploy enterprise Generative AI applications using foundation models such as GPT, Claude, Gemini, Llama, Mistral, and other open-source LLMs.
• Build advanced RAG solutions using vector databases, embeddings, reranking techniques, and hybrid search architectures.
• Develop prompt engineering strategies, evaluation frameworks, guardrails, and hallucination mitigation techniques.
• Fine-tune and optimize LLMs using techniques such as LoRA, QLoRA, PEFT, and reinforcement learning approaches.
• Integrate LLMs with enterprise applications, APIs, databases, and business workflows.
• Design and implement autonomous and semi-autonomous AI agents capable of reasoning, planning, memory management, tool usage, and workflow orchestration.
• Build multi-agent systems using frameworks such as LangGraph, LangChain, AutoGen, CrewAI, Semantic Kernel, MCP, A2A, or similar technologies.
• Build and maintain end-to-end machine learning pipelines, including data ingestion, feature engineering, model training, evaluation, deployment, and monitoring.
• Develop predictive models, recommendation systems, anomaly detection solutions, NLP applications, and deep learning models.
• Design CI/CD pipelines for machine learning and AI applications.
• Deploy and manage AI workloads using Docker, Kubernetes, and cloud-native services.
• Monitor model drift, performance degradation, latency, and data quality.
• Architect cloud-native AI solutions on AWS, Azure, or Google Cloud Platform.
• Partner with product managers, software engineers, data engineers, and business stakeholders to translate business requirements into AI solutions.
• Mentor junior engineers and contribute to architecture reviews, technical strategy, and AI best practices.

Requirements

We are looking for someone with:

• Upper C1 or higher English proficiency, with fluent spoken and written English.
• Ability to confidently participate in client-facing discussions, technical workshops, architecture reviews, stakeholder presentations, and cross-functional collaboration.
• Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Data Science, Machine Learning, Engineering, or a related field.
• 7+ years of software engineering, machine learning, or AI experience.
• Minimum 3+ years of hands-on experience developing and deploying Generative AI applications in production environments.
• Strong Python programming skills with experience building scalable backend systems.
• Extensive experience with machine learning frameworks including PyTorch, TensorFlow, scikit-learn, and Hugging Face.
• Proven experience developing RAG architectures and knowledge retrieval systems.
• Experience building Agentic AI solutions and multi-agent workflows.
• Strong understanding of vector databases, embeddings, semantic search, and retrieval optimization.
• Experience with cloud platforms including AWS, Azure, or GCP.
• Strong understanding of MLOps principles, CI/CD, containerization, and model lifecycle management.

Preferred Qualifications

Nice to have:

• Experience with managed AI services such as AWS SageMaker, AWS Bedrock, Azure OpenAI, Azure ML, or Vertex AI.
• Experience with distributed computing platforms such as Databricks, Spark, or Ray.
• Experience with AI governance, responsible AI practices, model monitoring, and production AI observability.
• Experience with secure AI architectures, role-based permissions, auditability, and compliance controls.
• Experience in retail, e-commerce, omni-channel commerce, or POS technology is a strong plus.

Why PredictSpring?

• Join a market-leading company shaping the future of omni-channel retail and modern POS.
• Work on high-impact challenges at the intersection of commerce, data, AI, and customer experience.
• Collaborate with a talented global team in a flexible remote work environment.
• Enjoy opportunities for growth, learning, and professional development.
• Contribute to products used by some of the world’s most recognized retail and lifestyle brands.
• Be part of an innovative, collaborative, and data-driven culture focused on delivering measurable business value.