Jobs Data Engineer

We are looking for a Senior Data Engineer. This is not a standard ETL role. You will work on a high-load system processing 10,000+ executions per minute, building scalable pipelines where AI and LLM logic are embedded directly into production data flows.

The role combines data engineering, backend thinking, infrastructure ownership, and real production AI integration.

What you will do:

• Design and maintain scalable data pipelines in Python on Google Cloud Platform
• Build and optimize AI-driven scoring flows using LLMs for real-time identity enrichment
• Work with BigQuery, Apache Spark, and Apache Iceberg for heavy data processing
• Maintain high-load orchestration with custom Python nodes and complex workflows
• Own deployment and release processes (CI/CD)
• Optimize PostgreSQL performance under heavy load
• Investigate incidents and resolve root causes permanently

• Use AI engineering tools such as Claude in daily development

   

Requirements:

• 4+ years of experience in Data Engineering / Backend Engineering
• Strong Python skills
• Commercial experience with production LLM integration (OpenAI / Anthropic / Gemini)
• Strong knowledge of GCP (Cloud Functions, IAM, Networking)
• Experience with BigQuery, Spark, Iceberg
• Good understanding of Kubernetes
• Experience with workflow orchestration tools (n8n is a strong plus)
• Strong practical knowledge of PostgreSQL

• English level B2+

   

What is important for us:

• Ownership mindset
• Fast learning ability
• Strong problem-solving approach
• Active AI usage in engineering work
• Focus on practical results over unnecessary complexity

Data is at the core of every product decision we make. In this role, you’ll help shape and scale the data ecosystem behind products used by millions of users across international markets. You’ll work on high-volume data platforms, build reliable pipelines, and create the foundation that enables analytics, business intelligence, and data science teams to make fast, data-driven decisions. This is an opportunity to solve complex engineering challenges, optimize large-scale data systems, and directly influence the way data powers our products and business growth

In this role, you will
 

• Design and develop scalable, cloud-native data architectures in BigQuery, ensuring optimal performance, maintainability, and cost efficiency
• Build, automate, and maintain ETL/ELT pipelines using modern data stack tools and custom solutions
• Create and optimize business-critical data marts, aggregations, and analytical datasets for reporting, dashboarding, and product insights
• Implement data quality frameworks, monitoring systems, logging, and alerting mechanisms to ensure data reliability
• Continuously improve performance, scalability, and cost efficiency of production data infrastructure
• Partner closely with Data Science teams to build reliable data services, infrastructure, and pipelines supporting machine learning and advanced analytics initiatives
• Collaborate with product, analytics, and engineering teams to define and deliver scalable data solutions across the organization
  
  It’s all about you
   
• 3+ years of experience as a Data Engineer working with large-scale data platforms and high-volume datasets
• Strong expertise in data modeling, data warehousing concepts, and analytical data architecture design
• Proven experience designing and developing complex ETL/ELT workflows following BI and data engineering best practices
• Hands-on experience with BigQuery, including partitioning, clustering, query optimization, and cost management
• Strong SQL skills with the ability to write and optimize complex analytical queries using Google SQL
• Proficiency in Python and data processing libraries such as Pandas and PySpark
• Experience working with ClickHouse, including architecture understanding, performance optimization, and migration approaches
• Solid knowledge of Google Cloud Platform services, including Cloud Storage, Pub/Sub, DataForm, IAM, and service accounts
• Experience with Git, CI/CD practices, automated deployments, and environment management for data workflows
• Strong understanding of monitoring, logging, data validation, and production-grade data quality practices
• Excellent analytical thinking, problem-solving skills, and ability to work independently in a fast-paced environment
• Strong communication and collaboration skills with both technical and non-technical stakeholders
• Understanding of product metrics, experimentation frameworks, and data-driven decision making
• 2+ years of previous Software Engineering experience
• Experience optimizing storage and compute resources to improve infrastructure efficiency and reduce operational costs
• Experience integrating data platforms with BI tools such as Tableau or Looker
  
  Would be a plus
   
• Familiarity with supporting machine learning workflows and data infrastructure for Data Science teams
• Experience working with AI-powered development tools, including Claude Code, Claude Agents, or similar AI-assisted engineering solutions
  
  What we offer
• Care and support: 
• 20 paid vacation days, 15 sick days, and 6 additional days off for family events
• Up to 10 additional days off for public holidays
• 100% medical insurance coverage
• Sports and equipment reimbursement
• Team building events, corporate gifts, and stylish merch
• Maternity Recovery Support Allowance
• Financial and legal support
• Position retention and support for those who join the Armed Forces of Ukraine
• Participation in social initiatives supporting Ukraine
   
• Comfortable working environment:
• Work from our Kyiv hub or remotely with a flexible schedule 
• Modern equipment or depreciation of your own tools
  
  Investment in your future:
• Collaborate with a highly-skilled team of Middle & Senior professionals, sharing practical cases and expertise in the social networking niche
• 70% of our heads and leads have grown into their roles here – so can you!
• Performance-oriented reviews and Individual Development Plans (IDPs)
• Reimbursement for professional courses and English classes
• Corporate library, book club, and knowledge-sharing events
   
• Hiring process
• Intro call
• Technical interview
• Final  interview
• Reference check
• Offer
   

Start: asap
Duration: long-term
Location: any
English: B2

Summary
We are looking for a senior GCP Data Engineer / Data Architect to lead and execute an on-premise GCP data migration project.

This is a hands-on data-focused role, not a classic DevOps/Kubernetes role.

Main responsibilities:
* Lead and implement the migration of data workloads from on-premise infrastructure to GCP
* Design and build GCP data lake / lakehouse architecture
* Work hands-on with GCP data services such as BigQuery, Dataproc, Cloud Storage, Spark, and related tools
* Design and implement scalable data pipelines and data processing flows
* Handle data ingestion, storage, security, reliability, and cost optimization
* Work independently with the existing technical team and drive technical decisions

Required experience:
* Strong hands-on GCP data engineering experience
* BigQuery, Dataproc, Spark, and Cloud Storage
* Data lake / lakehouse architecture
* On-premise to cloud migration experience
* Ability to lead technical decisions and work independently

Nice to have:
* Terraform / Infrastructure as Code
* Dataflow, Pub/Sub, Cloud Composer / Airflow
* GCP IAM, networking, monitoring, and security

Full-time availability is preferred.
Please apply only if you have real hands-on experience with GCP data platforms and data migration projects.

Data Engineer / ETL Developer

Location: Remote

Duration: 3 months contract with potential for long-term extension
Budget: Up to $2800-3800/month (depends on your experience)
 

About the Role

We are looking for a Data Engineer / ETL Developer with 2–4 years of experience to support the migration of existing SSIS processes into a Microsoft Fabric environment. 
 

Responsibilities

• Migrate existing SSIS packages and ETL processes to Microsoft Fabric.
• Design, develop, and optimize ETL/ELT pipelines.
• Work closely with stakeholders to understand data integration requirements.
• Ensure data quality, reliability, and performance throughout the migration process.
• Troubleshoot and resolve data pipeline issues.
   

Requirements

• 2–4 years of experience as a Data Engineer, ETL Developer, or similar role.
• Strong experience with ETL development and data integration.
• Hands-on experience with SQL and relational databases.
• Experience working with SSIS (SQL Server Integration Services).
• Familiarity with Microsoft Fabric (preferred).
• Understanding of data warehousing concepts and best practices.
• Ability to work independently in a remote environment.
• Good communication skills in English.
   

Nice to Have

• Experience with Azure Data Services.
• Experience with cloud-based data platforms and migration projects.
• Knowledge of data modeling and performance optimization.
   

Engagement Details

• Fully remote.
• Initial 3-month engagement.
• Potential for long-term collaboration based on project needs and performance.

Key Responsibilities

• Design, develop, and optimize deep learning models using PyTorch.
• Build and maintain scalable training and evaluation pipelines.
• Lead model selection, experimentation, and fine-tuning activities.
• Improve training performance, efficiency, and scalability across large datasets and compute environments.
• Work with multimodal data sources, including video, images, and audio.
• Provide technical guidance on AI architecture, training strategies, and best practices.
• Collaborate closely with product, engineering, and research teams to translate business objectives into AI solutions.
• Support deployment and operationalization of machine learning models in production environments.

Required Experience

• 5+ years of experience in Machine Learning, Data Science, or AI Engineering roles.
• Strong hands-on experience developing deep learning models with PyTorch.
• Proven experience building and optimizing training pipelines for machine learning systems.
• Deep understanding of model training, hyperparameter tuning, and performance optimization.
• Experience working with computer vision, audio processing, or multimodal AI applications.
• Strong software engineering skills and experience developing production-ready ML solutions.
• Experience with cloud-based or distributed training environments.

Preferred Qualifications

• Experience applying machine learning techniques within behavioral science, human behavior analysis, affective computing, or related domains.
• Experience with video understanding, speech/audio analysis, or multimodal foundation models.
• Familiarity with MLOps practices, model monitoring, and scalable deployment architectures.
• Experience working with large-scale datasets and distributed training frameworks.

About the company:

Kickbite is a fast growing all-in-one marketing analytic tool, which collects, analyzes, and visualizes e-commerce data all in one place. It provides actionable insights to all decision-makers to scale online sales and improve return of advertising spends. Website: https://www.kickbite.io/

Responsibilities:

- Creating and maintaining existing ETL processes

- Transform raw data and raw events into valuable data for business users

- Performing end-to-end data flow checks

- Integration with the new data sources

Requirements:

- 4-5 years of related experience

- Excellent knowledge of Python programming language

- Extensive experience with SQL DB's, Bigquery

- Strong knowledge of REST API

- Version Control Systems (GIT)

What we offer:

- Competitive level of compensation transferred directly to bank account

- Full-time remote work 40 hours per week (approximately from 10.00 to 18.00)

- 20 working days of vacation per year

- Paid sick leaves up to 2 weeks per year

- Free time for educational purposes

Description
The ideal candidate brings solid Machine Learning and AI expertise, with hands-on experience building and operating data-driven and intelligent systems in production environments. Candidates must be comfortable defending architectural decisions and writing production-grade code in a high-performance environment.

Key Qualifications

Strong experience with Apache Spark, including optimization techniques for large-scale workloads
Strong experience with Spark Structured Streaming
Strong experience with Spark SQL and query performance optimization
Hands-on experience with Python for data processing, ML, and AI use cases
Practical experience in Machine Learning (ML), including model development and integration
Solid understanding of AI / GenAI concepts, with ability to apply them in real-world scenarios
Experience with Terraform and Databricks Asset Bundles
AI / ML Focus (Added Emphasis)

Proven practical ML experience, not only theoretical knowledge
Experience working with AI solutions before and/or during the GenAI era (classical ML is valuable)
Ability to integrate ML/AI components into scalable data platforms
Understanding of data pipelines supporting AI/ML workflows
Nice to Have

Familiarity with Databricks (good working knowledge; deep expertise is a plus but not required)
Experience with application development and Databricks Apps
Comfort working in Linux environments
Background in financial markets, such as market making, trading, or related domains
Knowledge of German or French is a plus
Working Style

Able to work independently, prioritize effectively, and proactively identify opportunities for improvement
Comfortable operating in a loosely defined scope, contributing not only to execution but also to defining priorities and driving work forward

We're looking for the right person to build that foundation from the ground up.
This is a hands-on leadership role with real ownership: you'll shape how data is collected, structured, and delivered — and grow a team around you as the function matures.

 

WHAT YOU'LL BE WORKING ON
 

• Design the target-state data architecture: Postgres CDC → Kafka → Snowflake → client-facing data products.
• Own tooling decisions across ingestion, orchestration, transformation, and quality layers.
• Implement CDC-based ingestion from PostgreSQL services (RDS, Aurora, EC2, K8s operator) using Debezium or equivalent.
• Build streaming and near-real-time pipelines with defined SLAs.
• Build a data quality control layer: checksums, reconciliation, schema validation, anomaly detection, and alerting.
• Define quality checkpoints across the full pipeline — from source capture through Snowflake to client delivery.
• Define and enforce data contracts with service-owning teams for core entities: transaction, merchant, settlement, and processing status.
• Build the external data delivery layer: financial settlement, transaction status, processor reconciliation, and client analytics.
• Design tenant separation and implement replay/reload mechanisms for failure recovery.
• Start hands-on, then gradually hire and grow a small data engineering team as the function matures.
  Build a pragmatic roadmap with concrete deliverables at 3, 6, and 12 months.
   

WHAT YOU NEED TO SUCCEED IN THIS ROLE
 

• 5+ years in data engineering with end-to-end ownership of production pipelines.
• Hands-on with Snowflake, PostgreSQL CDC (Debezium preferred), and Kafka.
• Solid AWS experience — S3, RDS, Aurora, and cloud data infrastructure.
• Data quality engineering mindset: monitoring, reconciliation, lineage.
• Comfortable defining data contracts and driving requirements with backend engineering teams.
• Technical leadership experience: project ownership, cross-team alignment, delivery under constraints.
• Kubernetes, Airflow/Prefect/Dagster, and dbt are strong pluses.
• Payments domain knowledge — settlement, transaction lifecycle, processor integrations — is a strong plus.
• Familiarity with gRPC, RabbitMQ, and reading Go/Python service code is expected.

• Project Description:

  Owns the technical direction, quality, and delivery enablement of the data platform within the team. Ensures scalable architecture, reliable releases, and strong engineering practices while enabling the team to deliver efficiently within defined engineering and release standards.

• Responsibilities:

  Architecture & Technical Ownership
  • Owns end-to-end technical design across Snowflake, dbt, ADF, and Azure components
  • Acts as the final technical decision authority within the team
  • Ensures alignment with enterprise architecture and data governance standards
  • Maintains architecture documentation and diagrams
  ________________________________________
  Engineering Excellence
  • Defines and enforces coding standards and best practices
  • Reviews critical PRs and ensures maintainability and scalability
  • Drives performance optimization and cost efficiency
  • Continuously improves development efficiency by reducing technical complexity and removing bottlenecks
  ________________________________________
  CI/CD & Release Ownership
  • Owns and improves CI/CD pipelines and deployment processes
  • Defines and enforces the release process, ensuring it is standardized and consistently followed
  • Establishes release criteria based on Definition of Done, testing, and required approvals
  • Ensures all changes meet release criteria before deployment to production
  • Oversees and coordinates production releases, including validation of readiness and risk assessment
  • Has authority to block releases that do not meet defined quality and governance standards
  • Ensures pipelines enforce quality gates (tests, validations, approvals where required)
  • Continuously improves release safety and reduces production incidents
  ________________________________________
  Quality & Definition of Done
  • Ensures adherence to Definition of Done (including testing and documentation)
  • Works with QA and team to ensure test coverage aligns with acceptance criteria
  ________________________________________
  Team Enablement & Mentoring
  • Mentors engineers and supports technical growth
  • Leads knowledge sharing and supports complex problem-solving
  ________________________________________
  Planning & Delivery Support
  • Contributes to sprint planning with technical estimates, risks, and dependencies
  • Owns identification and prioritization of technical debt in collaboration with the Product Owner
  ________________________________________
  Production Ownership
  • Leads incident analysis and root cause resolution
  • Ensures monitoring, alerting, and data quality validation mechanisms are in place across pipelines and in production
  • Maintains runbooks for operational readiness
  ________________________________________
  Ways of Working
  • Collaborates closely with Product Owner and Scrum Master to align technical delivery with business priorities; supports team continuity and helps facilitate meetings when the Scrum Master is absent
  • Proactively works with upstream data source teams and other squads to resolve cross-team dependencies, data issues, and production bugs
  • Drives technical alignment across teams/squads where shared data models, pipelines, or integrations are involved
  • Acts as the primary technical point of contact for external teams and stakeholders

• Mandatory Skills Description:

  7+ experience in Data Engineering and development of DWH solutions
  Hands-on experience with Snowflake, dbt, Azure Data Factory
  Experience with Azure Cloud Solutions, Azure DevOps
  Practical experience and understanding of CI/CD processes
  Experience of technical leadership in Agile SDLC

• Nice-to-Have Skills Description:

  Experience with AWS and/or GCP cloud solutions
  Certifications related to DWH

• Languages:
  • English: C1 Advanced

Gypsy Collective is looking for a DWH Engineer to join our team and help build scalable, reliable, and high-performance data warehouse solutions that support analytics and business decision-making. We are looking for an experienced and proactive engineer who can work across the full data lifecycle - from ingestion and modeling to optimization, monitoring, and automation - while collaborating closely with analysts, developers, and business stakeholders.

    📅 Your daily adventures include:

• Designing, building, and maintaining scalable Data Warehouse architectures aligned with business needs;
• Developing and optimizing ETL/ELT pipelines using Python, Airflow, and custom solutions;
• Working with DWH/Data Lake technologies including PostgreSQL, Trino, and BigQuery;
• Implementing incremental loads, CDC, backfills, and reprocessing strategies;
• Optimizing query performance, data models, and pipeline execution;
• Ensuring data quality through validation, automated testing, monitoring, and alerting;
• Integrating new data sources (APIs, third-party systems, raw data) without disrupting existing pipelines;
• Collaborating with analysts, engineers, BI teams, and business stakeholders to translate requirements into scalable data solutions;

• Reviewing code, mentoring engineers, and contributing to data engineering standards and best practices.

  
  🧭 What makes you a great match:

• Strong SQL skills: complex queries, CTEs, window functions, analytical queries (5+ years experience);
• Knowledge of Python or other scripting languages for data transformations (4+ years experience);
• Deep understanding of DWH concepts: ETL/ELT, Data Vault, Kimball, Star/Snowflake schemas (5+ years experience);
• Experience with Airflow or other data pipeline orchestrators (5+ years experience);
• Hands-on experience with modern DWH and query engines: BigQuery, Snowflake, Redshift, ClickHouse, Vertica, AWS Athena, Trino (5+ years experience);
• Confident use of Git; experience with team workflows (pull requests, rebasing, merge conflict resolution (5+ years experience);

• Understanding of server and cloud infrastructure: basic skills in configuration, maintenance, monitoring, and load control (3+ years experience).
   

  ⭐ Nice to have:

• Experience with CDC tools and streaming data sources;
• Knowledge of Docker, Kubernetes, and Infrastructure as Code (Terraform);
• Experience with cloud platforms such as AWS, GCP, or Azure;

• Familiarity with data governance, data cataloging, and lineage tools.
   

  Benefits:

• 💸 Flexible payment options: choose the method that works best for you.
• 🎁 Financial perks: Bonuses for holidays, B-day, work milestones and more - just to show we care.
• 📈 Learn & grow: We cover courses and certifications — and offer real opportunities to grow your career with us.
• 🥐 Benefit Сafeteria: Choose what suits you — sports, language courses, therapy sessions, and more.
• 🎉 Stay connected: From team-building events to industry conferences — we bring people together online, offline, and on stage.
• 💻 Modern Equipment: We provide new laptops along with essential peripherals like monitors and headphones for a comfortable workflow.
• 🕘 Your schedule, your rules: Start your day at 9, 10, or even 11 — we care about results, not clock-ins.
  
  If this role sounds like a good fit for you, we’d love to hear from you — apply and tell us a bit about yourself.

We are looking for a Senior Data Engineer to join the Identity team at Equals5.  We are building a dynamic data ecosystem in which AI is deeply integrated—both as a productivity multiplier and as a core component of our data-processing logic for identity data enrichment and scoring.

You will own the infrastructure that handles over 10,000 executions per minute, ensuring stability, scalability, and data integrity. You will work with a modern stack on Google Cloud Platform, utilizing Cloud Functions and Kubernetes.

We are looking for an engineer who improves infrastructure, automates everything, and is eager to implement LLM-based logic directly into high-load data flows.
 

Responsibilities
 

• AI-Driven Data Scoring: Design and implement pipelines that utilize LLMs to analyze and score identity data in real-time. You will integrate AI models directly into the decision-making loop, balancing accuracy with latency and cost.
• Own the Data Architecture: Architect scalable data solutions using GCP and Python. You will manage data storage and retrieval using BigQuery and Apache Iceberg to support querying of TBs of data.
• Heavy Data Processing: Utilize Apache Spark for data transformations and batch processing when lightweight cloud functions are not enough.
• Manage High-Load Orchestration: Maintain and optimize our system instances. It involves complex dataflows, custom Python nodes, and performance tuning for 10k+ execs/minute.
• Release Lifecycle (CI/CD): Take ownership of the deployment process, ensuring that updates to pipelines and infrastructure are released safely with proper testing and rollback strategies.
• Database Optimization: Manage PostgreSQL performance under heavy load, optimizing complex queries and indexing strategies.
• Active AI Usage: Use Claude Code and other AI engineering tools to accelerate your own development, refactoring, and testing processes.
• Incident Resolution: Proactively monitor the system. When alerts fire, you investigate the root cause—whether it’s a database lock or an LLM hallucination—and fix it permanently.
   

Requirements
 

• 4-5+ years of experience in Data Engineering or Backend Engineering with a strong data focus.
• Production AI Integration: Experience integrating LLMs (OpenAI, Anthropic, Gemini) into production applications via API. You understand latency, token limits, and how to structure data for AI scoring.
• Expertise in GCP: Understanding of Google Cloud Platform (Cloud Functions, IAM, Networking).
• Strong Python: You write clean, efficient, and testable code. You are comfortable building custom logic where standard tools fall short.
• Big Data Stack: Experience with BigQuery, Apache Spark, and modern table formats like Apache Iceberg.
• Kubernetes (K8s): Experience deploying and scaling services in containerized environments.
• Workflow Automation: Understanding of workflow orchestration tools at a deep technical level. N8N is a big part of our domain, so familiarity with it is highly valuable.
• PostgreSQL Mastery: Proven ability to handle heavy write/read loads and optimize schemas.
• English: B2+ (Upper-Intermediate) or higher.
   

Culture & Mindset
 

• Self-improvement: You are a fast learner. You don't fear AI replacing you; you master it to replace your manual tasks.
• Ownership: You treat the Identity domain as your own business. If a scoring model drifts or a pipeline slows down, you notice it and fix it without being asked.
• Internal Locus of Control: You take responsibility for outcomes. If an external API fails, you build a fallback mechanism instead of just blaming the provider.
• Get It Done: You prioritize shipping value. You know when to use a simple script and when to build a complex architecture.
• Openness: You share knowledge freely. If you find a better way to prompt the AI for scoring, you share it with the team.
   

What We Offer
 

• Fully remote with flexible hours (aligned with EU timezones for syncs).
• AI-Native Environment: We provide licenses for Claude Code and encourage using the bleeding edge of AI tech for both daily coding and product features.
• High-Impact Role: You will directly influence how we identify and score users, impacting the core business logic.
• Cross-functional visibility: Work closely with Product and Tech Leads to shape the future of Identity.
• No Bureaucracy: Fast decisions, no legacy processes, focus on results.

About the Company

The firm is a specialized consultancy in statistics, data science, and AI. We work across industries - including the public sector, insurance, and other data-intensive domains. For this role, we are specifically looking to strengthen our teams working on public-sector projects.

In these projects, we develop solutions that are not only technically challenging but also create a tangible positive impact on society. We are looking for strong Senior Data Engineers who are drawn to complex problems rather than intimidated by them - and who want to build systems that truly make a difference.

Role Overview

Do you want to do more than maintain existing data infrastructures? Are you interested in strategically leading data engineering initiatives and helping shape sustainable data platforms from the ground up? Do you think in terms of data architecture, scalability, quality, security, and operational excellence - and want your work in the public sector to have a visible impact? Then you are in the right place.

As a Senior Data Engineer, you will take technical ownership of the data engineering domain: from architecture and key technical decisions to design, implementation, operations, monitoring, and continuous optimization of data pipelines and platforms. You will develop robust, scalable data solutions that not only work today but remain maintainable and reliable in production over the long term.

What You'll Be Doing

• Lead data engineering topics within projects, drive strategic decisions, and take overall responsibility for this area.
• Design, develop, maintain, and optimize ETL/ELT pipelines.
• Ensure data pipelines remain scalable, maintainable, and efficient as data volumes and the number of data sources grow.
• Define and establish standards and best practices for pipeline design, testing, monitoring, operations, and performance.
• Review changes to data pipelines and data architecture, ensuring compliance with quality standards.
• Work hands-on with the Cloudera/Apache stack as well as modern data stack technologies.
• Ensure data quality, access control, privacy, and security throughout the entire data lifecycle.

Requirements

• Several years of solid experience in data engineering and in a technical leadership role.
• Proven experience designing and implementing scalable ETL/ELT pipelines.
• Experience with both batch and streaming data pipelines.
• Practical experience with modern data stack tools such as dbt and S3/Object Storage.
• Strong understanding of data architecture, scalability, maintainability, and data engineering best practices.
• Excellent SQL skills and solid programming skills, ideally in Python and/or Java.
• Experience with code reviews, establishing engineering standards, and performance optimization.
• A strong sense of ownership and the ability to independently lead and deliver technical initiatives.
• Experience building custom data pipelines from scratch, including without relying on prebuilt frameworks or specialized cloud services.
• Demonstrated experience processing various data formats, such as tabular data, JSON, text, images, and other formats.
• Experience handling confidential, security-sensitive, and personal data.
• Knowledge of Cloudera tooling and the Apache ecosystem is a plus, but not required.

Candidate Requirements

• Ukrainian native speaker
•  English proficiency: B2 or higher

We are looking for a Big Data Engineer to join our growing team. In this role, you will be responsible for the design and development of our big data processing engine — a heavy ETL pipeline handling complex logic including geo-location analysis, running across both cloud and on-prem environments.

Requirements:

• 4+ years of experience in Data Engineering
• 2+ years of hands-on experience with Scala (must) or Java (optional)
• 2+ years of experience with SQL and NoSQL DBs
• Hands-on experience with Spark & PySpark (must)
• Experience with Airflow (must)
• Experience with AWS and/or GCP
• Experience with Kubernetes (k8s) and on-prem deployments
• Solid understanding of distributed systems design, pipeline architecture, and partitioning strategies
• Deep understanding of Agile principles, practices and values
• Solid understanding of design patterns, software development techniques, and clean code practices
   

Advantage:

• Experience with Apache Iceberg, Helm Charts, or data catalog tools
• Experience with Spring Framework
• Experience with microservice architecture
• Experience working with DevOps and CI/CD practices
• Customer-facing experience or comfort working directly with clients
• Bachelor's, BSc or Master's degree in Computer Science or Engineering from a leading academic institution

N-iX is a global software development service company that helps businesses across the globe create next-generation software products. Founded in 2002, we unite 2,400+ tech-savvy professionals across 40+ countries, working on impactful projects for industry leaders and Fortune 500 companies. Our expertise spans cloud, data, AI/ML, embedded software, IoT, and more, driving digital transformation across finance, manufacturing, telecom, healthcare, and other industries. Join N-iX and become part of a team where your ideas make a real impact.

We are looking for a Senior Machine Learning Engineer to lead the development and deployment of advanced AI models. In this role, you will be responsible for the end-to-end lifecycle of our machine learning systems, from architectural design and data preprocessing to model training, optimization, and production deployment. 

You will work at the intersection of generative AI and traditional machine learning, building the engines that power two strategic AI initiatives spanning NLP-based document intelligence and predictive analytics. Operating within a structured agile delivery model with formal review gates, you will ensure our models are not just accurate, but also robust, explainable, and deployable within highly secure, on-premise environments . 

Key Responsibilities:

1. LLM & NLP Pipelines 

• Design and fine-tune Large Language Model (LLM) pipelines to interpret complex regulatory texts (e.g. standards, building codes) and extract structured rules. 
• Convert natural language requirements into computer-processable formats (e.g., logic tuples) that can be executed by downstream compliance engines. 
• Implement RAG (Retrieval-Augmented Generation) architectures to enable semantic querying of technical documentation and historical project data. 
• Optimize prompt strategies (few-shot learning, chain-of-thought) to improve model performance on domain-specific tasks without extensive retraining.

1. Predictive & Analytical Models (Supply Chain) 

• Develop time-series forecasting models to predict material demand and spend categories, integrating internal ERP data with external market signals. 
• Build classification and anomaly detection models to assess supplier risk profiles based on financial health, delivery performance, and geopolitical factors. 
• Design algorithms for multi-objective optimization(e.g., balancing cost vs. lead time vs. risk) to support procurement decision-making. 

1. MLOps & Productionization 

• Containerize models using Docker/Kubernetes and deploy them into secure, on-premise inference environments. 
• Build automated training and inference pipelines using tools like Kubeflow or MLflow to ensure reproducibility and scalability. 
• Optimize model inference latency and resource usage (e.g., quantization, distillation) to run efficiently on available hardware. 
• Implement monitoring systems to track model drift and performance in production, establishing feedback loops for continuous improvement. 

Requirements:

•  5+ years of experience in Machine Learning Engineering, with a proven track record of deploying models into production environments. 
• Expert proficiency in Python and standard ML libraries (PyTorch, TensorFlow, Scikit-learn, Pandas, NumPy). 
• Strong experience with transformer architectures (BERT, GPT, Llama) and NLP frameworks (Hugging Face, LangChain). 
• Proficiency with MLOps tools and practices, including containerization (Docker), orchestration (Kubernetes), and experiment tracking (MLflow). 
• Ability to design data preprocessing pipelines for both structured (SQL, tabular) and unstructured (text, PDF) data. 
• Strong grasp of algorithmic principles for implementing custom logic, such as graph traversal or geometric computations.
• Ability to quickly learn and apply ML techniques to specialized domains like engineering, supply chain, or other highly regulated industries. 
• Experience working in agile environments (Sprints) while adhering to rigorous engineering standards and documentation requirements.
• Strong communication skills to work effectively with Data Scientists, Backend Engineers, and Domain Experts to align technical solutions with business needs. 

Why This Role? 

You will be building the intelligence that drives critical enterprise infrastructure. Your models will not just generate text or predictions; they will directly influence the design of complex industrial systems and the resilience of supply chains. If you are ready to apply advanced ML to tangible, high-stakes problems in a rigorous engineering environment, join us.

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers

About the Role

We are seeking a highly analytical and detail-oriented Data QA Engineer to ensure the quality, accuracy, and reliability of data platforms, pipelines, and reporting systems. The ideal candidate has strong SQL expertise, a solid understanding of data engineering concepts, and the ability to independently investigate data issues, validate complex data flows, and support data quality initiatives across the organization.

Key Responsibilities

• Validate data accuracy, completeness, and consistency across multiple systems and environments.
• Perform data quality testing for ETL/ELT pipelines, data warehouses, and reporting solutions.
• Design and execute test plans, test cases, and validation strategies for data-centric applications.
• Write complex SQL queries for data validation, reconciliation, and root cause analysis.
• Analyze and troubleshoot data discrepancies across source and target systems.
• Review, understand, and debug Python-based data pipeline code when necessary.
• Collaborate with data engineers, software engineers, analysts, and business stakeholders to ensure data integrity.
• Utilize AI-powered tools to improve testing efficiency, investigation, analysis, and productivity.
• Work independently and take ownership of data quality initiatives throughout the project lifecycle.

Required Skills

Database Engineering & SQL

• Strong knowledge of relational databases and database concepts.
• Advanced SQL skills with the ability to write complex queries for:
  • Data validation
  • Data reconciliation
  • Root cause analysis
  • Data quality investigations
• Experience working with large datasets and optimizing data validation processes.

Data Infrastructure & Concepts

• Solid understanding of data engineering principles and best practices.
• Experience validating data pipelines and data processing workflows.
• Understanding of ETL/ELT architectures and data movement processes.
• Knowledge of data warehouse concepts and dimensional modeling.
• Familiarity with OLAP technologies and modern cloud data platforms such as Redshift, Snowflake, BigQuery, or equivalent.

Programming Skills

• Strong proficiency in Python.
• Ability to read, understand, analyze, and debug data pipeline code.
• Experience using Python for data validation, testing, or data analysis activities.

Additional Requirements

• Experience utilizing AI tools to improve testing, analysis, and productivity.
• Strong analytical thinking and problem-solving skills.
• Excellent attention to detail.
• Ability to work independently with minimal supervision.
• Strong communication and collaboration skills.

Nice to Have

• Experience with data quality frameworks and monitoring tools.
• Experience in testing cloud-based data platforms.
• Knowledge of Apache Airflow, dbt, Spark, or similar data technologies.
• Experience working in Agile/Scrum environments.
• Familiarity with CI/CD practices for data engineering projects.