Jobs Data Engineer

Join the Dataforest Team!

We are actively expanding our team and looking for strong new talent to join our ranks. If you are a Data Engineer seeking a friendly team, a healthy working environment, and a flexible schedule — you have found the right place to send your CV!

You'll be a great fit if you have:

- 2+ years of experience with Python;
- 2+ years of experience as a Data Engineer;
- Experience with Pandas;
- Experience with SQL DB / NoSQL (Redis, Mongo, Elasticsearch) / BigQuery;
- Familiarity with Amazon Web Services;
- Experience with Kafka;
- Knowledge of data algorithms and data structures is a MUST;
- Working with high volume tables 10m+.

What you'll do:

- Create ETL pipelines and data management solutions (API, Integration logic);
- Different data processing algorithms;
- Involvement in creation of forecasting, recommendation, and classification models.

Would be a plus:

- Experience with Spark (pyspark);
- Experience with Airflow;
- Experience in statistics;
- Knowledge of DS and Machine learning algorithms.

We ensure your growth with:

- Great networking opportunities with international clients, challenging tasks;
- Building interesting projects from scratch using new technologies;
- Personal and professional development opportunities;
- Competitive salary fixed in USD;
- Paid vacation and sick leaves;
- Flexible work schedule;
- Friendly working environment with minimal hierarchy;
- Team building activities, corporate events.

Dataforest is looking for a growth-oriented Data Engineer to join our friendly team. As a Data Engineer, you will solve engaging problems using cutting-edge technologies for data collection, processing, analysis, and monitoring.

If you aren't afraid of challenges, this position is just for you!

You'll be a great fit if you have:

- 1+ years of experience as a Data Engineer;
- Strong experience with Python;
- Proven experience with Databricks and Datafactory;
- Experience with AWS/Azure;
- Experience building ETL / ELT pipelines;
- Proficiency in SQL.

What you'll do:

- Developing ETL/ELT pipelines and data management solutions;
- Implementing data processing algorithms;
- Writing and optimizing SQL queries for data extraction and analysis;
- Analyzing data and applying processing algorithms to solve business problems.

We ensure your growth with:

- Working with a high-skilled engineering team on engaging and complex projects;
- Hands-on experience with the latest technologies;
- Direct communication with international clients and challenging tasks;
- Opportunities for personal and professional growth;
- Competitive salary pegged to USD;
- Paid vacation and sick leave;
- Flexible working hours;
- Friendly working atmosphere with zero bureaucracy;
- Plenty of team traditions — corporate parties, team buildings, themed events, and much more!

Join Sigma Software to build large-scale data infrastructure powering a real-time AdTech platform processing hundreds of millions of auction requests daily. We are looking for a Senior Data Engineer who enjoys solving complex distributed data challenges and building production-grade ML-oriented data systems.

You will become part of a dedicated Sigma Software team developing predictive modeling and optimization capabilities for a live advertising ecosystem. The role combines large-scale event processing, streaming and batch pipelines, experimentation infrastructure, and high-throughput data engineering in a cloud-native environment.

We as a company offer the opportunity to work on impactful global products, collaborate with experienced engineers, and contribute to architecture decisions while growing your expertise in large-scale distributed systems and modern data platforms.

Customer

Our Customer is a technology company operating supply-side infrastructure within the programmatic advertising ecosystem. The company manages a large-scale ad exchange handling hundreds of millions of auction requests per day and is actively investing in predictive decisioning technologies to optimize advertising outcomes in real time.

Project

The project focuses on building a predictive modeling and optimization platform on top of a live ad exchange environment. The platform performs real-time supply scoring and filtering, contextual performance estimation, look-alike audience generation, and multi-objective optimization under business constraints.

The solution processes massive-scale event and auction datasets and includes feature engineering pipelines, streaming and batch ingestion, experimentation infrastructure, point-in-time-correct training data generation, and ML-oriented data services with strict operational reliability and compliance requirements.

Requirements

• 5+ years of experience in Data Engineering
• At least 2 years of experience working with production ML or large-scale analytics pipelines
• Expert-level SQL skills including window functions and incremental processing patterns
• Strong Python skills for production-grade pipeline development
• Hands-on experience with Spark or PySpark
• Experience designing ETL / ELT pipelines with Airflow, Cloud Composer, Dagster, or similar tools
• Experience working with cloud data warehouses at scale, preferably BigQuery
• Strong understanding of data modeling and point-in-time correctness
• Experience working with event-driven or clickstream datasets at very large scale
• Experience supporting business-critical production pipelines

• Upper-Intermediate English level or higher

   

WILL BE A PLUS

• Experience with GCP services including Dataflow, Pub/Sub, GCS, and Beam
• Experience building streaming or near-real-time ingestion systems
• Understanding of feature stores, train/serve skew, and label leakage prevention
• Experience in AdTech or auction-based environments
• Experience handling delayed or incomplete labels in ML systems
• Experience with dbt or similar transformation frameworks
• Experience delivering solutions into Customer-owned infrastructure
• Knowledge of GDPR/CCPA-related privacy engineering practices
• Experience with experimentation infrastructure and statistical validation pipelines
• Experience working in hybrid cloud/on-prem Linux environments
• Terraform and Kubernetes experience

• Experience optimizing warehouse cost and performance

   

Personal Profile

• Strong analytical and problem-solving skills
• Ownership-oriented mindset
• Ability to work independently in a client-facing environment
• Strong communication and documentation skills
• Comfortable working in a fast-paced engineering environment
• Collaborative and proactive attitude

Шукаємо Senior / Lead Software Engineer (Data), який допоможе будувати й масштабувати production-grade data systems: пайплайни, сервіси та storage-рішення для збору, обробки, зберігання й доставки даних. Це роль для інженера з сильним software engineering mindset, який вміє працювати не лише на рівні реалізації, а й на рівні архітектури, data modeling, reliability, performance та technical leadership. Очікуємо, що людина зможе брати end-to-end ownership за критичні data workflows і впливати на стандарти команди.

Обовʼязки:

• Проєктувати, розробляти та підтримувати batch/streaming data pipelines і data services.
• Приймати технічні рішення щодо ingestion, orchestration, storage, трансформацій і delivery.
• Будувати ETL/ELT-рішення з використанням Python, Airflow, Cron.
• Інтегрувати дані з API, логів, NoSQL, IoT/embedded-джерел та внутрішніх сервісів.
• Проєктувати схеми зберігання, моделі даних і структури баз даних.
• Оптимізувати рішення з погляду performance, scalability, cost та maintainability.
• Забезпечувати data quality, observability, reliability і стабільність production-процесів.
• Планувати та реалізовувати міграції даних між платформами й форматами.
• Документувати схеми, трансформації, data lineage та ключові технічні рішення.
• Брати участь у code review, design discussions, incident resolution.
• Менторити інших інженерів і розвивати engineering standards у data-напрямку.
   

Вимоги:

• Значний досвід як Senior / Lead Software Engineer, Data Engineer або Backend Engineer у data-intensive середовищі.
• Сильний Python і досвід побудови production-ready рішень.
• Досвід роботи з Apache Airflow.
• Досвід роботи з MongoDB або іншими NoSQL-рішеннями.
• Сильне розуміння проєктування баз даних і data models: schema design, indexing, normalization/denormalization.
• Досвід роботи з великими обсягами даних, high-throughput або streaming workloads.
• Досвід із Azure Blob Storage або аналогічними object storage.
• Впевнене знання SQL та оптимізації запитів.
• Глибоке розуміння ETL/ELT, incremental processing, datalake/warehouse principles.
• Розуміння testing, observability, fault tolerance, idempotency, recovery для data systems.
• Досвід із Git, CI/CD, production deployment.
• Досвід технічного лідерства, mentoring, design review і роботи зі стейкхолдерами.
   

Буде плюсом:

• Snowflake, Elasticsearch
• Kafka або інші streaming-платформи
• Grafana / Prometheus
• Ansible або інші IaC / automation tools
• Power BI
• Досвід із cloud DWH, governance, lineage, internal data tools
  
  Надаємо бронювання*
  Офіс з повноцінним укриттям*

Join an exciting journey to build and scale a global Media Data Foundation for a leading global organization based in Europe. This platform brings together paid media, digital analytics and marketing performance data from dozens of advertising and measurement platforms. It unlocks reliable media reporting, marketing ROI measurement and data-driven investment decisions across markets.

As a Senior Data Engineer, you will play a pivotal role in shaping and implementing the platform's architecture. The focus is on hands-on technical execution and collaboration with cross-functional teams.

Your work will turn raw media and marketing data into trusted, harmonised data products used by marketing, media and analytics teams around the world. You will use modern tools to solve complex challenges, working alongside skilled architects, engineers and business stakeholders.

Key Responsibilities:

• Hands-On Development: Build, maintain and optimize data pipelines for ingestion, transformation and delivery of media data (Meta, Google Ads, DV360, TikTok, The Trade Desk, GA4, CM360 and others).
• Create and implement scalable solutions to handle many data sources, frequent API and schema changes, and high volumes of data.
• Data Modeling & Warehousing: Design and maintain efficient, harmonised data models for cross-platform media reporting in Snowflake.
• Lead the move of transformation logic to dbt, including orchestration, testing, documentation and CI/CD.
• Collaboration: Partner with Solution Architects to translate high-level designs into detailed implementation plans.
• Work closely with Technical Product Owners, media agencies and market teams to align data solutions with business needs.
• Platform Development: Enable data readiness for media dashboards (Power BI), marketing mix modelling / ROI analytics and campaign taxonomy governance.
• Implement robust frameworks to monitor data freshness, completeness, quality and pipeline performance, with automated alerting.
• Security & Compliance: Implement robust access controls and data protection across the pipeline and for data shared with internal and external consumers.
• Ensure compliance with data privacy regulations (e.g. GDPR, CCPA) and internal data governance policies (data product tiers, tagging, naming standards).
• Monitor and address potential vulnerabilities, including service account and secret management, ensuring the platform follows security best practices.

Requirements:

• 4+ years of experience showing technical expertise and critical thinking in data engineering.
• Hands-on experience with dbt and strong Python programming skills.
• Proficiency in Snowflake and strong expertise in data modeling are essential.
• Demonstrated experience with marketing, media or advertising data (ad platform APIs, campaign hierarchies, web analytics) is required.
• In-depth understanding of security and governance in Snowflake: RBAC, secure views, data sharing, masking and row access policies, object tagging.
• Ability to design, implement and promote solution patterns and standards for complex, multi-source data platforms.
• Familiarity with multiple cloud platforms (Azure or GCP preferred, with a focus on Azure, including ADF).
• Experience with ETL/ELT and data integration tools (Adverity, Fivetran or similar) and batch orchestration.
• Competence in SQL, relational database design and efficient methods for data retrieval and preparation at scale.
• Proven ability to collect and process raw data at scale, including scripting, REST API integration and SQL querying.
• Experience with BI tools (Power BI preferred) and supporting reporting consumers.
• Experience working in global environments and collaborating with virtual teams and external agencies.
• A Bachelor's or Master's degree in Data Science, Computer Science, Economics or a related discipline.

We are looking for a Mid Data Engineer in Kraków, Poland who will own the company data lake — the system of record for millions of financial events a day (bets, wallet movements, live odds) across 12M+ active users. You decide how that data lands, is stored, retained, and governed on S3 + Snowflake/Databricks, so analytics, finance, and regulators all see accurate, reconciled data with zero drift from source.

Domain: Regulated iGaming / wallet & ledger data. Audit-heavy: regulators, finance and analytics all consume the same tables. Millions of financial events per day, terabyte-plus scale.
 

Key Responsibilities

• Own lakehouse architecture, including Bronze/Silver/Gold layers, Iceberg/Delta tables, and schema evolution.
• Build CDC streaming pipelines using Kafka, Debezium, or similar technologies, handling late and duplicate events.
• Optimize data storage and query performance through partitioning, compaction, and file sizing.
• Manage data retention, archival, regulatory requirements, and GDPR deletion.
• Ensure data accuracy through freshness SLAs, drift detection, and reconciliation with wallet and ledger systems.
• Own data governance, including cataloging, lineage, access control, PII masking, encryption, and audit trails.
• Monitor ingestion health, data quality, and cloud storage/compute costs.

Requirements

• 5+ years of experience in Data Engineering, with proven ownership of large-scale data lake or lakehouse architectures.
• Production experience with cloud storage (S3/GCS) and Snowflake, Databricks, or BigQuery.
• Hands-on experience with open table formats (Iceberg, Delta, or Hudi).
• Strong understanding of partitioning, file layout, and query optimization at terabyte scale.
• Experience with CDC and streaming ingestion (Kafka, Debezium, or equivalent).
• Strong SQL and data modeling skills, with a solid understanding of relational databases and financial ledger systems.
• Experience with data governance, including catalogs, lineage, access control, PII, and retention.
• Python and workflow orchestration experience (Airflow, Dagster, or similar).

• Hands-on experience building and deploying AI agents and LLM-powered workflows to automate complex processes at scale.

   

  Nice to have: Experience in fintech, iGaming, or other regulated, audit-heavy industries.

You'll own the company's data lake, managing millions of financial events daily (bets, wallet movements, and live odds) across 12M+ active users. You'll define how data is ingested, stored, retained, and governed across S3 and Snowflake/Databricks, ensuring accurate, reconciled, and reliable data for analytics, finance, and regulatory needs.

Key Responsibilities

• Own lakehouse architecture, including Bronze/Silver/Gold layers, Iceberg/Delta tables, and schema evolution.
• Build CDC streaming pipelines using Kafka, Debezium, or similar technologies, handling late and duplicate events.
• Optimize data storage and query performance through partitioning, compaction, and file sizing.
• Manage data retention, archival, regulatory requirements, and GDPR deletion.
• Ensure data accuracy through freshness SLAs, drift detection, and reconciliation with wallet and ledger systems.
• Own data governance, including cataloging, lineage, access control, PII masking, encryption, and audit trails.
• Monitor ingestion health, data quality, and cloud storage/compute costs.

Requirements

• 5+ years of experience in Data Engineering, with proven ownership of large-scale data lake or lakehouse architectures.
• Production experience with cloud storage (S3/GCS) and Snowflake, Databricks, or BigQuery.
• Hands-on experience with open table formats (Iceberg, Delta, or Hudi).
• Strong understanding of partitioning, file layout, and query optimization at terabyte scale.
• Experience with CDC and streaming ingestion (Kafka, Debezium, or equivalent).
• Strong SQL and data modeling skills, with a solid understanding of relational databases and financial ledger systems.
• Experience with data governance, including catalogs, lineage, access control, PII, and retention.
• Python and workflow orchestration experience (Airflow, Dagster, or similar).
• Hands-on experience building and deploying AI agents and LLM-powered workflows to automate complex processes at scale.
• Nice to have: Experience in fintech, iGaming, or other regulated, audit-heavy industries.

Warsaw (hybrid)!

We’re looking for a Senior Data Engineer to join an international product company developing a complex B2B platform for enterprise customers worldwide.

You’ll build and optimize distributed data-processing systems using Java and modern data technologies, handling complex data workflows across interconnected services. This role offers end-to-end ownership, challenging performance problems, and the opportunity to influence technical decisions in a large-scale production environment.

What you’ll do:

• Build scalable data pipelines and distributed processing solutions.
• Develop Java services for data ingestion, transformation, and integration.
• Design data models and optimize database performance for large datasets.
• Improve system reliability, fault tolerance, and processing efficiency.
• Troubleshoot production issues and optimize distributed workloads.
• Use AI-assisted development tools to accelerate delivery while maintaining code quality.

What we’re looking for:

• Strong professional Java development experience.
• Deep understanding of distributed systems, concurrency, and JVM performance.
• Hands-on experience building and optimizing distributed data pipelines.
• Strong knowledge of data modeling, database design, and query optimization.
• Experience with messaging systems and integration between distributed services.
• Ability to take ownership of complex technical challenges and production systems.
• Practical experience with AI-assisted coding tools.

Nice to have: Real-time data processing, analytical databases, CI/CD, Docker, Kubernetes, TypeScript, or React.

Brainway is looking for a Senior Data Engineer to join a client project developing an AI-powered B2B platform.

You will help turn product data, customer data and external business data into reliable analytics that people can use to make decisions. Working with an existing product and analytics environment, you will develop data models, define meaningful metrics and deliver analytical features together with engineering and product teams.

This role combines hands-on data work with product thinking. You should be comfortable investigating unfamiliar datasets, asking what a metric actually means and following the work through to a useful, validated result.

What you will do

• Work with product and business stakeholders to turn questions into clear analytical requirements and measurable outcomes.
• Explore existing datasets, understand their business context and resolve inconsistencies in how data is interpreted.
•  Design analytical data models, write efficient SQL and build the transformations needed for reporting and product analytics.
•  Develop dashboards and customer-facing analytical features that make complex information easy to understand.
•  Define how new data sources should be integrated, including field mappings, transformation rules and acceptance checks.
•  Validate calculations and outputs, investigate discrepancies and improve data quality, performance and reliability.
•  Work with engineers on secure handling of customer data and appropriate access to analytical outputs.

•  Use AI-assisted tools to accelerate investigation and delivery while keeping results explainable, reproducible and checked.

   

What we are looking for

• 5+ years of relevant experience across data engineering, analytics engineering, BI or a closely related discipline.
• Strong SQL and practical experience designing data models and defining business metrics.
• Experience delivering dashboards or analytical products used in production.
• Hands-on experience with PostgreSQL, BigQuery or comparable relational and analytical platforms.
• A solid understanding of data transformation, integration and validation practices.
• The ability to work through incomplete requirements, ask useful questions and make sound product decisions.
• Clear communication with technical and non-technical stakeholders, including customers when needed.
• Confidence using AI tools alongside independent technical judgment and careful verification.
• Strong written and spoken English.
   

Useful additional experience

• Embedded analytics or customer-facing BI.
• Semantic layers, shared metric definitions, data governance or data-quality monitoring.
• Integrations with CRM, ERP, HRIS or other enterprise systems.
• APIs, Node.js or TypeScript.
• Version-controlled data and analytics workflows.

• Product analytics or analytics for enterprise operations.

   

Why this role

You will work at the intersection of engineering, analytics and product, with ownership of how business questions become dependable analytical capabilities. The work includes improving an existing foundation and helping shape how new data sources and analytical features fit into the product.
 

If you enjoy understanding the problem behind a report as much as building the data model behind it, we would like to hear from you.

We are an IT company looking for proactive, communicative engineers to join a product team on a long-term basis. We focus on results and care about the growth of our people.

Currently, we are looking for a Data Engineer. If you want to build a data platform from the ground up and see how your decisions shape a product, join us.

About the project: An AI-driven product that analyses financial documents. We are building its data platform on AWS and Snowflake.

Required skill set:

• 3+ years of experience in data engineering.
• Hands-on experience with Snowflake.
• Experience with AWS data services (S3, Glue, Lambda, etc.).
• Strong SQL and Python.
• Experience implementing ETL/ELT processes.
• Experience building and maintaining data pipelines.

Will be a plus:

• dbt, Airflow or similar orchestration tools.
• Experience processing unstructured documents.
• Understanding of GDPR and personal data handling.
• Experience with AI-assisted development tools (Claude Code, Copilot, etc.).

About the team
We are a focused engineering team of 5 to 9 people at a multi-product tech SaaS company. We work across personalization, recommendation, fraud detection, and some GenAI, through real-time, near-real-time, and batch pipelines. The team is a blend of Python backend engineers, ML engineers, QA (manual and automation), a data engineer, and data scientists. Our shared mandate is to take experimentation from our data scientists and turn it into
robust, production-grade systems, either as standalone services or integrated into wider platforms.

The role
You will build the data pipelines that power our experimentation and production systems, the backbone behind fraud detection, recommendations, and personalization. You will work primarily with Kafka (sometimes Flink), Spark on Databricks, and Airflow, modelling and optimizing data for both experimentation and production serving, and partnering closely with data scientists and backend engineers on data contracts and reliability.

Responsibilities
• Build and operate batch and streaming data pipelines (structured & unstructured data)
• Work primarily with Kafka, Spark on Databricks, and Airflow
• Model and optimize data for both experimentation and production serving
• Partner with data scientists and backend engineers on data contracts and reliability
• Own pipeline quality: testing, monitoring, and performance
• Take part in the teams on-call rotation for the pipelines you build and operate

Requirements
• 3+ years of data engineering or software engineering for data

• Strong Python, including PySpark
• Kafka and stream processing
• Databricks experience
• Airflow or similar orchestration tools
• Strong SQL and data modeling, plus solid relational database skills
• Software engineering mindset
• Fluent spoken English, as you’ll work with an international team and all interviews will be conducted in English.

Nice to have
• Flink

• ML Feature stores
• MLOps and ML pipeline exposure
• Docker and Kubernetes
• SaaS or high-scale product domain experience

Why join us
• Work on high-scale, high-throughput systems
• Broad range of technologies, stacks, and problems
• Contribute to turning experimentation into production systems
• A strong engineering culture that values technical depth and ownership
• Flexibility through hybrid and remote working
• A distributed team across Dubai and Europe
 

You will get

• A position in a product development company with a dynamic environment and several concurrent projects;
• Continuous self-improvement and growth, including budget for certifications and courses;
• Competitive salary plus financial bonuses;
• Medical insurance coverage;
• Company prepaid AI agent;
• Wellbeing package: online-yoga classes, Yakaboo, BetterMe App: Health Coaching, BetterMe App: Mental Health;
• Corporate events and fun team-building activities.
• Remote-first culture
   

Interview Stages

1. HR Interview (30-45 minutes) — Initial conversation to discuss your experience, career goals, and cultural fit.
2. Technical Interview (1 hour) — In-depth technical interview covering relevant skills.
3. Final Interview (1,5 hour) — A comprehensive discussion with the team, focusing on role-specific competencies and alignment with company values.
4. Reference check & Job Offer

We are looking for Data Engineer/Golang. 

Details: 

Engagement: Full-time contract, 6 months to start, with a strong chance of extension

Fully remote

Monday to Friday, 8 hours a day, with overlap with the team’s core working hours

Start: as soon as possible

Tech stack

Go (orchestration), PostgreSQL with pgvector, embedding models served through Amazon Bedrock.

What you’ll do

Take over the current ingestion flow and document how it works: sources, transformations, weak points and failure scenarios

Automate ingestion, validation, deduplication and incremental updates, and turn onboarding of new data sources into a repeatable process

Build safe recovery mechanisms (retries, checkpoints, replay and backfill, quarantine) so that no data is lost or duplicated

Track record counts, metadata, source identifiers and data lineage; catch schema changes and run quality checks

Support the interfaces of the matching service and take part in improving embeddings, indexing and match quality

Set up monitoring and alerting together with DevOps (freshness, failures, throughput, latency, cost)

Keep tests, deployment practices and runbooks in shape so the system can be run and recovered without you

Apply the agreed rules on data consent, access, retention and deletion, and flag any open questions early

Work closely with Product, Engineering, DevOps and ML specialists, keep the team updated on progress

What we expect

Proven ownership of production data pipelines, including debugging and recovering failures

Strong SQL and PostgreSQL data modelling skills

Practical experience with Go for orchestration and migrations

Solid understanding of APIs, batch and async processing, schema evolution, metadata and lineage

Experience with idempotent processing, reconciliation and safe reruns

Hands-on work with vector search and embedding pipelines in production (search, matching or ML products)

Ability to trace quality problems across inputs, embeddings, indexes and downstream results

Cloud experience, automated testing, code review and disciplined deployments

Monitoring, alerting, runbooks, privacy-aware data modelling and access control

Hiring process

CV review, technical interview, live coding session, final interview.

N-iX is a global software engineering and technology services company, building solutions for clients since 2002. Our teams work across Europe, the Americas, and Asia for Fortune 500 companies and fast-growing businesses in finance, retail, manufacturing, telecom, healthcare, and more.

Our Data & Analytics practice designs and delivers modern data platforms on Snowflake, Databricks, Azure and AWS, from first assessment to production at scale.

We are looking for a Data Architect to lead the design of cloud data platforms for our clients. You will own the target architecture end-to-end, work directly with client stakeholders, and guide engineering teams through delivery.
The role mixes hands-on technical leadership with consulting: you will shape solutions in pre-sales, run discovery workshops, and stay accountable for architecture quality once the project is live.

Responsibilities:

• Design end-to-end data architectures on Snowflake and/or Databricks, deployed on Azure and/or AWS: ingestion, storage, processing, serving and consumption layers.
• Define lakehouse and data warehouse patterns (medallion architecture, data vault, dimensional models) and choose the right one for each use case.
• Lead migrations from legacy platforms (on-prem DWH, Hadoop, Teradata, Oracle, SQL Server) to the cloud, including assessment, roadmap and cutover plan.
• Set standards for data modelling, pipeline design, CI/CD, infrastructure as code and environment strategy.
• Design data governance, security and compliance: access control, data lineage, cataloguing (Unity Catalog, Snowflake Horizon, Purview), GDPR and local LATAM privacy laws.
• Optimize platform cost and performance (warehouse sizing, cluster policies, FinOps practices).
• Run discovery workshops and architecture reviews with client business and technical stakeholders, in English.
• Support pre-sales: solution design, effort estimates, technical sections of proposals and client presentations.
• Mentor data engineers and review their designs and code; act as the technical point of escalation on the project.
• Prepare data foundations for BI, advanced analytics, ML and GenAI use cases.

Requirements: 

• 8+ years in data engineering or data platforms, including 3+ years as a Data or Solution Architect.
• Hands-on production experience with Snowflake and/or Databricks (at least one in depth).
• Strong knowledge of Azure and/or AWS data services, for example:
  • Azure: ADLS Gen2, Data Factory, Synapse/Fabric, Event Hubs, Purview, Key Vault
  • AWS: S3, Glue, Redshift, Kinesis, Lambda, Lake Formation, IAM
• Expert SQL and solid Python or Scala; experience with Spark.
• Proven experience with data modelling (dimensional, data vault) and lakehouse design.
• Experience with orchestration and transformation tools such as Airflow, dbt, Azure Data Factory or Databricks Workflows.
• Working knowledge of infrastructure as code (Terraform, Bicep or CloudFormation) and CI/CD for data.
• Track record of leading at least one cloud data platform build or migration from design to production.
• Client-facing consulting experience: running workshops, presenting architecture decisions, managing expectations.
• At least B2 English, written and spoken.

Nice to have:

• Certifications: SnowPro Advanced Architect, Databricks Certified Data Engineer Professional, Azure Solutions Architect Expert, AWS Solutions Architect Professional or AWS Data Analytics/Data Engineer.
• Streaming architectures (Kafka, Spark Structured Streaming, Snowpipe Streaming, Delta Live Tables).
• Data mesh, data products and data contracts in practice.
• MLOps and GenAI data foundations (feature stores, vector search, RAG pipelines).
• Domain experience in banking, insurance, retail, telecom or energy.

Our customer is a leader in global prestige retail, teaching and inspiring clients to play in a world of beauty. The company has earned its reputation all over the world as a beauty trailblazer with its expertise, innovation, and entrepreneurial spirit. Our project is about developing a full-stack e-Commerce application for the world's biggest cosmetics retailer chain of stores.

PROJECT TECHNOLOGIES AND TOOLS

SQL, HIVE

Hadoop

Kafka

Spark

Spark Streaming

Cloud (Azure)

Scala or Python

RESPONSIBILITIES

Investigation and requirements clarification

Solution architecture design from scratch

PoC development

Full project delivery

Interactions with the customer development team and 3-rd party vendor

REQUIREMENTS

5+ years of experience in Java/Scala or Python

Experience with HIVE

Strong communication skills

Intermediate or higher English level, both spoken and written (B1+)

NICE TO HAVE

Databricks

Avro

Spark

Apache Hadoop

WE OFFER

Competitive compensation depending on experience and skills

Individual career path in engineering

Social package - medical insurance, sports

Compensation for sick lists and regular vacations

English classes with certified English teachers

Flexible work hours

Project Description:

Surveillance Enhancement Program (SEP) is a multi-year program (MYP) to build and enhance the surveillance platform to detect potential instances of market misconduct, market manipulation and market abuse. The Bank met its commitment to FCA (Financial Conduct Authority, FCA) to meet Market Abuse Regulation ("MAR") requirements for critical data sources in September 2017. Following that, subsequent projects were initiated to further enhance, expand and complement the current coverage of the automated surveillance platform. This project will focus on Alerts enhancements, Strategic data sourcing from FM order and trade lakes, UI / workflow enhancements to meet regulatory obligations (i.e. Market Abuse Regulation, Dodd Frank). This also includes Control Room & Suitability Monitoring related data, UI and scenario enhancements.

Responsibilities:

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development.
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed.
• Perform code reviews and recommend improvements.
• Review business requirements and recommend changes to develop reliable applications.
• Develop coding documentation and other technical specifications for assigned projects.
• Act as primary contact for development queries and concerns.
• Analyze and resolve development issues accurately.

Mandatory Skills Description:

• 15+ years of experience in data intensive Pyspark development ,Window Functions , Performance Optimization ,Spark Architecture,Salting ,Column Pruning
• Must have experience in leading a Development Team
• Proven experience as a Core Python developer
• Good understanding of Gen AI
• Experience developing Classes, OOPS, exception handling, parallel processing
• Strong knowledge of DB connectivity, data loading , transformation, calculation
• Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations
• Lambda Functions, Decorators
• Vector operations on Pandas dataframes /series
• Application of apply map, apply, map functions
• Concurrency and error handling data pipeline batch of size [1-10 gb]
• Ability to understand business requirements and translate them into technical requirements
• Ability to design architecture of data pipeline for concurrent data processing
• Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services
• Debugging application

Nice-to-Have Skills Description:

• Knowledge of web backend technology - Django, Python, PostgreSQL.
• Apache Airflow
• Atlassian Jira
• Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
• Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus