Jobs Data Engineer

LITSLINK is looking for a Senior Azure Data Engineer to join an enterprise client project. We are looking for a proactive engineer with strong experience in the Azure data ecosystem who can quickly take ownership of tasks and deliver high-quality solutions.

Responsibilities

• Migrate data processing logic from SQL Server stored procedures to Azure Databricks
• Build and maintain ETL/ELT pipelines using Azure Data Factory
• Replace and integrate new data sources in Power BI
• Optimize data models and reporting performance
• Work with large datasets and optimize SQL queries
• Collaborate with data, BI, and engineering teams
   

Requirements

• 7+ years of experience in Data Engineering
• Strong hands-on experience with the Azure data ecosystem
• Experience with:
  • Azure Data Factory
  • Azure Databricks
  • SQL
  • Power BI
• Experience building and optimizing data pipelines
• Experience migrating data workloads to cloud environments
• Python for data processing
• Proactive mindset and strong ownership of tasks
   

Engagement Details

• Full-time
• Remote
• 3-month contract (renewable)
• Start ASAP

We are looking for a Senior Data Engineer to join the Identity team at Equals5. This is not a standard ETL role. We are building a dynamic data ecosystem where AI is deeply integrated—both as a productivity multiplier and as a core component of our data processing logic for identity data enrichment and data scoring.

You will own the infrastructure that handles over 10,000 executions per minute, ensuring stability, scalability, and data integrity. You will work with a modern stack on Google Cloud Platform, utilizing Cloud Functions and Kubernetes.

We are looking for an engineer who improves infrastructure, automates everything, and is eager to implement LLM-based logic directly into high-load data flows.
 

Responsibilities

• AI-Driven Data Scoring: Design and implement pipelines that utilize LLMs to analyze and score identity data in real-time. You will integrate AI models directly into the decision-making loop, balancing accuracy with latency and cost.
• Own the Data Architecture: Architect scalable data solutions using GCP and Python. You will manage data storage and retrieval using BigQuery and Apache Iceberg to support querying of TBs of data.
• Heavy Data Processing: Utilize Apache Spark for data transformations and batch processing when lightweight cloud functions are not enough.
• Manage High-Load Orchestration: Maintain and optimize our system instances. It involves complex dataflows, custom Python nodes, and performance tuning for 10k+ execs/minute.
• Release Lifecycle (CI/CD): Take ownership of the deployment process, ensuring that updates to pipelines and infrastructure are released safely with proper testing and rollback strategies.
• Database Optimization: Manage PostgreSQL performance under heavy load, optimizing complex queries and indexing strategies.
• Active AI Usage: Use Claude Code and other AI engineering tools to accelerate your own development, refactoring, and testing processes.
• Incident Resolution: Proactively monitor the system. When alerts fire, you investigate the root cause—whether it’s a database lock or an LLM hallucination—and fix it permanently.
   

Requirements

• 4-5+ years of experience in Data Engineering or Backend Engineering with a strong data focus.
• Production AI Integration: Experience integrating LLMs (OpenAI, Anthropic, Gemini) into production applications via API. You understand latency, token limits, and how to structure data for AI scoring.
• Expertise in GCP: Understanding of Google Cloud Platform (Cloud Functions, IAM, Networking).
• Strong Python: You write clean, efficient, and testable code. You are comfortable building custom logic where standard tools fall short.
• Big Data Stack: Experience with BigQuery, Apache Spark, and modern table formats like Apache Iceberg.
• Kubernetes (K8s): Experience deploying and scaling services in containerized environments.
• Workflow Automation: Understanding of workflow orchestration tools at a deep technical level. N8N is a big part of our domain, so familiarity with it is highly valuable.
• PostgreSQL Mastery: Proven ability to handle heavy write/read loads and optimize schemas.
• English: B2+ (Upper-Intermediate) or higher.
   

Culture & Mindset

• Self-improvement: You are a fast learner. You don't fear AI replacing you; you master it to replace your manual tasks.
• Ownership: You treat the Identity domain as your own business. If a scoring model drifts or a pipeline slows down, you notice it and fix it without being asked.
• Internal Locus of Control: You take responsibility for outcomes. If an external API fails, you build a fallback mechanism instead of just blaming the provider.
• Get It Done: You prioritize shipping value. You know when to use a simple script and when to build a complex architecture.
• Openness: You share knowledge freely. If you find a better way to prompt the AI for scoring, you share it with the team.
   

What We Offer

• Fully remote with flexible hours (aligned with EU timezones for syncs).
• AI-Native Environment: We provide licenses for Claude Code and encourage using the bleeding edge of AI tech for both daily coding and product features.
• High-Impact Role: You will directly influence how we identify and score users, impacting the core business logic.
• Cross-functional visibility: Work closely with Product and Tech Leads to shape the future of Identity.
• No Bureaucracy: Fast decisions, no legacy processes, focus on results.

Description

The GlobalLogic technology team is focused on next-generation health capabilities that align with the client’s mission and vision to deliver Insight-Driven Care. This role operates within the Health Applications & Interoperability subgroup of our broader team, with a focus on patient engagement, care coordination, AI, healthcare analytics, and interoperability. These advanced technologies enhance our product portfolio with new services while improving clinical and patient

experiences.

Requirements

An AWS Data Engineer designs, develops, and maintains scalable data solutions using AWS cloud services.

Key Responsibilities:
    • Design, build, and manage ETL (Extract, Transform, Load) pipelines using AWS services (e.g., Glue, Lambda, EMR, Redshift, S3).
    • Develop and maintain data architecture (data lakes, warehouses, databases) on AWS.
    • Implement data quality and governance solutions.
    • Automate data workflows and monitor pipeline health.
    • Ensure data security and compliance with company policies.
Required Skills:
    • Proficiency with AWS cloud services, especially data-related offerings (S3, Glue, Redshift, Athena, EMR, Kinesis, Lambda).
    • Strong SQL and Python skills.
    • Experience with ETL tools and frameworks.
    • Familiarity with data modelling and warehousing concepts.
    • Knowledge of data security, access management, and best practices in AWS.
Preferred Qualifications:
    • AWS certifications (e.g., AWS Certified Data Analytics – Speciality, AWS Certified Solutions Architect).
    • Background in software engineering or data science.

• Hands-on experience with Oracle Database and log-based Change Data Capture (CDC) replication using AWS Database Migration Service (DMS) for near real-time data ingestion.

Job responsibilities

– Develops, documents, and configures systems specifications that conform to defined architecture standards, address business requirements, and processes in the cloud development & engineering.

– Involved in planning of system and development deployment, as well as responsible for meeting compliance and security standards.

– API development using AWS services in a scalable, microservices-based architecture

– Actively identifies system functionality or performance deficiencies, executes changes to existing systems, and tests functionality of the system to correct deficiencies and maintain more effective data handling, data integrity, conversion, input/output requirements, and storage.

– May document testing and maintenance of system updates, modifications, and configurations.

– May act as a liaison with key technology vendor technologists or other business functions.

– Function Specific: Strategically design technology solutions that meet the needs and goals of the company and its customers/users.

– Leverages platform process expertise to assess if existing standard platform functionality will solve a business problem or if a customisation solution would be required.

– Test the quality of a product and its ability to perform a task or solve a problem.

– Perform basic maintenance and performance optimisation procedures in each of the primary operating systems.

– Ability to document detailed technical system specifications based on business system requirements

– Ensures system implementation compliance with global & local regulatory and security standards (i.e. HIPAA, SOCII, ISO27001, etc.)

What we offer

Culture of caring. At GlobalLogic, we prioritize a culture of caring. Across every region and department, at every level, we consistently put people first. From day one, you’ll experience an inclusive culture of acceptance and belonging, where you’ll have the chance to build meaningful connections with collaborative teammates, supportive managers, and compassionate leaders. 

Learning and development. We are committed to your continuous learning and development. You’ll learn and grow daily in an environment with many opportunities to try new things, sharpen your skills, and advance your career at GlobalLogic. With our Career Navigator tool as just one example, GlobalLogic offers a rich array of programs, training curricula, and hands-on opportunities to grow personally and professionally.

Interesting & meaningful work. GlobalLogic is known for engineering impact for and with clients around the world. As part of our team, you’ll have the chance to work on projects that matter. Each is a unique opportunity to engage your curiosity and creative problem-solving skills as you help clients reimagine what’s possible and bring new solutions to market. In the process, you’ll have the privilege of working on some of the most cutting-edge and impactful solutions shaping the world today.

Balance and flexibility. We believe in the importance of balance and flexibility. With many functional career areas, roles, and work arrangements, you can explore ways of achieving the perfect balance between your work and life. Your life extends beyond the office, and we always do our best to help you integrate and balance the best of work and life, having fun along the way!

High-trust organization. We are a high-trust organization where integrity is key. By joining GlobalLogic, you’re placing your trust in a safe, reliable, and ethical global company. Integrity and trust are a cornerstone of our value proposition to our employees and clients. You will find truthfulness, candor, and integrity in everything we do.

Laba Group is one of the largest international EdTech companies in Eastern Europe. Currently, our team consists of over 800 colleagues united by a single mission: to improve people's lives through relevant and systematic knowledge.

In 10 years, we have launched over 1,400 courses and educated more than 200,000 students globally. We have also created 5 online education brands:

• Laba – an online business school
• Skvot – a school of popular culture
• robot_dreams – a platform for growth in IT
• Vocabulaba – a school of English for professionals
• elvtr – an online school in the US and the UK

We are seeking a Middle  Data Engineer.  The primary purpose or objective of this role is to be responsible for designing, building, and maintaining a scalable data infrastructure that supports analytics, reporting, and business decision-making within the organization. 

The role focuses on developing robust data pipelines, integrating multiple data sources, ensuring data quality, and delivering business-driven data solutions that align with the company's goals.

What will be the main responsibilities ?

• Design, build, and maintain scalable ETL/ELT pipelines.
• Integrate data from internal and external systems (CRMs, LMS, marketing platforms, databases, etc.).
• Develop and maintain API integrations (REST/JSON, OAuth, etc.).
• Automate recurring data processes using Python (or similar scripting languages).
• Design and maintain data models optimized for analytics and operational reporting.
• Work with cloud data warehouses (e.g., BigQuery, Snowflake) to ensure efficient storage and retrieval.
• Implement data validation and monitoring mechanisms to ensure accuracy and reliability.
• Proactively monitor and resolve data pipeline failures.
• Create dashboards and data applications using low-code/no-code platforms (e.g., Power BI, Retool, Coda, Tableau, Google Sheets).
• Translate complex datasets into clear, actionable insights for business stakeholders.
• Collaborate with analysts and business teams to define KPIs and reporting requirements.
• Suggest improvements to existing data flows and business processes.
• Maintain structured and comprehensive technical documentation (pipelines, integrations, data definitions, architecture).
• Participate in cross-functional discussions to align technical implementation with business needs

What skills and experience are considered necessary?

• Strong proficiency in Python (or similar scripting language) for data manipulation and automation.
• Advanced SQL knowledge and experience in data modeling.
• Hands-on experience with API integrations (REST, JSON, OAuth) and related tools (e.g., Postman).
• Experience working with cloud platforms (GCP, AWS) and cloud data warehouses.
• Experience with orchestration tools (e.g., Airflow, dbt, or similar frameworks).
• Familiarity with Git and CI/CD processes.
• Experience with low-code/no-code platforms for data visualization or workflow automation.
• Solid understanding of business processes and KPI-driven decision-making.
• 2–4 years of experience in a data engineering or data-focused role.

Skills that would be "nice to have":

• Experience developing and customizing internal tools using Retool, with working knowledge of JavaScript.
• Experience optimizing query performance and cost efficiency in cloud environments.
• Exposure to real-time data streaming solutions.
• Experience supporting AI/ML data pipelines.
• Strong data storytelling and stakeholder communication skills

-

Why Laba Group is the Place to Be🔥:

• Flexible full-time. We focus on results, not hours worked: manage your own time, stay connected with the team, and meet deadlines.
•  Remote, onsite, or hybrid — it’s up to you! Our modern workspace in Kyiv is fully equipped with everything you need.

As part of our team, you’ll get:

• Access to all Laba Group courses for you and your family.
• Compensation for external training, sports, and mental and physical health, helping you stay balanced and prioritize your well-being.
• Unlimited vacation after 6 months of working with us. Plan according to your needs and adjust based on workload intensity because rest is essential for your health. 
• We create a supportive environment where facing challenges and making mistakes isn’t scary. A place to find like-minded people and become part of a larger community. A service that truly puts the client first.
• No micromanagement or bureaucracy — only automated processes. Feel free to experiment, propose, and implement your ideas.
• We champion a proactive mindset, bold ideas, and a willingness to embrace change. Laba isn’t about going with the flow — it’s about pushing the world forward.
• We provide only up-to-date insights from market experts—knowledge that can be immediately applied in practice, transforming workflows even during the learning process. Our products are designed for decisive action in business and careers.
• A socially responsible environment: we regularly launch charity projects and fundraisers to support Ukraine and its defense forces.
• Vertical and horizontal growth: 83% of our Heads of Departments and Team Leaders have been promoted internally.

Our recruitment process is transparent and straightforward. 

Three stages: an interview with the Rectuiter→ Test Task→ an interview with the HM’s team 

After that, the job offer will be in your  inbox.🙌

We’re not looking for perfect candidates. We’re looking for bold and “one of us” people. 💛

Join the team. It’s going to be an amazing journey.

Join the Data Engineering team to contribute to the ongoing maintenance and improvement of an internal LLM-powered assistant that uses hosted LLM APIs and internal knowledge sources, with a focus on reliability, retrieval quality, and operational excellence.

Responsibilities

Maintain and enhance ingestion/enrichment pipelines for internal content (parsing/extraction, normalization, metadata enrichment, deduplication, and quality monitoring)

Improve indexing and retrieval performance and quality (chunking/segmentation refinements, embedding/index update workflows, metadata filtering, caching) and support hybrid retrieval capabilities (vector + keyword/BM25 + metadata)

Implement and maintain access-aware retrieval by propagating/enforcing document permissions through indexing and query-time filters, including audit logs and validation tests

Improve source attribution so responses reliably point to the correct documents and sections in a consistent format.

Extend and harden tool/workflow execution and automations (scheduled/trigger-based), including retries, timeouts, idempotency, concurrency controls, and run history

Develop and maintain evaluation and regression testing (golden sets, automated scoring) and support structured comparisons across LLM providers/models as required

Operate the platform in production: observability (logs/metrics/tracing), alerting, incident support, performance tuning, and cost controls, plus runbooks and handover documentation

Skills

Must have

8+ years of hands-on experience in Data Science and 5+ years in Machine Learning, with a proven track record, demonstrated through a robust portfolio of projects.

Strong programming skills in languages such as Python and familiarity building ETL pipelines.

Expertise in SQL and experience with both relational (preferably Postgres) and NoSQL databases (Open Search or Elastic Search)

Familiarity with cloud platforms (e.g., AWS) and their services.

Experience with version control systems (e.g., Git) and CI/CD pipelines.

Ability to build scalable infrastructure to embed and search very large number of documents.

Ability to move fast in an environment where things are sometimes loosely defined and may have competing priorities or deadlines.

Expertise in ML inference optimizations

Solid experience with Hybrid RAG, chunking/segmentation refinements, embedding/index update workflows, metadata filtering, caching, etc.

Knowledge of network optimization for distributed ML training and inference.

Understanding of distributed training patterns and checkpointing strategies.

Strong English skills (B2 and higher)

Strong verbal and written communication skills.

Ability to work independently and collaborate in a group.

Nice to have

Agile certification

AWS/Oracle/Microsoft attestations and certifications

Domain knowledge

Trading and Capital Markets

About The Company

MWDN is an international recruitment and outstaffing company with 23+ years of experience connecting exceptional tech talent with leading companies across Israel, the USA, Great Britain, and Western Europe. In this role, MWDN acts as the recruitment partner, and the selected candidate will work directly with the client’s team on their product in an international environment.

Why does MWDN rock?

Here’s what you can expect when you join MWDN:

• Security: We carefully vet our clients to minimize risks and ensure reliability and timely payments—no fraud or unpleasant surprises.
• Career support: If a project isn’t the right fit, we support you and actively help find new opportunities that match your skills and career goals.
• Legal assistance: We provide guidance on legal matters, including opening and managing your independent contractor or sole proprietorship status, taxes, and related processes.

• Professional development: We offer English courses and professional growth opportunities, as well as team-building events.

   

Why choose us? MWDN is ranked among the top 5 IT employers in our region according to DOU. We take pride in our transparency and strong commitment to our team. Curious to learn more? See what our employees say about working with us on .

What is your new project?

• Domain: AI Consulting
• Client Location: US
• Company size: up to 10 people

• Founded in: 2020

   

What makes this project exciting?

Join a forward-thinking consulting environment that helps companies scale through modern cloud, data, and AI-driven solutions. The team partners with organizations across industries to design and implement robust, multi-cloud architectures, combining deep infrastructure expertise with advanced data platforms.

Built for complex, real-world challenges, the approach focuses on turning fragmented systems into scalable, high-performing ecosystems. From enabling seamless cross-cloud operations to supporting data pipelines and analytics, you’ll help drive solutions that power smarter decision-making and long-term growth.

What makes you a great fit

• 7+ years in data engineering, analytics engineering, or cloud data platform roles - with demonstrable breadth across both pipeline and analytics layers
• Hands-on multi-cloud experience across at least two of AWS, Azure, GCP and their native data services
• Experienced with usage/oversight of generated code via AI
• Strong Python proficiency for pipeline development and automation
• Hands-on experience with dbt (Core or Cloud) or equivalent transformation frameworks
• Experience with orchestration tools (Airflow, Prefect, Dagster, or equivalent)
• Familiarity with streaming platforms (Kafka, Kinesis, Pub/Sub, or equivalent)
• Strong SQL proficiency and experience with cloud data warehouses (Snowflake, BigQuery, Redshift, Synapse)
• Understanding of data modeling patterns - star schema, data vault, and/or lakehouse

• Comfortable presenting technical content to non-technical client stakeholders

   

Preferred

• Experience with lakehouse platforms (Databricks, Apache Iceberg, Delta Lake)
• Infrastructure-as-code experience (Terraform or equivalent)
• Exposure to BI tooling and semantic layer design (Looker, Power BI, Tableau, or equivalent)
• Fintech or regulated industry background

• Prior experience in a consultancy or client-delivery environment

   

Your day-to-day in this position

• Lead implementation of end-to-end data platforms across the cloud pipeline and analytics/transformation layers
• Build and optimize data ingestion pipelines - batch and streaming - using modern orchestration tools
• Design and implement data transformation logic and semantic data models using dbt or equivalent
• Evaluate and recommend data stack components across the full delivery lifecycle, contributing to the team's stack selection mandate
• Participate in client-facing technical sessions, sprint reviews, and architecture working groups alongside the Senior Data Architect & Advisor
• Provide engineering leadership and code review for the three internal mid-level engineers
• Implement infrastructure-as-code practices to ensure consistent cloud deployments
• Build and maintain data marts, aggregated layers, and reporting-ready datasets aligned to project requirements

• Document data lineage, transformation logic, and pipeline architecture consistently and clearly

   

Why work with us?

• People-first management with minimal bureaucracy
• A friendly company culture, proven by employees who choose to return
• Flexible working hours
• Full financial and legal support for independent contractors
• Free English classes, with native speakers or Ukrainian teachers

• Dedicated HR support

   

Our next steps

✅ Intro call with a Recruiter —✅ HR interview with the client — ✅ Tech interview — ✅ Final client interview— ✅ Offer

Job Description

Amazon Web Services (AWS) – Kinesis, DMS, EMR, Glue, Lambda, Athena, Redshift, S3
Expertise in building data ingestion, ETL/ELT pipelines and orchestration on AWS
Experience with Change Data Capture (CDC) and real-time streaming data architectures
Proficiency in SQL and data modeling
Understanding of distributed systems and big data processing
Python for data transformations and scripting
Hands-on experience with orchestration frameworks like AWS Step Functions or Data Pipeline

Nice to Have:

Experience with data quality frameworks and monitoring
Knowledge of HIPAA or similar healthcare data compliance standards
Experience working in highly regulated environments

Job Responsibilities

• Design and development of scalable and reliable data pipelines in AWS using tools such as Kinesis, DMS, Glue, EMR, Athena, and Lambda.
• Define the architecture of the data ingestion and processing layer, considering performance, cost-efficiency, and maintainability.
• Collaborate closely with business stakeholders and technical teams to gather requirements, validate solutions, and align data flows with business needs.
• Propose and validate technical approaches and best practices for CDC, ETL/ELT, orchestration, data storage, and query optimization.
• Own the implementation of key data workflows, ensuring quality, security, and adherence to architectural standards.
• Conduct technical reviews, mentor junior engineers, and foster a culture of continuous improvement.
• Participate in strategic planning of the data platform and actively contribute to its evolution.
• Monitor and optimize performance of AWS-based data solutions, addressing bottlenecks and proposing enhancements.

Department/Project Description

The client is a pioneer in medical devices for less invasive surgical procedures, ranking as a leader in the market for coronary stents. The company’s medical devices are used in a variety of interventional medical specialties, including interventional cardiology, peripheral interventions, vascular surgery, electrophysiology, neurovascular intervention, oncology, endoscopy, urology, gynecology, and neuromodulation.
The client’s mission is to improve the quality of patient care and the productivity of health care delivery through the development and advocacy of less-invasive medical devices and procedures. This is accomplished through the continuing refinement of existing products and procedures and the investigation and development of new technologies that can reduce risk, trauma, cost, procedure time and the need for aftercare.

PwC is a global network of more than 370,000 professionals in 149 countries that turns challenges into opportunities. We create innovative solutions in audit, consulting, tax and technology, combining knowledge from all over the world.

PwC SDC Lviv, opened in 2018, is part of this global space. It is a place where technology is combined with team spirit, and ambitious ideas find their embodiment in real projects for Central and Eastern Europe.

What do we guarantee?

• Work format: Remote or in a comfortable office in Lviv - you choose.
• Development: Personal development plan, mentoring, English and Polish language courses.
• Stability: Official employment from day one, annual review of salary and career prospects.
• Corporate culture: Events that unite the team and a space where everyone can be themselves.

We are looking for a talented and experienced Cloud Engineer to join our growing team. As well as experience with cloud infrastructure, networking, and security, the Cloud Platform Engineer will be responsible for operating, monitoring, and maintaining our many Azure subscriptions and associated global applications. As a Cloud Platform Engineer with cloud architecture, you will work closely with other data professionals, software engineers, and business stakeholders to build, maintain and support efficient, scalable, and secure data pipelines. This role involves both engineering and operations, requiring comfort with project work and handling incident tickets as part of the team.

Responsibilities: 

• Design, build and maintain Azure based infrastructure with Terraform modules (IaaS and PaaS resources) as part of new projects or updates needed to production environments;
• Troubleshoot IaaS and PaaS issues raised via SNow incident and meet ticket closure SLA’s;
• Documentation and knowledge sharing: create and maintain environment diagrams, playbooks, runbooks, and best practice documents;
• Communicate with internal business partners including guidance on application design.

Qualifications:

• 2+ years of experience with Azure core services: Resource Manager, Virtual Machines/Scale Sets, Storage (Blob/Files), Networking (VNets, NSGs, Load Balancer, App Gateway, DNS, Private Link);
• 2+ year of experience in Windows or Linux administration, with strong problem-solving abilities;
• 2+ years of experience with Azure Kubernetes Service;
• 2+ years of experience with Terraform;
• Ability to work independently and initiate tasks without supervision and utilize strong interpersonal and organizational skills;
• Excellent problem-solving and analytical skills.

Policy statements:
https://www.pwc.com/ua/uk/about/privacy.html

About the Product

We are building a US-based, data-driven product with a strong focus on scalability, performance, and cost efficiency.

Our mission is to design a modern data platform that transforms raw behavioral and monetization data into reliable, actionable business insights — in near real-time.

For us, data engineering is not just about moving data.
It’s about:

• Designing resilient architecture
• Optimizing for performance and cost
• Building reliable automation
• Ensuring architectural integrity at scale
•  

Role Overview

We are looking for a Senior Data Engineer who will take ownership of the data platform architecture and drive technical excellence across ingestion, modeling, and performance optimization.

This role requires deep expertise in SQL, Python, AWS infrastructure, and modern data stack principles. You will not only build pipelines — you will define standards, lead architectural decisions, and proactively improve system efficiency.

You will play a critical role in ensuring that data flows seamlessly from event streams to business-ready datasets while maintaining high performance, reliability, and cost control.

What Makes This Role Senior-Level

As a Senior Data Engineer, you will:

• Own architectural decisions for the data platform
• Identify scalability bottlenecks before they become incidents
• Optimize data infrastructure for performance and cost
• Lead technical code reviews and set engineering standards
• Mentor mid-level engineers
• Act as a technical partner to Product and Analytics stakeholders
• Balance real-time and batch processing strategies strategically

Technical Requirements

Must-Have

Expert-Level SQL

• Complex analytical queries and window functions
• Query optimization and execution plan analysis
• Identifying and eliminating performance bottlenecks
• Reducing query complexity and compute costs
• Designing partitioning and clustering strategies

Python

• Advanced data manipulation
• Building scalable ETL/ELT frameworks
• Writing production-grade data services
• Automation and monitoring scripts

AWS Core Infrastructure

• AWS Kinesis Firehose (near-real-time data streaming)
• Amazon S3 (data lake architecture and storage optimization)
• Designing reliable ingestion layers

Version Control

• Git (GitHub / GitLab)
• Branching strategies
• Leading technical code reviews
• Enforcing best practices in code quality

Nice-to-Have

Modern Data Stack

• dbt (modular SQL modeling, documentation, testing)
• Experience structuring layered data models (staging → intermediate → marts)

Data Warehousing

• Google BigQuery
• Slot management
• Cost-efficient querying
• Storage and compute optimization

Advanced Optimization Techniques

• Partitioning
• Clustering
• Bucketing
• Storage layout optimization

Integrations & Infrastructure

• Salesforce data integration
• Docker / ECS
• CI/CD for data workflows

AI / ML Exposure

• Supporting feature pipelines
• Understanding data requirements for ML systems

Key Responsibilities

Data Platform Architecture

• Design and maintain a scalable real-time and batch data platform
• Architect ingestion pipelines using AWS Kinesis and Python
• Ensure high availability and reliability of data flows

Real-Time Processing

• Enable near-real-time (seconds–minutes latency) data processing
• Build systems for operational alerting and anomaly detection
• Ensure early detection of monetization and traffic issues

Data Modeling

• Transform raw event data into business-ready datasets using dbt
• Design scalable, maintainable schemas aligned with product evolution

Performance & Cost Engineering

• Optimize SQL queries and storage structures
• Design cost-efficient partitioning strategies
• Monitor and reduce warehouse and infrastructure costs
• Balance real-time and batch processing appropriately

Engineering Excellence

• Lead and participate in code reviews
• Enforce high standards of performance, security, and maintainability
• Improve observability and monitoring across pipelines

Cross-Functional Collaboration

• Work closely with Data Analysts and Product Managers
• Translate business requirements into scalable technical solutions
• Clearly communicate trade-offs between speed, cost, and complexity

Type of Data We Process

• User behavior events (page views, clicks, searches, conversions)
• Ad & monetization events (impressions, clicks, CTR, attribution)
• System and integration logs (latency, errors, rate limits)

Why Real-Time Is Critical

• Detect broken ads or impression drops before revenue is lost
• Identify traffic anomalies or abuse early
• Enable same-day operational intervention
• Prevent negative user and advertiser experience

Near-real-time (seconds to minutes latency) is required for operational awareness.
Batch processing remains important for historical analysis and reporting — but not for incident detection.

Working Schedule

• Monday – Friday
• 16:00 – 00:00 Kyiv time
• Full alignment with US-based stakeholders

What We Value

• Strong ownership mindset
• Strategic thinking about architecture
• Focus on scalability, reliability, and cost efficiency
• Proactive problem-solving
• Clear communication with both technical and non-technical teams
• Ability to think beyond “just making it work”

PwC is a global network of more than 370,000 professionals in 149 countries that turns challenges into opportunities. We create innovative solutions in audit, consulting, tax and technology, combining knowledge from all over the world.

PwC SDC Lviv, opened in 2018, is part of this global space. It is a place where technology is combined with team spirit, and ambitious ideas find their embodiment in real projects for Central and Eastern Europe.

What do we guarantee?

• Work format: Remote or in a comfortable office in Lviv - you choose.
• Development: Personal development plan, mentoring, English and Polish language courses.
• Stability: Official employment from day one, annual review of salary and career prospects.
• Corporate culture: Events that unite the team and a space where everyone can be themselves.

PwC is seeking an experienced Databricks Platform Engineer to accelerate the maturation of our Databricks platform as a core component of the firm’s Data Mesh and AI-first strategy. This role will help shape the architectural direction, governance model, and enterprise standards that underpin secure and scalable Databricks adoption.

Role Overview

The Databricks Platform Engineer  will be responsible for executing against this strategy by driving service optimization, implementing advanced platform capabilities, and partnering with Product and Engineering teams to enable enterprise-scale adoption across client and corporate use cases.

Key Responsibilities:

• Lead the rollout and optimization of Databricks services at the PwC account level.
• Implement advanced capabilities including Unity Catalog, AI/ML models, Delta Sharing, Automatic Identity Management (AIM/SCIM), and Budget Policies for cost governance and tagging.
• Collaborate with Databricks and PwC platform teams to enhance and enable secure data exchange.
• Support networking design and optimization across Databricks environments.
• Partner with Product and Engineering teams to refine user experience, service provisioning models, and upcoming Databricks capabilities.
• Coordinate cross-functional efforts to enable AI-driven workflows and advanced analytics capabilities within Databricks.

Required Qualifications:

• 5+ years of experience in data engineering, cloud data platforms, or platform architecture.
• 3+ years of hands-on experience with Databricks or comparable modern data platforms.
• Expertise in Unity Catalog, Delta Sharing, Compute (classic and serverless), Microsoft Entra ID (SCIM/AIM), and Azure networking.
• Strong understanding of data governance, access control models, and FinOps best practices.
• Excellent communication skills with the ability to translate complex technical solutions into business value.

Preferred Qualifications:

• Databricks Certified Professional or Azure Databricks Certification.
• Hands-on experience with Databricks, Microsoft Fabric / OneLake, or other enterprise data platforms.
• Prior consulting experience or enterprise-scale platform transformation experience.

Policy statements:
https://www.pwc.com/ua/uk/about/privacy.html

PwC is a global network of more than 370,000 professionals in 149 countries that turns challenges into opportunities. We create innovative solutions in audit, consulting, tax and technology, combining knowledge from all over the world.

Join PwC’s pioneering data & AI team and help build the data foundations behind impactful AI solutions. We’re growing quickly due to a strong pipeline of client work, and we’re hiring across multiple seniority levels — from junior to senior Data Engineers and Data Scientists.

Across our projects, Python is the core skill we expect. Depending on your strengths, you may focus more on data engineering (pipelines, platforms, SQL) or data architecture (designing scalable data solutions). Many roles also include “full-solver” flexibility — contributing where needed, including automation, integration work, or enabling AI/GenAI use cases on modern platforms (including Microsoft technologies where relevant).

Key responsibilities:

• Data Pipeline Design & Development: Design, implement, and maintain scalable data pipelines and ETL/ELT processes, primarily using Python and Spark (PySpark), to ingest, transform, and deliver data from various sources into analytics and ML platforms.
• Data Modelling & Warehousing: Design and optimize data models (e.g. star/snowflake schemas), build and manage data warehouses and data lakes, and ensure data structures support reporting, analytics, and ML use cases.​
• Data Preparation for ML: Collaborate closely with data scientists and ML engineers to understand data requirements, implement robust preprocessing and feature engineering steps, and ensure datasets are clean, consistent, and suitable for machine learning models.
• Performance & Reliability: Optimize data processing jobs and SQL queries for performance and cost efficiency, monitor data pipelines in production, and ensure reliability, scalability, and adherence to SLAs.
• Client Collaboration: Work directly with clients across Western Europe and the USA to understand their data landscape and requirements, translate business needs into technical designs, and provide expert guidance on data architecture and best practices.
• Governance, Quality & Security: Implement data quality checks, validation frameworks, and governance standards; ensure data security, privacy, and compliance in line with PwC and client requirements.
• Learning & Development: Stay at the forefront of data engineering, big data, and cloud technologies, continuously improving existing solutions, tools, and processes.
• Mentorship: Support the growth of junior team members by sharing knowledge, reviewing code, and guiding them in data engineering best practices and project work.

Who We're Looking For:

• Programming Skills (Key Requirement)
  Strong programming skills in Python (e.g. pandas, PySpark, SQLAlchemy, airflow-like tools). Ability to write clean, maintainable, and testable code is essential. Experience with other programming languages is a plus.
• Demonstrated hands-on experience with building data pipelines, ideally using Spark (PySpark) or similar distributed processing frameworks.
• Designing and implementing ETL/ELT workflows.
• Working with large datasets and complex data transformations.
• Database Expertise: Proficiency in SQL databases (designing schemas, writing complex queries, optimization). 
• Cloud & Big Data (Nice to Have / Advantage): Experience with cloud data platforms (preferably Azure: Synapse, Databricks, Data Factory, Azure SQL, Data Lake) or similar services on AWS/GCP.
• Professional Background: At least 2 years of relevant professional experience in data engineering, BI engineering, or similar data-focused roles.
• Analytical Thinking: Strong analytical mindset with the ability to understand complex data landscapes, debug data issues, and design logical, efficient data flows.
• Language Skills: English at B2 level or higher.

Policy statements:
https://www.pwc.com/ua/uk/about/privacy.html

Join the Data Engineering team to contribute to the ongoing maintenance and improvement of an internal LLM-powered assistant that uses hosted LLM APIs and internal knowledge sources, with a focus on reliability, retrieval quality, and operational excellence.

• Responsibilities:

  • Maintain and enhance ingestion/enrichment pipelines for internal content (parsing/extraction, normalization, metadata enrichment, deduplication, and quality monitoring)
  • Improve indexing and retrieval performance and quality (chunking/segmentation refinements, embedding/index update workflows, metadata filtering, caching) and support hybrid retrieval capabilities (vector + keyword/BM25 + metadata)
  • Implement and maintain access-aware retrieval by propagating/enforcing document permissions through indexing and query-time filters, including audit logs and validation tests
  • Improve source attribution so responses reliably point to the correct documents and sections in a consistent format.
  • Extend and harden tool/workflow execution and automations (scheduled/trigger-based), including retries, timeouts, idempotency, concurrency controls, and run history
  • Develop and maintain evaluation and regression testing (golden sets, automated scoring) and support structured comparisons across LLM providers/models as required
  • Operate the platform in production: observability (logs/metrics/tracing), alerting, incident support, performance tuning, and cost controls, plus runbooks and handover documentation

• Mandatory Skills Description:

  8+ years of hands-on experience in Data Science and 5+ years in Machine Learning, with a proven track record, demonstrated through a robust portfolio of projects.
  Strong programming skills in languages such as Python and familiarity building ETL pipelines.
  Expertise in SQL and experience with both relational (preferably Postgres) and NoSQL databases (Open Search or Elastic Search)
  Familiarity with cloud platforms (e.g., AWS) and their services.
  Experience with version control systems (e.g., Git) and CI/CD pipelines.
  Ability to build scalable infrastructure to embed and search very large number of documents.
  Ability to move fast in an environment where things are sometimes loosely defined and may have competing priorities or deadlines.
  Expertise in ML inference optimizations
  Solid experience with Hybrid RAG, chunking/segmentation refinements, embedding/index update workflows, metadata filtering, caching, etc.
  Knowledge of network optimization for distributed ML training and inference.
  Understanding of distributed training patterns and checkpointing strategies.
  Strong English skills (B2 and higher)
  Strong verbal and written communication skills.
  Ability to work independently and collaborate in a group.

• Nice-to-Have Skills Description:

  Agile certification
  AWS/Oracle/Microsoft attestations and certifications
  Domain knowledge - Trading and Capital Markets

Job Description

• You have 4+ years of experience on similar position.
• You have significant experience with Python. Familiarity with Java or Scala is a plus.
• Hands-on experience building scalable solutions in AWS.
• Proficiency in NoSQL and SQL databases and in high-throughput data-related architecture and technologies (e.g. Kafka, Spark, Hadoop, MongoDB, AWS Batch, AWS Glue, Athena, Airflow, dbt).
• Excellent SQL and data transformation skills.
• Excellent written and verbal communication skills with an ability to simplify complex technical information.
• Experience guiding and mentoring junior team members in a collaborative environment.

Job Responsibilities

• Work in a self-organised agile team with a high level of autonomy, and you will actively shape your team's culture.
• Design, build, and standardise privacy-first big data architectures, large-scale data pipelines, and advanced analytics solutions in AWS.
• Develop complex integrations with third-party partners, transferring terabytes of data.
• Align with other Data experts on data (analytics) engineering best practices and standards, and introduce those standards and data engineering expertise to the team in order to enhance existing data pipelines and build new ones.
• Successfully partner up with the Product team to constantly develop further and improve our platform features.

Department/Project Description

We are a global audience and location intelligence company that helps marketers connect the digital and physical world. We provide data-driven solutions to enhance marketing campaigns by leveraging location and audience data to reveal consumer behavior and enable more precise targeting and measurement. We work on high-end / high-performance / high-throughput systems for in-time analysis of data for autonomous driving and other big data applications e.g. for E-commerce.

Skill Category

• Data Engineering

• Project Description:

  Join the Data Engineering team to contribute to the ongoing maintenance and improvement of an internal LLM-powered assistant that uses hosted LLM APIs and internal knowledge sources, with a focus on reliability, retrieval quality, and operational excellence.

   

• Responsibilities:

  • Maintain and enhance ingestion/enrichment pipelines for internal content (parsing/extraction, normalization, metadata enrichment, deduplication, and quality monitoring)
  • Improve indexing and retrieval performance and quality (chunking/segmentation refinements, embedding/index update workflows, metadata filtering, caching) and support hybrid retrieval capabilities (vector + keyword/BM25 + metadata)
  • Implement and maintain access-aware retrieval by propagating/enforcing document permissions through indexing and query-time filters, including audit logs and validation tests
  • Improve source attribution so responses reliably point to the correct documents and sections in a consistent format.
  • Extend and harden tool/workflow execution and automations (scheduled/trigger-based), including retries, timeouts, idempotency, concurrency controls, and run history
  • Develop and maintain evaluation and regression testing (golden sets, automated scoring) and support structured comparisons across LLM providers/models as required
  • Operate the platform in production: observability (logs/metrics/tracing), alerting, incident support, performance tuning, and cost controls, plus runbooks and handover documentation

   

• Mandatory Skills Description:

  8+ years of hands-on experience in Data Science and 5+ years in Machine Learning, with a proven track record, demonstrated through a robust portfolio of projects.
  Strong programming skills in languages such as Python and familiarity building ETL pipelines.
  Expertise in SQL and experience with both relational (preferably Postgres) and NoSQL databases (Open Search or Elastic Search)
  Familiarity with cloud platforms (e.g., AWS) and their services.
  Experience with version control systems (e.g., Git) and CI/CD pipelines.
  Ability to build scalable infrastructure to embed and search very large number of documents.
  Ability to move fast in an environment where things are sometimes loosely defined and may have competing priorities or deadlines.
  Expertise in ML inference optimizations
  Solid experience with Hybrid RAG, chunking/segmentation refinements, embedding/index update workflows, metadata filtering, caching, etc.
  Knowledge of network optimization for distributed ML training and inference.
  Understanding of distributed training patterns and checkpointing strategies.
  Strong English skills (B2 and higher)
  Strong verbal and written communication skills.
  Ability to work independently and collaborate in a group.

Project Description:

Join the Data Engineering team to contribute to the ongoing maintenance and improvement of an internal LLM-powered assistant that uses hosted LLM APIs and internal knowledge sources, with a focus on reliability, retrieval quality, and operational excellence.

Responsibilities:

• Maintain and enhance ingestion/enrichment pipelines for internal content (parsing/extraction, normalization, metadata enrichment, deduplication, and quality monitoring)
• Improve indexing and retrieval performance and quality (chunking/segmentation refinements, embedding/index update workflows, metadata filtering, caching) and support hybrid retrieval capabilities (vector + keyword/BM25 + metadata)
• Implement and maintain access-aware retrieval by propagating/enforcing document permissions through indexing and query-time filters, including audit logs and validation tests
• Improve source attribution so responses reliably point to the correct documents and sections in a consistent format.
• Extend and harden tool/workflow execution and automations (scheduled/trigger-based), including retries, timeouts, idempotency, concurrency controls, and run history
• Develop and maintain evaluation and regression testing (golden sets, automated scoring) and support structured comparisons across LLM providers/models as required
• Operate the platform in production: observability (logs/metrics/tracing), alerting, incident support, performance tuning, and cost controls, plus runbooks and handover documentation

Mandatory Skills Description:

8+ years of hands-on experience in Data Science and 5+ years in Machine Learning, with a proven track record, demonstrated through a robust portfolio of projects.
Strong programming skills in languages such as Python and familiarity building ETL pipelines.
Expertise in SQL and experience with both relational (preferably Postgres) and NoSQL databases (Open Search or Elastic Search)
Familiarity with cloud platforms (e.g., AWS) and their services.
Experience with version control systems (e.g., Git) and CI/CD pipelines.
Ability to build scalable infrastructure to embed and search very large number of documents.
Ability to move fast in an environment where things are sometimes loosely defined and may have competing priorities or deadlines.
Expertise in ML inference optimizations
Solid experience with Hybrid RAG, chunking/segmentation refinements, embedding/index update workflows, metadata filtering, caching, etc.
Knowledge of network optimization for distributed ML training and inference.
Understanding of distributed training patterns and checkpointing strategies.
Strong English skills (B2 and higher)
Strong verbal and written communication skills.
Ability to work independently and collaborate in a group.

Nice-to-Have Skills Description:

Agile certification
AWS/Oracle/Microsoft attestations and certifications
Domain knowledge - Trading and Capital Markets