Jobs Data Engineer

About Traffic Label
Traffic Label is a performance marketing and technology company with nearly two decades of experience driving engagement and conversion across the iGaming and digital entertainment sectors.
We’re now building a Customer Data Platform (CDP) on Snowflake and AWS - unifying player data across multiple brands to power automation, insights, and personalization.

The Role
We’re looking for a Data Engineering Lead to own the technical delivery and development of this platform. You’ll architect scalable pipelines, lead a small team, and ensure data reliability, accuracy, and performance.
Team size: 3–4 engineers/analysts

Key Responsibilities

• Design and implement scalable data pipelines processing millions of events daily
• Own Snowflake data warehouse architecture, optimization, and cost control
• Lead the engineering team through delivery and performance improvements
• Ensure >95% data accuracy and 99.9% pipeline uptime
• Collaborate with marketing, analytics, and compliance teams to align data with business goals

Requirements

• 5+ years in data engineering, 2+ in leadership roles
• Expert in SQL, and Python
• Proficient with AWS (S3, Lambda, IAM) and orchestration tools (Airflow, dbt, etc.)
• Strong understanding of data governance, cost optimization, and performance tuning
• Experience with iGaming data, Kafka/Kinesis, or MLflow is a plus

Why Join Us

• Build a core data platform from the ground up
• Competitive salary and performance bonuses
• Flexible remote or hybrid work across Europe
• Supportive, innovative, data-driven culture

Ready to lead a data platform that powers smarter decisions across global iGaming brands?
Apply now to join Traffic Label’s Data & Technology team.

We are developing a new tool to evaluate the effectiveness of retailer (e.g. large groceries chains) promotions. This solution will enhance promotion effectiveness forecasting, support choosing right way of introducing it (time, volume, type of discounts etc.) based on engeneering of historical data and introducing forecasting models.

The tool incorporates modern solution stack requirements, which will improve user experience, back-end robustness, and data processing efficiency. We anticipate these enhancements will meet stakeholder needs and contribute to business goals.

Job Description

• Expert in Tableau or Power BI with ability to build complex, interactive visualizations from scratch quickly and 6+ years of experience in the field
• Proven ability to compare and contrast BI tools (specifically the trade-offs between Tableau’s visual flexibility and Looker’s governed modeling).
• Expert-level SQL for data manipulation and performance tuning within cloud warehouses (Snowflake, BigQuery, or Databricks).
• Cross-Platform Literacy: Proven ability to compare and contrast BI tools (specifically the trade-offs between Tableau’s visual flexibility and Looker’s governed modeling).
• English proficiency 
• Good to have: Experience with open table format LakeHouse architectures (Delta Lake, Apache Iceberg, DuckLake)

As a Senior BI specialist, your main responsibilities will be:

• Maintaining the Business Intelligence repository and catalog. Working closely with data architects, data analysts, and a variety of end-users to ensure technical compatibility and user satisfaction.
• Data Ingestion & Cleaning: Analyze and process raw log files (CSV/Excel), handling data inconsistencies and structuring it for analysis.
• Dashboard Creation: Design and build interactive dashboards in Tableau or PowerBI to visualize key metrics.
• Work with the engineering team to define the long-term data requirements for the future platform.
• Conduct a comprehensive audit of current BI workflows to recommend a "future-state" platform (e.g., Looker, Sigma, or ThoughtSpot) based on scalability, cost, and user adoption.

We are looking for a Senior Data Engineer. This is not a standard ETL role. You will work on a high-load system processing 10,000+ executions per minute, building scalable pipelines where AI and LLM logic are embedded directly into production data flows.

The role combines data engineering, backend thinking, infrastructure ownership, and real production AI integration.

What you will do:

• Design and maintain scalable data pipelines in Python on Google Cloud Platform
• Build and optimize AI-driven scoring flows using LLMs for real-time identity enrichment
• Work with BigQuery, Apache Spark, and Apache Iceberg for heavy data processing
• Maintain high-load orchestration with custom Python nodes and complex workflows
• Own deployment and release processes (CI/CD)
• Optimize PostgreSQL performance under heavy load
• Investigate incidents and resolve root causes permanently

• Use AI engineering tools such as Claude in daily development

   

Requirements:

• 4+ years of experience in Data Engineering / Backend Engineering
• Strong Python skills
• Commercial experience with production LLM integration (OpenAI / Anthropic / Gemini)
• Strong knowledge of GCP (Cloud Functions, IAM, Networking)
• Experience with BigQuery, Spark, Iceberg
• Good understanding of Kubernetes
• Experience with workflow orchestration tools (n8n is a strong plus)
• Strong practical knowledge of PostgreSQL

• English level B2+

   

What is important for us:

• Ownership mindset
• Fast learning ability
• Strong problem-solving approach
• Active AI usage in engineering work
• Focus on practical results over unnecessary complexity

Our client is a global jewelry manufacturer that is transforming its data and analytics footprint and is also on a journey from IaaS-based Microsoft/QV solutions to Azure PaaS.

As a Data Engineer, you will collaborate with product owners, architects, and other key profiles across the global organisation. You will help business verticals and clusters maximize the value of the data and drive business change as a result.

Your primary focus will be building and implementing scalable data products on the global data platform together with other smart engineers in the organization.

Responsibilities

• Work closely with your colleagues within a product team to design scalable, efficient, and stable products
• Work closely with product owners to build and deliver features designed to drive commercial business cases and efficiencies within reporting and analytics, and other data-driven use cases
• Create and maintain a high-quality data pipeline architecture
• Obtain relevant business and system knowledge to convert requirements efficiently to technical design documentation and conceptual data modelling
• Ingest large, complex data sets, automate manual processes, optimize data delivery, improve data quality, etc.
• Own your data products and track continuous stability and benefits
• Live and breathe "build -once-consume-many" culture in which we enable others to use our data products for maximum value-add
• Work with stakeholders and teams to assist with data-related technical challenges
• Follow best practice and existing guidelines, but also push the limits
• Thrive in a fast-paced environment, and you get motivated by challenges as a true problem solver.

Skills Required

• Relevant education within Computer Science, Mathematics, Economics, Information Management, or Statistics
• 3+ years of experience as a Data Engineer or in a Software engineering role with a focus on data
• Proven experience with ADF, Data Lake, Synapse, T/U-SQL, Databricks, MDS, DAX, SSIS, and SQL Server
• Experience with at least one object-oriented or functional/scripting language - Java, Scala, Python, etc.
• DevOps and CI/CD focus
• Hands-on with git and coding best practices
• Unit testing
• Experience in working with relational data sets using SQL
• Experience with data warehousing, dimensional modelling, and cubes
• Experience in building and optimizing data pipelines, architectures, and datasets
• Experience in working with Azure, AWS, or other cloud providers
• Designed, built, and delivered production-ready data products at an enterprise level
• Highly proficient in English and able to communicate with all levels
• Agile mindset and experience of Scrum

It's a plus if you have worked with:

• Big data: Spark, Hadoop
• Data pipeline and workflow tools: Airflow
• Infrastructure: Kubernetes and Helm; we're using Airflow running on Kubernetes
• Stream-processing systems: Kafka or Spark Streaming

Paycord is a PayTech company with a high-load platform for payment processing. We combine fintech expertise with merchant insights to create innovative solutions. We’ve successfully developed a strong product that helps businesses succeed in new markets.

Our primary focus is on solution-driven development, and we prioritize the needs of our business clients. We provide access to a wide range of local and international payment methods, supporting businesses in reaching new heights and achieving excellence.

We’re rapidly growing and inviting a Data Engineer to our team.

You would be running tasks such as:

• Develop and maintain ETL pipelines using Airflow, Python, SQL, and AWS services such as S3;
• Write efficient Python code for data processing and transformation (ETL);
• Working with Data Lake and Snowflake based DWH;
• Manage database systems and optimize SQL queries for performance;
• Collaborate with different departments and teams to understand data requirements and deliver scalable solutions;
• Implement best practices for data management, security, and compliance;
• Monitor and troubleshoot data pipelines to ensure reliability and performance;
• Expected to take the initiative by suggesting optimizations and improvements to the BI system.
   

Required skills and expertise:

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Mathematics, Statistics, Physics, or related field;
• Experience with AWS services (including S3, Athena will be a plus);
• Advanced programming skills in Python;
• Proved skills in SQL and database management;
• Experience with DWH such as Snowflake or similar (BigQuery, RedShift);
• Experience building and optimizing ETL pipelines in a production environment
• Experience with Airflow.
  
  Will be a plus:
• Experience with Tableau;
• Experience with JS, React ;
• Experience with FinTech or related.
   

We offer:

Care for your health and well-being

• 100 % paid sick leaves;
• 20 working days of paid vacation;
• Medical support;
• Benefits Cafeteria (budget for gym/stomatology/psychological service & etc.);
• Corporate gifts & events.

Professional growth & development

• Competitive salary with annual salary promotions;
• The annual budget for professional courses, conferences, workshops, and books;
• Internal training courses;
• Work with a team of professionals and have the opportunity to share knowledge.

Corporate Culture

• Dynamic and result-oriented work environment;
• The ability to influence product development at an early stage;
• Openness to new ideas and approaches, healthy team discussions;
• No “red tape” culture.

Candidate journey:
HR Interview — Interview with Team Lead — Reference check — Job Offer

PwC is a global network of more than 370,000 professionals in 149 countries that turns challenges into opportunities. We create innovative solutions in audit, consulting, tax and technology, combining knowledge from all over the world.

PwC SDC Lviv, opened in 2018, is part of this global space. It is a place where technology is combined with team spirit, and ambitious ideas find their embodiment in real projects for Central and Eastern Europe.

What do we guarantee?

• Work format: Remote or in a comfortable office in Lviv - you choose.
• Development: Personal development plan, mentoring, English and Polish language courses.
• Stability: Official employment from day one, annual review of salary and career prospects.
• Corporate culture: Events that unite the team and a space where everyone can be themselves.

We are looking for a talented and experienced Cloud Engineer to join our growing team. As well as experience with cloud infrastructure, networking, and security, the Cloud Platform Engineer will be responsible for operating, monitoring, and maintaining our many Azure subscriptions and associated global applications. This is a cloud infrastructure role.

As a Cloud Platform Engineer with cloud architecture, you will work closely with other data professionals, software engineers, and business stakeholders to build, maintain and support efficient, scalable, and secure data pipelines. This role involves both engineering and operations, requiring comfort with project work and handling incident tickets as part of the team.

Responsibilities:

• Design, build and maintain Azure based infrastructure with Terraform modules (IaaS and PaaS resources) as part of new projects or updates needed to production environments;
• Troubleshoot IaaS and PaaS issues raised via SNow incident and meet ticket closure SLA’s;
• Documentation and knowledge sharing: create and maintain environment diagrams, playbooks, runbooks, and best practice documents;
• Communicate with internal business partners including guidance on application design.

Qualifications:

• 3+ years of experience with Azure core services: Resource Manager, Virtual Machines/Scale Sets, Storage (Blob/Files), Networking (VNets, NSGs, Load Balancer, App Gateway, DNS, Private Link);
• 3+ year of experience in Windows or Linux administration, with strong problem-solving abilities;
• 3+ years of experience with Azure Kubernetes Service;
• 2+ years of experience with Terraform;
• Ability to work independently and initiate tasks without supervision and utilize strong interpersonal and organizational skills;
• Excellent problem-solving and analytical skills.

NICE TO HAVE:

• Experience with BICEP and/or ARM templates;
• CI/CD and automation: Azure DevOps Pipelines or GitHub Actions (YAML);
• Familiarity with configuration management tools, preferably Ansible;
• Familiarity with configuring and managing Kafka/MongoDB/Elasticsearch;
• Competency in scripting languages such as Bash, Shell, or PowerShell.

Policy statements:
https://www.pwc.com/ua/uk/about/privacy.html

CrunchCode — міжнародна сервісна ІТ-компанія з досвідом близько 7 років у розробці вебсервісів і вебзастосунків. Ми працюємо у форматах staff augmentation (outstaff) та outsourcing і підключаємо спеціалістів до проєктів клієнтів у довгостроковій моделі співпраці.

Ми працюємо переважно з проєктами в доменах логістики (включно з last mile),e-commerce, fintech та банкінгу, а також enterprise-рішеннями.
Для нас важливо, щоб проєкт був “чистим” і зрозумілим з точки зору етики та цінності для користувачів.

Ми принципово не беремо проєкти, пов’язані з:
● gambling / гемблінгом,
● adult-контентом та порнографією,
● шахрайством або будь-якою розробкою, що спрямована на обман чи маніпуляції.

What We Offer:
● Fully remote work
● Long-term, stable project
● High level of autonomy and trust
● Minimal bureaucracy
● Direct impact on business-critical logistics systems
● Long-term engagement, not a short-term contract.

Tech Stack
Python, SQL, GCP, BigQuery, ETL/ELT, Data Modeling, DBT, Airflow / Cloud Composer
Duration: 9+ months

What We Offer:
● Fully remote work
● Long-term, stable project
● High level of autonomy and trust
● Minimal bureaucracy
● Direct impact on business-critical logistics systems
● Long-term engagement, not a short-term contract.

Requirements (Must-have):
- Strong hands-on Python experience for data engineering
- Strong experience with GCP and cloud-based data workflows
- Experience building and maintaining production data pipelines
- Strong SQL and data modeling skills
- Ability to work with large-scale or unstructured data
- Experience collaborating with data scientists and cross-functional engineering teams
- Ability to deliver practical, reliable solutions in a fast-evolving environment

Responsibilities:
- Build new data pipelines and adapt existing ETL/ELT workflows
- Move and organize data in cloud storage and analytical systems (BigQuery)
- Work closely with data scientists on tables, transformations, and methodology implementation
- Connect services and data sources required for the solution
- Optimize queries and support scalable data processing
- Work with large, unstructured datasets including voice-related data
- Help operationalize evolving architecture decisions into production-ready data flows
- Collaborate with engineering and data teams to ensure reliable execution

Nice to Have:
- Experience with DBT
- Experience with Airflow or Cloud Composer
- Experience with large, unstructured, or voice-related datasets
- Lead-level coordination experience
- Experience supporting production data workflows in rapidly evolving projects

Hiring Process:
- Intro call
- Technical discussion
- Offer
Start: ASAP

We are looking for a Senior Data Engineer who can design, implement, and evolve scalable data systems in AWS.

Core Responsibilities
 

• Design and implement batch and streaming data pipelines using Apache Spark.
• Build and evolve a scalable AWS-based data lake architecture.
• Develop and maintain real-time data processing systems (event-driven pipelines).
• Own performance tuning and cost optimization of Spark workloads.
• Define best practices for data modeling, partitioning, and schema evolution.
• Implement monitoring, observability, and data quality controls.
• Contribute to infrastructure automation and CI/CD for data workflows.
• Participate in architectural decisions and mentor other engineers.

Required Qualifications
 

• 5+ years of experience in Data Engineering.
• Strong hands-on experience with Apache Spark (including Structured Streaming).
• Experience building both batch and streaming pipelines in production environments.
• Proven experience designing AWS-based data lake architectures: S3, EMR, Glue, Athena.
• Experience with event streaming platforms such as Apache Kafka or Amazon Kinesis.
• Experience implementing lakehouse formats such as Delta Lake.
• Strong understanding of partitioning strategies and schema evolution.
• Experience using SparkUI and AWS CloudWatch for profiling and optimization.
• Strong understanding of Spark performance tuning (shuffle, skew, memory, partitioning).
• Proven track record of cost optimization in AWS environments.
• Experience with Docker and CI/CD pipelines.
• Experience with Infrastructure as Code: Terraform, AWS CDK.
• Familiarity with monitoring and observability practices.
• Experience in the Financial domain.
• Experience running Spark workloads on Kubernetes.
• Experience implementing data quality frameworks or metadata/lineage systems.

Company: German E-Commerce BI Solutions Provider
Engagement: Part-time to start, with clear expansion potential

About Us

We are a German-based provider of Business Intelligence solutions for the e-commerce sector. Our mission is to enable data-driven decision-making for online businesses through scalable data architectures, robust analytics pipelines, and actionable insights.

To strengthen our data team, we are looking for a hands-on Data Engineer who enjoys building reliable data pipelines and working at the intersection of engineering and analytics.
 

Your Role

As a Data Engineer, you will design, build, and maintain scalable data pipelines and data models that power our BI and analytics solutions for e-commerce clients. You will work closely with analytics, product, and business stakeholders to ensure high data quality and availability.

This role starts as a part-time engagement with strong potential to grow into a larger scope as the collaboration evolves.
 

Responsibilities

• Design, develop, and maintain robust data pipelines using Python
• Transform and process data using Pandas DataFrames
• Write and optimize complex SQL queries (SQL proficiency is a core requirement)
• Build and maintain data workflows using orchestration tools (e.g., Dagster or similar)
• Work with cloud data warehouses (e.g., Amazon Redshift or similar technologies)
• Ensure data quality, reliability, and performance across pipelines
• Collaborate with BI and analytics teams to translate business requirements into technical solutions
• Continuously improve data architecture and engineering best practices
   

Must-Have Qualifications

• Minimum 4 years of professional experience with Python, including extensive hands-on experience with Pandas
• Strong, production-level SQL skills (natural prerequisite for the role)
• Solid understanding of data modeling and ETL/ELT processes
• Experience building and maintaining data pipelines in a professional environment
• Ability to work independently in a part-time setup and take ownership of deliverables
   

Nice-to-Have

• Experience with Amazon Redshift
• Experience with Dagster
• Experience with other SQL dialects beyond Redshift
• Experience with orchestration tools such as Airflow or similar
• Background in e-commerce, BI, or analytics-driven environments
• Familiarity with cloud-based data architectures
   

What We Offer

• Flexible part-time engagement with scalability toward a larger role
• Remote setup
• High impact in a growing e-commerce BI environment
• Opportunity to shape and improve modern data architecture
• Collaborative, pragmatic, and engineering-focused culture

Numerical Technologies LTD is a leading provider of custom software solutions and services specializing in risk, limits, and regulatory capital. Right now, we are looking for a Python/Data Engineer to join our team.
 

What you will be doing:

As part of the team, you will be participating in a project with a large US bank. You will be helping us develop data pipelines and support and maintain the ETL framework. You will be writing scalable and fault-tolerant code, debugging data flows, and integrating applications with third-party web services. You will be working closely with the client and product manager to understand the system’s functional and non-functional requirements, and contribute to code quality through unit testing, integration testing, code review, and system design using Python to build robust solutions that align with the client’s business needs.

What are we looking for:

• A strong programming background with Python and OOP skills.
• Hands-on experience building and maintaining scalable data pipelines (ETL/ELT).
• Proficiency with SQL databases.
• Experience with Python data processing libraries such as Pandas, Polars.
• Experience with workflow orchestration tools (e.g., Airflow).
• Experience working with cloud platforms (preferably GCP).
• Hands-on experience with Docker, Kubernetes, and Linux.
• Strong problem-solving skills and ability to collaborate effectively in a team environment.

What we offer:

• Competitive compensation depending on experience and skills.
• Regular assessments and salary reviews.
• Paid personal days.
• Mental health support program.
• Language classes.
• Flexible working schedule.

We are looking for a Team Lead Python Data Engineer to lead the design and operation of large-scale data pipelines that generate property attributes from imagery using our internal AI models.

This role focuses on leading the development of distributed processing systems running on Kubernetes clusters and on-prem GPU infrastructure, capable of processing massive geospatial datasets at national scale.

You will work closely with AI engineers and SaaS platform teams to guide the development of robust pipelines that transform imagery into structured property intelligence used by the insurance and financial industries.

Responsibilities

• Design and build large-scale Python processing pipelines for satellite and street-level imagery

• Develop distributed workloads running on Kubernetes clusters

• Build and maintain orchestration workflows using Airflow

• Design and manage data storage and processing workflows using PostgreSQL

• Integrate large-scale AI inference pipelines running on GPU infrastructure

• Optimize processing performance for massive imagery datasets

• Improve observability, monitoring, and reliability of production pipelines

• Collaborate with AI engineers to support large-scale computer vision inference workflows

• Integrate pipelines with AWS-based services

Requirements

• 7+ years of Python engineering experience

• Strong experience designing distributed data pipelines or large-scale data processing systems

• Experience running workloads on Kubernetes

• Hands-on experience with Airflow or similar orchestration frameworks

• Experience processing large datasets (imagery, geospatial, or other high-volume data)

• Strong understanding of ETL architectures and distributed data systems

Nice to Have

• Experience working with satellite imagery, aerial imagery, or remote sensing datasets

• Experience with GPU-based inference pipelines

• Experience working with on-prem compute clusters

• Experience with C#

We are looking for a Senior Python Data Engineer to design and operate large-scale data pipelines that generate property attributes from imagery using our internal AI models.

This role focuses on building distributed processing systems running on Kubernetes clusters and on-prem GPU infrastructure, capable of processing massive geospatial datasets at national scale.

You will work closely with AI engineers and SaaS platform teams to build robust pipelines that transform imagery into structured property intelligence used by the insurance and financial industries.

Responsibilities

• Design and build large-scale Python processing pipelines for satellite and street-level imagery

• Develop distributed workloads running on Kubernetes clusters

• Build and maintain orchestration workflows using Airflow

• Design and manage data storage and processing workflows using PostgreSQL

• Integrate large-scale AI inference pipelines running on GPU infrastructure

• Optimize processing performance for massive imagery datasets

• Improve observability, monitoring, and reliability of production pipelines

• Collaborate with AI engineers to support large-scale computer vision inference workflows

• Integrate pipelines with AWS-based services

Requirements

• 7+ years of Python engineering experience

• Strong experience designing distributed data pipelines or large-scale data processing systems

• Experience running workloads on Kubernetes

• Hands-on experience with Airflow or similar orchestration frameworks

• Experience processing large datasets (imagery, geospatial, or other high-volume data)

• Strong understanding of ETL architectures and distributed data systems

Nice to Have

• Experience working with satellite imagery, aerial imagery, or remote sensing datasets

• Experience with GPU-based inference pipelines

• Experience working with on-prem compute clusters

• Experience with C#

Our client is an Israeli startup specializing in geodata.

We are looking for a Senior Python Data Engineer to design and operate large-scale data pipelines that generate property attributes from imagery using our internal AI models.

This role focuses on building distributed processing systems running on Kubernetes clusters and on-prem GPU infrastructure, capable of processing massive geospatial datasets at national scale.

You will work closely with AI engineers and SaaS platform teams to build robust pipelines that transform imagery into structured property intelligence used by the insurance and financial industries.

Responsibilities

• Design and build large-scale Python processing pipelines for satellite and street-level imagery

• Develop distributed workloads running on Kubernetes clusters

• Build and maintain orchestration workflows using Airflow

• Design and manage data storage and processing workflows using PostgreSQL

• Integrate large-scale AI inference pipelines running on GPU infrastructure

• Optimize processing performance for massive imagery datasets

• Improve observability, monitoring, and reliability of production pipelines

• Collaborate with AI engineers to support large-scale computer vision inference workflows

• Integrate pipelines with AWS-based services

Requirements

• 7+ years of Python engineering experience

• Strong experience designing distributed data pipelines or large-scale data processing systems

• Experience running workloads on Kubernetes

• Hands-on experience with Airflow or similar orchestration frameworks

• Experience processing large datasets (imagery, geospatial, or other high-volume data)

• Strong understanding of ETL architectures and distributed data systems

Nice to Have

• Experience working with satellite imagery, aerial imagery, or remote sensing datasets

• Experience with GPU-based inference pipelines

• Experience working with on-prem compute clusters

• Experience with C#

Our client is an Israeli startup specializing in geodata.

Team Lead Python Data Engineer (Python)

Our client is an Israeli startup specializing in geodata.

We are looking for a Team Lead Python Data Engineer to lead the design and operation of large-scale data pipelines that generate property attributes from imagery using our internal AI models.

This role focuses on leading the development of distributed processing systems running on Kubernetes clusters and on-prem GPU infrastructure, capable of processing massive geospatial datasets at national scale.

You will work closely with AI engineers and SaaS platform teams to guide the development of robust pipelines that transform imagery into structured property intelligence used by the insurance and financial industries.

Responsibilities

• Design and build large-scale Python processing pipelines for satellite and street-level imagery

• Develop distributed workloads running on Kubernetes clusters

• Build and maintain orchestration workflows using Airflow

• Design and manage data storage and processing workflows using PostgreSQL

• Integrate large-scale AI inference pipelines running on GPU infrastructure

• Optimize processing performance for massive imagery datasets

• Improve observability, monitoring, and reliability of production pipelines

• Collaborate with AI engineers to support large-scale computer vision inference workflows

• Integrate pipelines with AWS-based services

Requirements

• 7+ years of Python engineering experience

• Strong experience designing distributed data pipelines or large-scale data processing systems

• Experience running workloads on Kubernetes

• Hands-on experience with Airflow or similar orchestration frameworks

• Experience processing large datasets (imagery, geospatial, or other high-volume data)

• Strong understanding of ETL architectures and distributed data systems

Nice to Have

• Experience working with satellite imagery, aerial imagery, or remote sensing datasets

• Experience with GPU-based inference pipelines

• Experience working with on-prem compute clusters

• Experience with C#

Our client is an Israeli startup specializing in geodata.

Requirements

Total of 5+ years of development/design experience with a minimum of 5 years experience in Big Data technologies on-prem or on cloud.
Experience with architecting, building, implementing and managing Big Data platforms On Cloud, covering ingestion (Batch and Real time), processing (Batch and Realtime), Polyglot Storage, Data Analytics and Data Access
Good understanding of Data Governance, Data Security, Data Compliance, Data Quality, Meta Data Management, Master Data Management, Data Catalog
Proven understanding and demonstrable implementation experience of big data platform technologies on cloud (AWS and Azure) including surrounding services like IAM, SSO, Cluster monitoring, Log Analytics etc
Experience working with Enterprise Data Warehouse technologies, Multi-Dimensional Data Modeling, Data Architectures or other work related to the construction of enterprise data assets
Strong Experience implementing ETL/ELT processes and building data pipelines including workflow management, job scheduling and monitoring
Experience building stream-processing systems, using solutions such as Apache Spark, Databricks, Kafka etc…
Experience with Spark/Databricks technology is a must
Experience with Big Data querying tools
Solid skills in Python
Strong experience with data modelling and schema design
Strong SQL programming background
Excellent interpersonal and teamwork skills
Experience to drive solution/enterprise-level architecture, collaborate with other tech leads
Strong problem solving, troubleshooting and analysis skills
Experience working in a geographically distributed team
Experience with leading and mentorship of other team members
Good knowledge of Agile Scrum
Good communication skills

 #LI-OE1

#LI-Remote

Job responsibilities

Work directly with the Client teams to understand the requirements/needs and rapidly prototype data and analytics solutions based upon business requirements
Design, implement and manage large scale data platform/applications including ingestion, processing, storage, data access, data governance capabilities and related infrastructure
Support Design and development of solutions for the deployment of data analytics notebooks, tools, dashboards and reports to various stakeholders
Communication with Product/DevOps/Development/QA team
Architect data pipelines and ETL/ELT processes to connect with various data sources
Design and maintain enterprise data warehouse models
Take part in the performance optimization processes
Guide on research activities (PoC) if necessary
Manage cloud based data & analytics platform
Establishing best practices with CICD under BigData scope

What we offer

Culture of caring. At GlobalLogic, we prioritize a culture of caring. Across every region and department, at every level, we consistently put people first. From day one, you’ll experience an inclusive culture of acceptance and belonging, where you’ll have the chance to build meaningful connections with collaborative teammates, supportive managers, and compassionate leaders. 

Learning and development. We are committed to your continuous learning and development. You’ll learn and grow daily in an environment with many opportunities to try new things, sharpen your skills, and advance your career at GlobalLogic. With our Career Navigator tool as just one example, GlobalLogic offers a rich array of programs, training curricula, and hands-on opportunities to grow personally and professionally.

Interesting & meaningful work. GlobalLogic is known for engineering impact for and with clients around the world. As part of our team, you’ll have the chance to work on projects that matter. Each is a unique opportunity to engage your curiosity and creative problem-solving skills as you help clients reimagine what’s possible and bring new solutions to market. In the process, you’ll have the privilege of working on some of the most cutting-edge and impactful solutions shaping the world today.

Balance and flexibility. We believe in the importance of balance and flexibility. With many functional career areas, roles, and work arrangements, you can explore ways of achieving the perfect balance between your work and life. Your life extends beyond the office, and we always do our best to help you integrate and balance the best of work and life, having fun along the way!

High-trust organization. We are a high-trust organization where integrity is key. By joining GlobalLogic, you’re placing your trust in a safe, reliable, and ethical global company. Integrity and trust are a cornerstone of our value proposition to our employees and clients. You will find truthfulness, candor, and integrity in everything we do.