Jobs Data Engineer

Location: Warsaw

The Project
You'll be joining a platform that's redefining how private capital markets operate — think PE firms, venture funds, and hedge funds using AI to make faster, smarter decisions on complex financial data.
This isn't a maintenance role. You'll be one of the key engineers shaping the data foundation of an AI-native PaaS/SaaS product from the ground up: designing schemas that LLMs can actually reason over, building pipelines that don't fall apart under real enterprise load, and untangling the legacy architecture that got the product to where it is today.
The problems are genuinely hard. The data is messy, the domain is complex, and the stakes are high — which is exactly what makes it interesting.

What You'll Be Working On
 

• Architect cloud-native data topologies and AI-friendly ontologies for structured and unstructured financial data
• Design and own scalable ETL/ELT pipelines using Prefect or Airflow
• Reverse-engineer and optimize existing data infrastructure

• Implement data governance, security standards, and CI/CD practices across the data layer

   

What We're Looking For

Must-have:

• 3+ years with Python (Pandas, ETL frameworks)
• 5+ years with complex SQL and RDBMS
• 5+ years of ETL orchestration (Airflow, Prefect)
• Hands-on experience with cloud DW: Redshift, Snowflake, or RDS
• Strong grasp of data modeling — OLAP, OLTP, Star/Snowflake schemas
• English B2+ — you'll communicate directly with the founding team

Nice to have: Docker, Kubernetes, PySpark, AWS Lambda, cloud certifications

Why This Role
 

• You'll have real architectural ownership, not just ticket execution
• The domain (private capital + AI) is genuinely niche and growing fast
• Direct access to founders — short feedback loops, no bureaucracy
• A team that takes engineering quality seriously and actually means it
   

Hiring Process Intro call → Technical Interview → Final with Founding Team

 

Join a Company That Invests in You

Seeking Alpha is the world’s leading community of engaged investors. We’re the go-to destination for investors looking for actionable stock market opinions, real-time market analysis, and unique financial insights. At the same time, we’re also dedicated to creating a workplace where our team thrives. We’re passionate about fostering a flexible, balanced environment with remote work options and an array of perks that make a real difference.

Here, your growth matters. We prioritize your development through ongoing learning and career advancement opportunities, helping you reach new milestones. Join Seeking Alpha to be part of a company that values your unique journey, supports your success, and champions both your personal well-being and professional goals.

What We're Looking For

Seeking Alpha is looking for a Senior Data Engineer responsible for designing, building, and maintaining the infrastructure necessary for analyzing large data sets. This individual should be an expert in data management, ETL (extract, transform, load) processes, and data warehousing and should have experience working with various big data technologies, such as Hadoop, Spark, and NoSQL databases. In addition to technical skills, a Senior Data Engineer should have strong communication and collaboration abilities, as they will be working closely with other members of the data and analytics team, as well as other stakeholders, to identify and prioritize data engineering projects and to ensure that the data infrastructure is aligned with the overall business goals and objectives.

What You'll Do

• Work closely with data scientists/analytics and other stakeholders to identify and prioritize data engineering projects and to ensure that the data infrastructure is aligned with business goals and objectives
• Design, build and maintain optimal data pipeline architecture for extraction, transformation, and loading of data from a wide variety of data sources, including external APIs, data streams, and data stores. 
• Continuously monitor and optimize the performance and reliability of the data infrastructure, and identify and implement solutions to improve scalability, efficiency, and security
• Stay up-to-date with the latest trends and developments in the field of data engineering, and leverage this knowledge to identify opportunities for improvement and innovation within the organization
• Solve challenging problems in a fast-paced and evolving environment while maintaining uncompromising quality.
• Implement data privacy and security requirements to ensure solutions comply with security standards and frameworks.
• Enhance the team's dev-ops capabilities.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• 2+ years of proven experience developing large-scale software using an object-oriented or functional language.
• 5+ years of professional experience in data engineering, focusing on building and maintaining data pipelines and data warehouses
• Strong experience with Spark, Scala, and Python, including the ability to write high-performance, maintainable code
• Experience with AWS services, including EC2, S3, Athena, Kinesis/Firehose Lambda and EMR
• Familiarity with data warehousing concepts and technologies, such as columnar storage, data lakes, and SQL
• Experience with data pipeline orchestration and scheduling using tools such as Airflow
• Strong problem-solving skills and the ability to work independently as well as part of a team
• High-level English - a must. 
• A team player with excellent collaboration skills.

Nice to Have:

• Expertise with Vertica or Redshift, including experience with query optimization and performance tuning
• Experience with machine learning and/or data science projects
• Knowledge of data governance and security best practices, including data privacy regulations such as GDPR and CCPA.
• Knowledge of Spark internals (tuning, query optimization)

What We’re Looking For

• Strong foundation in cybersecurity
• Hands-on experience with at least one major cloud platform (AWS preferred; Azure or GCP also valid). Comfort navigating services, permissions models, and APIs/SDK concepts.
• Ability to translate security and customer needs into high-value product work (prioritization, impact vs effort, clear definitions of “done”).
• Good communication skills: you can explain security decisions and tradeoffs to both technical and non-technical stakeholders.
• Solid engineering literacy (you can read and review code, reason about reliability and edge cases). Deep coding expertise is not required.

Bonus Points For

• Experience with CSPM/CNAPP tools, security research, or building detection/analytics pipelines.
• Familiarity with cloud telemetry/log sources and correlating security signals.
• Comfort with Infrastructure as Code concepts (e.g., Terraform) and cloud-native environments.

We are looking for a Data / Automation Engineer to work from our office and help build, optimize, and scale automation workflows and data processes.

Location:

Office-based (full-time, on-site)

Requirements:

• Strong SQL skills (JOINs, aggregations, query optimization)
• Understanding of APIs (REST, webhooks)
• Experience with automation tools (n8n, Zapier, or similar)
• Basic understanding of data pipelines (ETL / ELT)
• Experience working with databases (ClickHouse, PostgreSQL, or others)

Nice to have:

• Basic or higher level of Python
• Experience with Airflow or other orchestration tools
• Experience working with CRM systems (Zoho CRM, Slack, or others)
• Experience with AI / LLM

Responsibilities:

• Develop and maintain automation workflows (n8n, webhooks, APIs)
• Write and optimize SQL queries
• Build and maintain data pipelines
• Integrate CRM systems
• Work with Airflow and other orchestration tools
• Debug and resolve issues in pipelines and automations
• Analyze and improve existing processes

We offer:

• Office-based work environment
• Stable workload and clear processes
• Opportunity to influence technical decisions
• Professional growth and development

Google Cloud Solutions Architect / Pre-Sales Engineer

Company: Softprom Solutions
Location: Remote (Ukraine)
Employment: Full-time, FOP (contractor)

About the role

Softprom Solutions is looking for a Google Cloud Solutions Architect / Pre-Sales Engineer to join our Cloud team.

This role is ideal for a specialist with hands-on experience in Google Cloud Platform and an active Google Cloud certification, who wants to work on real commercial projects, participate in pre-sales activities, and grow in cloud architecture.

Responsibilities

• Design and document Google Cloud architectures according to the Google Cloud Architecture Framework
• Participate in pre-sales activities:
  • technical discovery with customers
  • solution design
  • participation in demos and presentations
• Deployment and configuration of core GCP services, including:
  • Compute Engine, Cloud Storage, Cloud SQL
  • VPC, IAM, Load Balancing
  • Cloud Functions / Cloud Run
• Design and configure GCP networking:
  • VPC networks, subnets
  • Firewall rules
  • Routes
• Implement and support Infrastructure as Code (IaC) using Terraform
• Create technical and solution documentation
• Act as a technical point of contact for sales and customers

Requirements (Must have)

• Active Google Cloud certification
  (Associate Cloud Engineer or Professional Cloud Architect)
• Experience as Cloud Engineer / Solutions Architect / Pre-Sales Engineer
• Practical understanding of Google Cloud core services:
  • Compute, Storage, Databases, Networking, Security
• Solid networking knowledge:
  • TCP/IP (L3, L4)
  • IP addressing, subnetting
  • DNS
• Understanding of cloud principles:
  • High Availability
  • Fault Tolerance
  • Scalability
  • Security
• Linux skills (command line)

Nice to have

• Experience in pre-sales or customer-facing roles
• Experience with Terraform
• Basic scripting skills (Python, Bash)
• Experience with Docker, Kubernetes (GKE), Cloud Run
• Experience with AWS or multi-cloud environments

Soft skills

• Ability to communicate technical solutions to non-technical audiences
• Structured and analytical thinking
• Proactivity and ownership
• Ukrainian — fluent
• English — reading and basic spoken (technical / pre-sales)

We offer

• Compensation: $2000 USD + project-based bonuses
• FOP cooperation
• Full-time workload
• Real commercial Google Cloud projects
• Participation in pre-sales and architecture design
• Professional growth in Cloud & Solutions Architecture
• International projects and vendors
• Strong cloud team and mentorship

AWS Cloud Engineer

Softprom Solutions
Azerbaijan | Remote / Hybrid
Full-time | Contractor (Individual Entrepreneur)
Contract with Austria

About Softprom

Softprom Solutions is an international IT distributor and solutions provider working with leading global vendors in Cloud, Cybersecurity, Infrastructure, and Enterprise IT.

We are expanding our Cloud Practice and are looking for an AWS Cloud Engineer in Azerbaijan who wants to grow professionally, work on real customer projects, and collaborate with international teams under an Austrian contract.

About the role

This role is ideal for an AWS-certified engineer with solid fundamentals who wants to deepen hands-on experience in cloud architecture, deployments, automation, and customer-facing work.

You will work closely with senior architects, sales teams, and customers, participating in both technical delivery and pre-sales activities.

Responsibilities

• Support the design and documentation of AWS cloud architectures following the AWS Well-Architected Framework
   
• Participate in deployment and configuration of core AWS services, including
  VPC, EC2, S3, RDS, IAM, Lambda, Load Balancers
   
• Assist with AWS networking configuration:
  Subnets, Route Tables, Security Groups, NACLs
   
• Contribute to automation and Infrastructure as Code (IaC) initiatives using
  Terraform and/or AWS CloudFormation
   
• Create and maintain technical documentation for architectures and configurations
   
• Participate in customer meetings, presentations, and demos, explaining AWS solutions and capabilities
  
   

Requirements (Must have)

• Active AWS Certified Solutions Architect – Associate (SAA-C03)
   
• Solid theoretical understanding of AWS core services:
  Compute, Storage, Databases, Networking, Security
   
• Basic networking knowledge:
  TCP/IP (L3, L4), IP addressing, subnetting, DNS
   
• Understanding of cloud principles:
  High Availability, Fault Tolerance, Scalability, Security
   
• Basic Linux skills (command line)
   
• Ability to read and understand technical documentation in English
  
   

Nice to have

• Hands-on experience with Terraform or CloudFormation
   
• Basic scripting skills (Python, Bash)
   
• Familiarity with Docker, Kubernetes, ECS
   
• Personal, educational, or non-commercial projects deployed on AWS
  
   

Soft skills

• Strong motivation to learn and grow in cloud engineering
   
• Structured and analytical thinking
   
• Clear and confident communication
   
• Ukrainian or Russian — fluent
   
• English — reading technical documentation (spoken English is a plus)
  
   

We offer
 

• Individual Entrepreneur (FOP) / contractor model
   
• Official contract with Austria
   
• Full-time workload
   
• Real commercial AWS projects (not internal labs)
   
• Participation in architecture design and pre-sales activities
   
• Professional growth within Cloud & Solutions Architecture
   
• International customers and vendors
   
• Supportive, senior cloud team and mentorship

N-iX is a global software development service company that helps businesses across the globe create next-generation software products. Founded in 2002, we unite 2,400+ tech-savvy professionals across 40+ countries, working on impactful projects for industry leaders and Fortune 500 companies. Our expertise spans cloud, data, AI/ML, embedded software, IoT, and more, driving digital transformation across finance, manufacturing, telecom, healthcare, and other industries. Join N-iX and become part of a team where your ideas make a real impact.
 

The project focuses on building a cloud-based logistics analytics platform that provides standardized, high-quality reporting and insights across complex, global supply chain processes. The solution is designed to support data-driven decision-making by making logistics data transparent, reliable, and easy to consume for business users across multiple organizational units.

Built on modern cloud technologies, the platform combines scalable data warehousing, automated data pipelines, and strong data governance to transform large volumes of logistics data into trusted KPIs and analytical datasets. It plays a critical role in improving efficiency, visibility, and operational performance across the end-to-end logistics value stream.

We’re looking for someone who truly understands data flows, thinks analytically, and can see the bigger picture beyond individual tasks. If you’re curious, ask the right questions, and enjoy bringing smart ideas to the table, we’re open to some flexibility in specific requirements. Finding the right person matters more to us than ticking every box. 

Responsibilities

• Implement business logic in the Data Warehouse in accordance with functional and technical specifications
• Perform light business analysis to ensure data is delivered accurately and in a meaningful, business-relevant way
• Translate business requirements into scalable and efficient data models
• Design, build, and maintain ETL pipelines using Azure Data Factory
• Optimize data loading processes and tune query performance
• Collaborate closely with senior stakeholders on the customer side to refine requirements and propose data-driven improvements
• Actively contribute ideas and solutions to improve data architecture and pipeline reliability

Requirements

• 3+ years of experience working with Microsoft Azure and readiness to work primarily with SQL (up to 80% of the time)
• Proven experience in database development using MS SQL Server / T-SQL
• Ability to write highly performant SQL and implement performance optimization techniques
• Hands-on experience with version control systems (e.g. Git)
• Solid understanding of data warehousing concepts and dimensional modeling
• Experience working in an Agile development environment
• Practical knowledge of creating and maintaining Azure DevOps and Azure Data Factory pipelines
• Experience developing robust data pipelines using dbt
   

Nice to Have

• Experience with Databricks
• Background in Supply Chain & Logistics projects
• Familiarity with SAP MM data structures

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers

Description

We are looking for a Middle/Senior DWH Engineer to design, build, and maintain scalable data warehouse solutions that power analytics and business decision-making. You will work across the full data lifecycle — from ingestion and modeling to optimization, reliability, and automation — collaborating closely with analysts, developers, and business stakeholders.

Requirements

• Strong SQL skills: complex queries, CTEs, window functions, analytical queries (5+ years experience);
• Knowledge of Python or other scripting languages for data transformations (3+ years experience);
• Deep understanding of DWH concepts: ETL/ELT, Data Vault, Kimball, Star/Snowflake schemas (4+ years experience);
• Experience with Airflow or other data pipeline orchestrators (3+ years experience);
• Hands-on experience with modern DWH and query engines: BigQuery, Snowflake, Redshift, ClickHouse, Vertica, AWS Athena, Trino (2+ years experience);
• Confident use of Git; experience with team workflows (pull requests, rebasing, merge conflict resolution) (5+ years experience);
• Understanding of server and cloud infrastructure: basic skills in configuration, maintenance, monitoring, and load control (2+ years experience).

Nice to Have

• Experience with CDC tools and streaming data sources;
• Knowledge of Docker, Kubernetes, and Infrastructure as Code (Terraform);
• Experience with cloud platforms: AWS, GCP, or Azure;
• Familiarity with data governance, data cataloging, and lineage tools.

Responsibilities:

• Design, build, and maintain scalable Data Warehouse architectures aligned with business needs;
• Develop and optimize ETL/ELT pipelines using Python/Airflow and custom solutions;
• Work with DWH\Datalake: PostgreSQL, Trino, BigQuery;
• Implement incremental loads, CDC, backfills, and reprocessing strategies;
• Optimize query performance, data models, and pipeline execution;
• Ensure data quality through validation, automated testing, monitoring, and alerting;
• Integrate new data sources (APIs, third-party systems, raw data) without disrupting existing pipelines;
• Collaborate with analysts, engineers, BI teams, and business stakeholders to translate requirements into data solutions;
• Mentor engineers, review code, and contribute to data standards and best practices.

Benefits

💸 Flexible payment options: choose the method that works best for you;
🧾 Tax assistance included: we handle part of your taxes and provide guidance on the local setup;
🎁 Financial perks: Bonuses for holidays, B-day, work milestones and more - just to show we care;
📈 Learn & grow: We cover courses and certifications — and offer real opportunities to grow your career with us;
🥐 Benefit Сafeteria: Choose what suits you — sports, language courses, therapy sessions, and more;
🎉 Stay connected: From team-building events to industry conferences — we bring people together online, offline, and on stage;
💻 Modern Equipment: We provide new laptops along with essential peripherals like monitors and headphones for a comfortable workflow;
🕘 Your schedule, your rules: Start your day at 9, 10, or even 11 — we care about results, not clock-ins.

PwC is a global network of more than 370,000 professionals in 149 countries that turns challenges into opportunities. We create innovative solutions in audit, consulting, tax and technology, combining knowledge from all over the world.

PwC SDC Lviv, opened in 2018, is part of this global space. It is a place where technology is combined with team spirit, and ambitious ideas find their embodiment in real projects for Central and Eastern Europe.

What do we guarantee?

• Work format: Remote or in a comfortable office in Lviv - you choose.
• Development: Personal development plan, mentoring, English and Polish language courses.
• Stability: Official employment from day one, annual review of salary and career prospects.
• Corporate culture: Events that unite the team and a space where everyone can be themselves.

We are looking for a talented and experienced Cloud Engineer to join our growing team. As well as experience with cloud infrastructure, networking, and security, the Cloud Platform Engineer will be responsible for operating, monitoring, and maintaining our many Azure subscriptions and associated global applications. As a Cloud Platform Engineer with cloud architecture, you will work closely with other data professionals, software engineers, and business stakeholders to build, maintain and support efficient, scalable, and secure data pipelines. This role involves both engineering and operations, requiring comfort with project work and handling incident tickets as part of the team.

Responsibilities: 

• Design, build and maintain Azure based infrastructure with Terraform modules (IaaS and PaaS resources) as part of new projects or updates needed to production environments;
• Troubleshoot IaaS and PaaS issues raised via SNow incident and meet ticket closure SLA’s;
• Documentation and knowledge sharing: create and maintain environment diagrams, playbooks, runbooks, and best practice documents;
• Communicate with internal business partners including guidance on application design.

Qualifications:

• 2+ years of experience with Azure core services: Resource Manager, Virtual Machines/Scale Sets, Storage (Blob/Files), Networking (VNets, NSGs, Load Balancer, App Gateway, DNS, Private Link);
• 2+ year of experience in Windows or Linux administration, with strong problem-solving abilities;
• 2+ years of experience with Azure Kubernetes Service;
• 2+ years of experience with Terraform;
• Ability to work independently and initiate tasks without supervision and utilize strong interpersonal and organizational skills;
• Excellent problem-solving and analytical skills.

Policy statements:
https://www.pwc.com/ua/uk/about/privacy.html

We’re looking for a Data Engineer to improve our data infrastructure and integration workflows.

This role is a great fit for someone who can quickly step into an existing environment, bring strong technical expertise, and help us build reliable, scalable solutions with a practical hands-on approach.

What you’ll do:

• Design and build data templates for integration workflows
• Optimize and improve existing data pipelines
• Ensure smooth and reliable integration across SaaS systems
• Work closely with the team to improve performance, stability, and maintainability of data flows

What we’re looking for:

• Commercial experience as a Data Engineer
• Hands-on experience with Azure
• Knowledge of Snowflake
• Python skills for data engineering tasks
• Experience working with SaaS products and integrations
• Understanding of building efficient, scalable, and maintainable data pipelines

Would be a plus:

• Experience with ETL / ELT
• Experience with cloud-based data platforms and modern data stack
• Ability to work independently and move fast in a short-term project setup

We are looking for an experienced Data Engineer to leverage their skillset in a fast-paced financial trading environment. You will be responsible for designing, developing, and maintaining data pipelines and infrastructure across GCP , and Oracle-based systems, ensuring high-performance data flows that support trading, risk, and compliance functions. This role requires a deep understanding of cloud architecture, large-scale data processing, and a strong grasp of agile delivery practices.
 

About the client
 

An international consulting firm that helps companies of all sizes have a better impact on the world. Company capabilities focus on supporting the private and public sectors with their people, processes, and digital technology challenges.
 

What You Will Do
 

● Design, build, and maintain scalable data pipelines primarily within Google Cloud Platform
 

● Develop integration and transformation workflows between cloud data services and on-prem Oracle databases
 

● Work closely with trading, risk, and analytics teams to understand data requirements and deliver real-time and batch data solutions
 

● Optimise and monitor performance of data systems to support latency-sensitive trading applications
 

● Collaborate with cross-functional teams using Agile/Scrum methodologies to deliver business-critical data projects
 

● Ensure robust data governance, lineage, and compliance (including MiFID II, FCA, and other regulatory standards)
 

● Automate data workflows using Terraform, CI/CD pipelines, and containerisation tools (Docker/Kubernetes)
 

What you will bring
 

● Strong experience with Google Cloud Platform (GCP) e.g. BigQuery, DataPlex, DBT, Pub/Sub, Dataflow, S3, Glue, Redshift
 

● Expertise in Oracle SQL, PL/SQL, and working with complex stored procedures and large datasets
 

● Proficiency in programming languages such as Python, Java, or Scala
 

● Experience with streaming and messaging systems (e.g. Jenkins, GitLab), and container orchestration (Kubernetes)
 

● Experience working with Infrastructure as Code, preferably Terraform, to manage cloud data infrastructure.
 

● Deep understanding of data modelling, data warehousing, and ETL/ELT design patterns
 

● Familiarity with Agile development practices (Scrum, Kanban, Jira)
 

● Exposure to financial markets, trading systems, or related high-performance environments is a strong plus
 

Nice to have
 

● GCP certification (e.g., Professional Data Engineer or similar) will be considered a strong advantage.
 

● Experience with Amazon Web Services (AWS)
 

● Knowledge of regulatory reporting, market data, or trade surveillance systems
 

● Experience with Apache Airflow, DBT, or similar orchestration tools
 

● Understanding of data security practices and compliance frameworks

Team Lead Python Data Engineer (Python)

Our client is an Israeli startup specializing in geodata.

We are looking for a Team Lead Python Data Engineer to lead the design and operation of large-scale data pipelines that generate property attributes from imagery using our internal AI models.

This role focuses on leading the development of distributed processing systems running on Kubernetes clusters and on-prem GPU infrastructure, capable of processing massive geospatial datasets at national scale.

You will work closely with AI engineers and SaaS platform teams to guide the development of robust pipelines that transform imagery into structured property intelligence used by the insurance and financial industries.

Responsibilities

• Design and build large-scale Python processing pipelines for satellite and street-level imagery

• Develop distributed workloads running on Kubernetes clusters

• Build and maintain orchestration workflows using Airflow

• Design and manage data storage and processing workflows using PostgreSQL

• Integrate large-scale AI inference pipelines running on GPU infrastructure

• Optimize processing performance for massive imagery datasets

• Improve observability, monitoring, and reliability of production pipelines

• Collaborate with AI engineers to support large-scale computer vision inference workflows

• Integrate pipelines with AWS-based services

Requirements

• 7+ years of Python engineering experience

• Strong experience designing distributed data pipelines or large-scale data processing systems

• Experience running workloads on Kubernetes

• Hands-on experience with Airflow or similar orchestration frameworks

• Experience processing large datasets (imagery, geospatial, or other high-volume data)

• Strong understanding of ETL architectures and distributed data systems

Nice to Have

• Experience working with satellite imagery, aerial imagery, or remote sensing datasets

• Experience with GPU-based inference pipelines

• Experience working with on-prem compute clusters

• Experience with C#

Our client is an Israeli startup specializing in geodata.

Our client is an Israeli startup specializing in geodata.

We are looking for a Senior Python Data Engineer to design and operate large-scale data pipelines that generate property attributes from imagery using our internal AI models.

This role focuses on building distributed processing systems running on Kubernetes clusters and on-prem GPU infrastructure, capable of processing massive geospatial datasets at national scale.

You will work closely with AI engineers and SaaS platform teams to build robust pipelines that transform imagery into structured property intelligence used by the insurance and financial industries.

Responsibilities

• Design and build large-scale Python processing pipelines for satellite and street-level imagery

• Develop distributed workloads running on Kubernetes clusters

• Build and maintain orchestration workflows using Airflow

• Design and manage data storage and processing workflows using PostgreSQL

• Integrate large-scale AI inference pipelines running on GPU infrastructure

• Optimize processing performance for massive imagery datasets

• Improve observability, monitoring, and reliability of production pipelines

• Collaborate with AI engineers to support large-scale computer vision inference workflows

• Integrate pipelines with AWS-based services

Requirements

• 7+ years of Python engineering experience

• Strong experience designing distributed data pipelines or large-scale data processing systems

• Experience running workloads on Kubernetes

• Hands-on experience with Airflow or similar orchestration frameworks

• Experience processing large datasets (imagery, geospatial, or other high-volume data)

• Strong understanding of ETL architectures and distributed data systems

Nice to Have

• Experience working with satellite imagery, aerial imagery, or remote sensing datasets

• Experience with GPU-based inference pipelines

• Experience working with on-prem compute clusters

• Experience with C#

Our client is an Israeli startup specializing in geodata.

About the Company

Established in 2004, ALLSTARSIT was founded with a clear vision: to enhance the landscape of global IT employment by bridging the gap between companies and skilled professionals. The core belief was that assembling a team shouldn't be hindered by geographical constraints. Fast forward to the present day, ALLSTARSIT stands as an international outstaffing service provider committed to change the way businesses recruit, compensate, and oversee top talent worldwide. 

With operational hubs scattered across Europe, Asia, and LATAM, and its headquarters situated in San Francisco, US, the company boasts a workforce of over 1,000 adept professionals. Spanning across more than 20 countries, ALLSTARSIT offers a diverse range of skilled employees across various verticals, including AI, cybersecurity, healthcare, fintech, telecom, media, and so on.

About the Project

The client’s company is revolutionizing the way new drugs and vaccines are approved for market by replacing manual processes with deep tech. Their core product, the project, is an AI-powered statistical analysis workspace designed to improve workflows and accelerate regulatory approval processes.
Accurate analysis of clinical research data is critical to helping millions of people receive safe and effective medical treatments faster. The project provides assurance that the data underpinning these analyses are of the highest integrity. By joining the client’s team, you can directly impact the speed at which new medications reach patients—many of whom may be your family or friends. You will also help drive the translation of meaningful ideas into beneficial products with the support of an inclusive and diverse group of highly talented professionals.
We are seeking a highly motivated and experienced Senior Data Engineer to join the client’s team.

Required skills:

• 3+ years of experience in Python development (or 1+ year of Python plus 2+ years in another language)
• 2+ years of experience in production data environments
• Strong experience with modern data engineering tools and libraries (e.g., Pandas, PySpark, SQLAlchemy)
• Hands-on experience with Databricks or other distributed data processing frameworks
• Proficiency in working with structured and semi-structured data from various sources (databases, APIs, files, etc.)
• Experience with Agile methodologies and tools
• Familiarity with data modeling, data warehousing, and performance optimization
• Excellent analytical and problem-solving skills
• Strong English communication skills, both written and verbal
• Team player with strong collaboration and interpersonal abilities

Advantages

• Experience with Databricks or similar platforms for large-scale data processing and analytics, as well as data orchestration tools and MLOps frameworks
• Experience delivering AI-ready datasets and knowledge of AI/ML frameworks
• Proficiency with regular expressions
• Experience with NoSQL databases (e.g., MongoDB)
• Familiarity with cloud services (AWS and/or Azure)
• Hands-on experience with Python testing and debugging tools such as PyTest
• Experience building and maintaining CI/CD pipelines
• Background in the pharmaceutical or life sciences industry
• Bachelor’s degree in Computer Science, Engineering, a related technical field, or equivalent practical experience

Scope of work:

• Design and implement robust, scalable data pipelines and workflows using modern data engineering frameworks
• Collaborate closely with Data Scientists, Analysts, Developers, and Product Managers to support data needs and model deployment
• Take ownership of data quality, data architecture, and production readiness across systems
• Build and optimize data extraction, transformation, and loading (ETL) processes across structured and unstructured datasets
• Develop reusable data processing components and libraries
• Troubleshoot data issues, performance bottlenecks, and maintain high data integrity across environments
• Contribute to the evolution of the client’s cloud-based data infrastructure
• Document data workflows and participate in the development and execution of test strategies for data systems

Main Responsibilities:

Data Engineer is responsible for helping select, deploy, and manage the systems and infrastructure required of a data processing pipeline to support customer requirements.

You will work on cutting-edge cloud technologies, including Microsoft Fabric, Azure Synapse Analytics, Apache Spark, Data Lake, Data Bricks, Data Factory, Cosmos DB, HD Insights, Stream Analytics, Event Grid in the implementation projects for corporate clients all over EU, CIS, United Kingdom, Middle East.

Our ideal candidate is a professional passionated with technologies, a curious and self-motivated person.

Responsibilities revolve around DevOps and include implementing ETL pipelines, monitoring/maintaining data pipeline performance, model optimization

Mandatory Requirements:

– 3+ years of experience, ideally within a Data Engineer role.

– understanding of data modeling, data warehousing concepts, and ETL processes

– 2+ years of experience with Azure Cloud technologies

– experience in distributed computing principles and familiarity with key architectures, broad experience across a set of data stores (Azure Data Lake Store, Azure Synapse Analytics, Apache Spark, Azure Data Factory)

– Understanding of landing, staging area, data cleansing, data profiling, data security and data architecture concepts (DWH, Data Lake, Delta Lake/Lakehouse, Datamart)

– SQL-skills

– communication and interpersonal skills

– English —В2

Will be beneficial if a candidate has experience in SQL migration from on-premises to cloud, data modernization and migration, advanced analytics projects, and/or professional certification in data&analytics.

We offer:

– professional growth and international certification

– free of charge technical and business trainings and the best bootcamps (worldwide, including HQ Microsoft- Redmond courses)

– innovative data & analytics projects, practical experience with cutting-edge Azure data&analytics technologies at various customers’ projects

– great compensation and individual bonus remuneration

– medical insurance

– long-term employment

– ondividual development plan