Jobs Data Engineer

ORIL is seeking a highly experienced and autonomous Data Architect / Lead Data Engineer to lead a 2–3 month strategic project focusing on a comprehensive audit and redesign of our mission-critical, AI-driven real estate data platform.

The ideal candidate is a "rockstar" data expert with strong data architecture, data discovery, and data engineering expertise. You will take complete ownership of evaluating our current data layer, determining the target future state (Desired State), and defining a concrete execution roadmap for the engineering team.

About the Project

Our client is a US-based innovator providing a centralized data platform for multi-family real estate. The platform aggregates complex real estate datasets from various external integrations and sources to feed real-time BI analytics (Metabase), power automated operational insights, and drive an intelligent Text-to-SQL AI chatbot.

What are we looking for?

• 6+ years of experience as a Data Architect, Lead Data Engineer, or Senior Database Engineer;
• Proven track record of conducting end-to-end Data Discovery and Audits of complex, multi-source data architectures;
• Deep technical expertise in PostgreSQL and relational database design, including advanced query optimization, indexing, schema design, and data modeling;
• Hands-on experience with Data Quality, Data Governance, and Data Integrity frameworks (monitoring freshness, volume anomaly, schema drift, semantic metadata validation);
• Extensive background in evaluating and architecting ETL/ELT data pipelines and integrations (APIs, web scraping, CDC pipelines);
• Strong understanding of structuring metadata, schemas, and semantic layers for AI/LLM integrations (Text-to-SQL) and BI reporting tools (Metabase);
• Ability to translate complex business needs into high-level technical vision, concrete architecture blueprints, and resource estimates;
• English level: Fluent or Advanced (C1) with strong stakeholder-facing communication skills for direct interactions with US client leadership.

What will you do?

•  Evaluate the current state of all data collected across multiple integrations — how data is ingested, processed, stored, and presented across the platform.
• Analyze schema design, data freshness, bottleneck risks, and overall data health to identify system vulnerabilities and inaccuracies.
• Design a clean, scalable, and future-proof target data architecture optimized for fast querying, BI reporting (Metabase), and AI/LLM ingestion (Text-to-SQL).
• Build a detailed step-by-step execution plan (Roadmap) specifying required engineering steps, timelines, and resource allocations needed to reach the desired state.
• Present audit findings and strategic recommendations directly to the US client leadership and product team.

We Offer:

• Project-based contract engagement (2–3 months) with flexible Part-time or Full-time capacity;
• Potential for long-term cooperation upon successful project delivery;
• Competitive, market-leading compensation;
• Full accounting support;
• Fully remote work flexibility;
• Agile, transparent engineering culture.

Application Process:

1. Initial Screening Call with Recruiter.
2. Technical & Architecture Deep Dive with CTO.
3. Culture Index (CI) Test.

N-iX is looking for a Junior Data Engineer to join a data-focused team supporting the day-to-day operations and ongoing development of enterprise data solutions. This is a 6-month engagement with an opportunity to gain hands-on experience working with modern cloud data platforms, large-scale datasets, and production data pipelines.

You will work closely with experienced Data Engineers, contributing to pipeline maintenance, troubleshooting, onboarding new data sources, and implementing well-defined enhancements across the data platform. This role is ideal for engineers with 1-2 years of professional experience who want to deepen their expertise in data engineering and cloud technologies.

About the Role

As a Junior Data Engineer, you will help ensure the reliability and quality of data pipelines that support business-critical analytics and reporting. You'll participate in monitoring, troubleshooting, testing, and implementing data processing logic while following established engineering standards and delivery practices.

Under the guidance of senior team members, you will contribute to data ingestion processes, validation activities, technical documentation, and operational support, gaining exposure to modern technologies such as Azure, Databricks, PySpark, and Snowflake.

Responsibilities

• Monitor and support scheduled and event-driven data pipelines, data processing jobs, reporting refreshes, and data quality checks.
• Execute standard data ingestion and refresh activities from multiple sources, including APIs, CSV files, syndicated datasets, and configuration inputs.
• Implement well-defined Python, PySpark, and SQL changes, including new filters, transformations, mappings, validation rules, and minor pipeline enhancements.
• Perform schema validation, reconciliation checks, regression testing, and source-to-target data verification.
• Investigate pipeline failures, validation issues, missing data, rejected files, and reporting discrepancies.
• Troubleshoot first-line operational incidents using logs, job outputs, and monitoring tools.
• Escalate complex technical or business-data issues with clear diagnostic information and supporting evidence.
• Maintain technical documentation, deployment notes, runbooks, configuration files, and operational records.
• Follow established engineering practices, including Git workflows, pull requests, code reviews, testing, and deployment procedures.
• Support onboarding of new data sources and contribute to continuous improvement initiatives.

Requirements

Must Have

• 1-2 years of professional experience in Data Engineering, ETL/ELT development, Analytics Engineering, or Data Support roles.
• Good working knowledge of Python and SQL.
• Understanding of data pipelines, data transformations, batch processing, and ETL/ELT concepts.
• Familiarity with relational databases and data warehouse concepts.
• Experience working with API integrations and file-based data processing (CSV, Parquet, etc.).
• Hands-on experience with Git, version control workflows, pull requests, and code reviews.
• Understanding of data quality principles, including validation, reconciliation, schema checks, and error handling.
• Ability to analyze logs, troubleshoot issues, and communicate findings clearly.
• Strong analytical and problem-solving skills.
• Intermediate or higher English level.

Nice to Have

• Experience with Azure Data Factory, Azure Databricks, PySpark, Snowflake, Delta Lake, Azure Data Lake Storage, or similar cloud data technologies.
• Exposure to modern data platform architectures, including Medallion (Bronze-Silver-Gold) data processing patterns.
• Familiarity with Power BI or other BI/reporting platforms.
• Knowledge of cloud environments and DevOps practices.
• Experience with data validation frameworks such as Pandera.
• Understanding of CI/CD pipelines and deployment processes.

What Makes You a Great Fit

• You enjoy working with data and solving technical problems.
• You are detail-oriented and take ownership of the quality of your work.
• You are comfortable working within established processes while continuously learning new technologies.
• You communicate clearly and collaborate effectively with technical and non-technical stakeholders.
• You are proactive, curious, and eager to grow your data engineering skills in a production environment.

ZentixSoft is looking for a Senior Data Solutions Consultant / Solution Architect! 🚀

Format: Project-Based / Hourly Engagement (Remote). 

💡 Why ZentixSoft?

💸 Fair Compensation: Transparent hourly/project-based rates for expert consulting hours.

⚖️ Work-Life Balance: Flexible scheduling tailored around your primary availability.

🦾 Trust & Freedom: Pure professional autonomy with zero micromanagement and no time-trackers.

🎁 Culture & Community: A collaborative environment where your architectural decisions directly drive business outcomes.

🧩 Project & Role Overview:

You will support our technical and presales teams during the discovery and solution design phase of a complex product data management initiative (handling legacy data sources, supplier onboarding, taxonomy, attribute mapping, and multi-system synchronization).

 Discovery & Workshops: Participate in client-facing discovery sessions and technical workshops to unpack legacy architecture and business constraints.

 Architecture Design: Analyze product-data flows, identify systems of record, and distinguish between integration/ETL challenges versus semantic data-mapping issues.

 Roadmap & Strategy: Define realistic target architecture options, phased implementation roadmaps, and potential pilot use cases.

 PIM / MDM & AI Integration: Evaluate approaches for supplier data onboarding, taxonomy harmonization, and assess where AI/LLM automation adds real value vs. deterministic rules/governance.

 Proposal Support: Help formulate solution proposals, architecture diagrams, and technical recommendations for business and technical stakeholders.

🎓 Our Perfect Match (Requirements):

 Domain Experience: 5+ years of hands-on experience in data architecture, system integrations, and solution design.

 PIM / MDM / Product Data Expertise: Strong background in Product Information Management (PIM), Master Data Management (MDM), product taxonomy, attribute mapping, and supplier onboarding.

 Platform Exposure: Experience with platforms such as Akeneo, Salsify, Stibo Systems, Informatica, Contentserv, Pimcore, or inriver (or similar product catalog ecosystems).

 Technical Integration & Data Quality: Familiarity with ERP/PIM/e-commerce integrations, product data standards (e.g., BMEcat), data deduplication, golden-record concepts, and legacy SQL/Access environments.

 Modern Tech Evaluation: Understanding of AI/LLM-assisted data classification and automated mapping techniques.

 Consulting & Communication: Proven experience leading technical workshops with clients, with Fluent / Confident English for client-facing communication.

📍 Engagement Details:

⏳ Format: Project-based / Hourly consulting that can be easily combined with your full time occupation.

📍 Location: 100% Remote.

We look forward to reviewing your profile:

The Mission
Are you a cloud architect who finds beauty in a perfectly orchestrated pipeline? Do you believe that “manual” is a four-letter word? We are looking for a Senior SRE specialized in BI Ecosystems to join our mission-critical Infrastructure team.
In this role, you won’t just “maintain” systems; you will be the master weaver connecting our advanced Data Analytics tools with a rock-solid, automated GCP/AWS backbone. You will bridge the gap between traditional Business Intelligence and modern Cloud Engineering, ensuring our data flows are fast, our visualizations are seamless, and our infrastructure is invisible.

What You Will Do
1. Master the Orchestration (GCP & GKE)
You will take full ownership of our Google Kubernetes Engine (GKE) clusters. This isn’t just
about pods and services; you’ll manage advanced networking (VPC Service Controls, Shared VPC) and high-availability node configurations to ensure our data platforms never skip a beat.
2. Advance Data Architectures (Lakehouse)
You will design and support a unified Data Lakehouse architecture by integrating BigQuery and Cloud Storage. Your mission is to break down silos, ensuring that our raw data and analytical models coexist in a single, high-performance ecosystem. You will implement robust governance and security policies that ensure data quality and accessibility, providing a “single source of truth” that is both scalable and cost-effective
3. Evangelize Infrastructure as Code (IaC)
Our goal is 100% immutability. You will use Terraform to automate the entire lifecycle of our
GCP resources—from Compute Engine and Cloud Run to Cloud SQL—ensuring consistency
across Dev, Staging, and Production.
4. Empower DataViz Excellence
You will be the enabler for our Data Analysts. You’ll deploy, optimize, and scale Qlik Sense,
Tableau, or Similars environments on GCP, handling everything from SSO/IAM authentication to high-performance connectivity with our backend data sources.
5. Hybrid Cloud & Legacy Modernization
Complexity is our playground. You will manage critical AWS resources (EC2, S3) and ensure
seamless interoperability with GCP. Simultaneously, you will lead the charge in migrating legacy monolithic systems into modern, microservices-based architectures.
6. Data Engineering Collaboration
Working side-by-side with Data Engineers, you’ll help build and maintain robust pipelines using Cloud Dataflow, Cloud Composer (Airflow), and BigQuery. You ensure that the “plumbing” of our data world is leak-proof and high-performing.
7. Reliability, Security & FinOps
● CI/CD: Design sophisticated pipelines (GitLab CI/GitHub Actions) integrating DevSecOps best practices.
● Observability: Implement deep monitoring using Google Cloud Operations Suite and Grafana.
● FinOps: Be the guardian of our budget, identifying and eliminating cloud waste through
smart optimization.

Who You Are (Requirements)
● Cloud Expert: Proven experience managing complex environments in Google Cloud Platform (GCP) and AWS.
● Data Architect: Deep understanding of Data Lakehouse models and governance.
● Kubernetes Pro: Deep knowledge of GKE, including advanced networking and security policies.
● Terraform Ninja: Ability to write modular, scalable, and secure IaC.
● BI Infrastructure Savvy: Experience deploying and tuning enterprise visualization tools (Tableau, Qlik, or Power BI).
● Automation Mindset: Strong scripting skills (Python, Bash, or Go) and mastery of CI/CD engines.
● Data-Driven: Familiarity with Big Data tools like Airflow, BigQuery, and Dataflow.
● Security-First: Solid understanding of encryption, IAM roles, and network security standards.

Why Join Us?
● Impact: You are not a cog in the machine; you are building the machine.
● Tech Stack: Work with the latest tools in the Cloud and Data world (Lakehouse, GKE).
● Culture: We value technical excellence, proactive problem-solving, and a “fail fast, learn
faster” mentality.
● Growth: We invest in your certifications and continuous learning. Ready to redefine how BI meets SRE? If you are a builder who loves solving the puzzle of high-scale data infrastructure, we want to meet you.

Apply now and let’s build the future of data infrastructure together!

About the role 

We're looking for a Senior Data Architect to join a long-term engagement with an enterprise client. You'll take ownership of an existing, complex Azure data platform and quickly assess, stabilise and modernise it. You'll also design the data layer behind GenAI, RAG and Agentic AI applications. 

This is not a hands-on LLM or AI-agent development role. What matters is that you have designed, in production, how enterprise data and knowledge is ingested, prepared, stored, governed and made available for retrieval and AI consumption. 

What you'll do 

• Own the architecture of an existing Azure data platform: assess it fast, separate tactical fixes from long-term target architecture, and move both forward in parallel. 
• Deliver implementation-ready architecture: detailed documentation, diagrams and recommendations that engineering teams can act on. You'll be expected to show meaningful output within the first 1–2 weeks. 
• Design the data and knowledge layer for RAG and Agentic AI applications: document processing, OCR and text extraction, metadata, chunking, embeddings, and vector search and retrieval. 
• Design scalable, metadata- and configuration-driven ingestion for many databases, APIs and file-based sources. 
• Work with structured and genuinely unstructured data (PDFs, scans, images, emails, free text). 
• Make technology-neutral decisions about when Synapse, Databricks, ADF, Azure SQL or a Lakehouse are justified, and when they only add complexity. 
• Improve performance, scalability, maintainability and data quality, including batch, incremental processing and incremental refresh. 

Must-have 

• Strong, recent experience as an enterprise Azure Data Architect, with clear architecture ownership. 
• At least one real production project (not a POC) where you designed or owned the data architecture behind a RAG, GenAI or Agentic AI application. 
• Azure Cosmos DB, in depth (essential): data modelling, partition-key strategy, indexing, RU/throughput and cost optimisation, scaling, consistency, Change Feed, and knowing when Cosmos DB is the right choice and when it isn't. 
• Azure Synapse Analytics, current and in depth: Pipelines, Spark, Serverless SQL, Dedicated SQL Pools, and how Synapse compares with Azure SQL, ADF, ADLS, Databricks and Fabric. 
• Azure Databricks, Azure SQL MI / SQL Server / T-SQL, Azure Data Lake / Lakehouse architecture, and Azure Functions. 
• Strong data modelling: fact and dimension tables, star schemas, and optimising analytical models. 
• Proven experience taking over a problematic legacy data platform and modernising it. 
• Ability to work independently with incomplete information and turn analysis into concrete deliverables quickly. 
• Fluent English (C1 preferred) and strong communication with both technical teams and business stakeholders. 

What we offer 

• A 12-month full-time contract with a long-term perspective 
• A high-impact architecture role at the intersection of enterprise data platforms and GenAI 
• Fully remote work 
• A short hiring process: an intro call and a 1-hour technical interview 

How to apply 

Please send a CV tailored to this role. The client screens CVs strictly and won't review a generic CV or a list of keywords. For each relevant project, please include: 

• start and end dates, context, and your personal role and decisions 
• the Azure services you used (Cosmos DB, Synapse, Databricks and so on) and how you used them 
• for your RAG/GenAI project: what the application did, what you personally owned, how the data was prepared for retrieval, which storage and vector-search technologies you used, whether Cosmos DB was part of it, and whether it was in production 

Please also include: 

• your expected hourly rate (USD) 
• your English level 
• years of experience 
• your country of residence and your citizenship 
• your availability date 

Shortlisted candidates will be asked to complete a skills matrix. 

N-iX is looking for a Senior Data Engineer to support our client's Data Licensing team, which develops training datasets for frontier AI labs. You will own the onboarding and enrichment of new multimedia content. Extend scalable event-driven and batch-processing systems that produce reliable metadata for curation and customer-facing applications.
 

Our customer is a leading global provider of high-quality licensed images, videos, and music. Our customer helps inspire graphic designers, creative directors, video editors, filmmakers, web developers, and other creative professionals by providing diverse content to businesses, marketing agencies and media organizations around the world. Content creators contribute their work to our client, where end users buy and use it in a diverse array of creative personal and business projects. It's a two-sided marketplace that empowers the world's storytellers.

Responsibilities:

• Partner with the Principal Engineer, senior technical leaders and platform teams to implement and evolve the architectural direction for metadata-processing systems; independently own the design and delivery of assigned components.
• Evaluate and evolve orchestration for enrichment and backfill workloads, either by extending existing ingestion capabilities or integrating with new cloud-native services, while maintaining consistent contracts and lifecycle handling.
• Evolve shared frameworks so data scientists can package, test, validate and run production enrichment workloads with minimal data engineering support.
• Define and maintain metadata contracts, lineage, validation and quality controls.
• Contribute to and apply shared engineering standards, and provide technical guidance through design reviews, documentation and collaboration with data engineering and data science teams.
• Estimate and refine the cost, capacity and completion time of enrichment and backfill workloads.
• Partner with platform teams responsible for operating the underlying cloud infrastructure.
• Own time-sensitive customer and partner deliverables through production release.

Requirements:

• 6+ years of experience in data engineering, backend software engineering or distributed systems, including hands-on ownership of large-scale production data workloads.
• Demonstrated experience designing and operating large-scale ingestion or distributed data-processing systems.
• Strong Python software engineering and distributed processing experience.
• Practical knowledge of high-volume batch and event-driven architectures, including idempotency, replay, backpressure, retries and failure recovery.
• Experience enabling data science or ML workloads to move from prototype to production through reusable frameworks and engineering guardrails.
• Production experience with AWS and GCP, with the ability to design workflows spanning multiple cloud environments.
• Experience managing evolving data contracts, schemas, lineage and data-quality expectations.
• Ability to benchmark distributed workloads and translate results into credible capacity, execution-time and cloud-cost projections.
• Experience setting engineering standards, reviewing system designs and mentoring engineers or data scientists.
• Track record of delivering critical, time-sensitive systems into production.
• English level - upper-intermediate

Nice to have:

• Containerized CPU/GPU batch and ML workloads using managed compute services such as SageMaker, AWS Batch, ECS, Cloud Batch or GKE/EKS.
• Event-driven orchestration and messaging using services such as Step Functions, Lambda, EventBridge, Kinesis, Workflows or Pub/Sub.
• Snowflake, ClickHouse or Apache Iceberg.
• Production-scale image, video or audio processing

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Personalized career growth
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing
• Education reimbursement
• Memorable anniversary presents
• Corporate events and team buildings
• Other location-specific benefits

*not applicable for freelancers

Job Description 

We are seeking an experienced AWS Data Engineer / Data Platform Developer with 3–5 years of experience in data engineering, database development, and cloud-based application development.
The role focuses on designing, developing, and supporting scalable data platforms and cloud solutions using AWS, Snowflake, Amazon Redshift, SQL, and Python, while ensuring strong performance, reliability, security, and compliance.
The ideal candidate should have hands-on experience with:
 - Snowflake and Amazon Redshift
 - Strong SQL skills, including complex queries, stored procedures, UDFs, database objects, and performance optimization
 - Strong Python skills for data processing, scripting, integration, and automation
 - ETL/ELT pipeline development and data integration
 - Data modelling, data warehousing, and data lake concepts
 - Performance tuning, monitoring, troubleshooting, and production support
 - Data quality, reliability, and data integrity practices
 - Technical documentation and system specification development
Nice to have experience includes:
 - Familiarity with AWS services and cloud architecture
 - AWS Lambda
 - Node.js
 - REST API development and integration
 - Security concepts, authentication, authorization, and access management
 - CI/CD practices
 - Cloud-native and microservices-based architectures
 - Infrastructure as Code practices
 - Relevant AWS certifications such as AWS Certified Data Engineer – Associate or AWS Certified Solutions Architect
The successful candidate should demonstrate strong attention to detail, analytical and troubleshooting skills, effective communication with technical and business stakeholders, the ability to work independently and collaboratively, adaptability, ownership, and a continuous learning mindset.

Job Responsibilities

• Design, develop, document, and support scalable data and cloud solutions aligned with architecture standards and business requirements.
• Build and maintain ETL/ELT data pipelines and data processing workflows.
• Develop and support data solutions using Snowflake, Amazon Redshift, SQL, and Python.
• Design and maintain data lakes, data warehouses, databases, and related data platform components.
• Develop and optimize SQL queries, stored procedures, UDFs, views, and other database objects.
• Develop Python-based data processing, integration, and automation solutions.
• Develop and support REST APIs and cloud-based services using AWS technologies in scalable and microservices-oriented architectures.
• Participate in system and production deployment planning and support production releases.
• Monitor, troubleshoot, and optimize data pipelines, databases, APIs, and cloud services to improve performance, reliability, and scalability.
• Identify functionality, performance, or data quality deficiencies and implement improvements to data handling, integrity, conversion, storage, and system behavior.
• Implement data quality, validation, monitoring, and governance mechanisms.
• Ensure solutions comply with applicable security, regulatory, and company standards, including authentication, authorization, access management, and data protection requirements.
• Support compliance with standards and frameworks such as HIPAA, SOC 2, and ISO 27001, where applicable.
• Evaluate existing platform capabilities and determine whether standard functionality or custom development is required to address business needs.
• Create and maintain detailed technical documentation, system specifications, architecture documentation, and operational procedures.
• Collaborate with technical teams, architects, business stakeholders, and external technology vendors.
• Support testing, deployment, maintenance, and continuous improvement of platform components.

Department/Project Description

The project focuses on developing and supporting a scalable cloud-based data platform for healthcare and robotics data.

The platform enables efficient data storage, processing, transformation, and analytics using Snowflake, Amazon Redshift, SQL, Python, and AWS services. It is designed to support reliable data workflows, automation, and integration across multiple data sources while maintaining high standards of performance, security, and scalability.

The team is responsible for developing and optimizing data solutions, automating data processing workflows, monitoring platform performance, troubleshooting data-related issues, and supporting reliable production deployments.

Company Description

Are you a Senior Data Engineer passionate about building scalable, high-performance data platforms and working with modern Lakehouse technologies? Join Sigma Software’s Data Engineering Center of Excellence and contribute to the modernization of an enterprise-scale analytics ecosystem for the retail domain.
We are looking for a Senior specialist with strong Databricks, PySpark, and cloud data engineering expertise to participate in the migration of a large-scale analytical platform from BigQuery to Databricks. You will collaborate with international teams, contribute to architectural decisions, and help shape reliable and scalable data solutions.
We at Sigma Software create opportunities for continuous learning, technology growth, and meaningful engineering impact while working on complex international projects.

CUSTOMER

Our Customer is a leading retail technology company specializing in AI-driven pricing optimization solutions for enterprise retailers. The company helps businesses improve profitability and competitiveness through advanced analytics, automation, and intelligent pricing strategies. Their platform combines business intelligence with sophisticated algorithms to support data-informed pricing decisions at scale for global retail organizations.

PROJECT

The project focuses on the strategic migration of a large-scale analytical platform from a legacy BigQuery ecosystem to a modern Databricks Lakehouse architecture. The platform processes high-volume retail datasets, machine learning workloads, analytics pipelines, and customer-specific business logic.
As part of the modernization initiative, the engineering team is implementing scalable Spark-based processing, Delta Lake architecture, medallion data layers, and modern governance practices. The role offers an opportunity to work with distributed data processing systems, optimize large-scale workloads, and contribute to the evolution of an enterprise-grade data platform.

Key Technologies: Databricks, Apache Spark, PySpark, Delta Lake, Python, SQL, Airflow, GCP, CI/CD, Unity Catalog

Job Description

• Participate in the migration of a large-scale analytical platform from BigQuery to Databricks
• Design and implement scalable Lakehouse architectures using Databricks and Delta Lake
• Analyze existing ETL / ELT workloads and define migration approaches
• Develop and optimize data pipelines processing large volumes of retail and analytical data
• Implement incremental processing strategies and scalable transformation frameworks
• Build and maintain Spark-based data processing solutions using PySpark
• Design and maintain medallion architecture layers including Bronze, Silver, and Gold
• Implement data governance and security best practices using Unity Catalog
• Collaborate with Data Science, Analytics, Product, and Customer Engineering teams
• Participate in architecture discussions and technical solution design
• Develop reusable data platform components and engineering standards
• Conduct code reviews and contribute to platform reliability and maintainability
• Troubleshoot and optimize complex SQL and Spark workloads
• Support production deployments and platform modernization activities

Qualifications

• 5+ years of professional experience as a Data Engineer
• Strong programming skills in Python and advanced SQL
• Hands-on commercial experience with Databricks
• Strong knowledge of Apache Spark, primarily PySpark
• Experience designing and building modern cloud-based data platforms
• Experience developing ETL / ELT pipelines and large-scale data processing solutions
• Hands-on experience with Delta Lake
• Experience with Spark Declarative Pipelines
• Experience with cluster monitoring, metrics analysis, and performance optimization
• Strong understanding of distributed data processing architectures
• Solid understanding of data warehousing concepts and dimensional modeling
• Experience with Airflow or similar orchestration tools
• Experience optimizing complex analytical SQL workloads
• Experience implementing CI / CD practices for data engineering platforms
• Strong troubleshooting and performance optimization skills
• Ability to work collaboratively in cross-functional international teams
• Upper-Intermediate or higher English level

WILL BE A PLUS

• Experience working with GCP cloud services
• Experience with AWS or Azure cloud platforms
• Experience in retail analytics or pricing optimization domains
• Experience supporting machine learning or AI-related data workloads
• Experience with platform modernization and cloud migration initiatives

Additional information

PERSONAL PROFILE

• Strong analytical and problem-solving mindset
• Proactive and ownership-driven approach
• Ability to work independently and collaboratively
• Good communication and stakeholder collaboration skills
• Passion for scalable data engineering and modern data platforms
• Interest in continuous learning and technology innovation

Dataforest is looking for a Data Engineer to join an interesting software development project in the field of Cybersecurity. Our client’s platform offers Identity Insights drives real-time, multi-contextual digital forensics to protect new account opening workflows and expose fake accounts already in a customer database.

You'll be a great fit if you have:

- 3+ years of hands on in data engineering;
- 3+ years of commercial experience with Python;
- Advanced experience with SQL DBs (optimisations, monitoring, etc.);
- Advanced experience with PySpark;
- Solid understanding of ETL principles (architecture/ monitoring/ alerting/search and resolve bottlenecks);
- Familiar with AWS infrastructure (boto3, S3 buckets, etc);
- Experience working with large volumes of data;
- Understanding the principles of medallion architecture.

Would be a plus:

- Cassandra/Scylla
- PostgreSQL
- Pandas
- Experience with Kafka and Redis

What you'll do:

- Maintain data processing architecture using Python and PySpark that interacts with ScyllaDB & PostgreSQL.
- Work with Databricks infrastructure to ensure reliability, scalability, and cost efficiency.
- Proactively identify bottlenecks and suggest technical improvements.

We ensure your growth with:

- Working in a fast-growing company;
- Great networking opportunities with international clients, challenging tasks;
- Personal and professional development opportunities;
- Competitive salary fixed in USD;
- Paid vacation and sick leaves;
- Flexible work schedule;
- Friendly working environment with minimal hierarchy;
- Team building activities, corporate events.

Back End Data Engineer

Company: NeuCurrent

Location: Remote / Hybrid

NeuCurrent is an AI powered CRM and omnichannel marketing platform for retailers. NeuCurrent empowers brands to connect with their customers more intelligently – through real-time personalization across email, SMS, WhatsApp, and push. 

We are looking for a motivated, bright and independent Data and Back End Engineer with at least 3 years experience who is willing to take broad responsibilities across product development and wants to progress quickly to a senior role within a company. 

Why you should apply:

This is a challenging job. Building something new is not easy.

We are building a new product.

It takes time to grow and get results.

Working for a start up you are going through lots of challenges.

We often multi-task as we grow our company.

BUT If you want to be part of the fast growing business and team, generate your own ideas, and see the direct impact of your work on the business of the users, you should consider applying for this job. 

Apply if you want to try yourself in something new!

About company:

At NeuCurrent we are building a disruptive technology for retail which enables all retailers to implement data driven customer retention without expensive data analytics and marketing resources. We are actively expanding in Europe at present. 

Our values:

We care for our people and customers

We develop innovation that generates real benefit

We are not afraid to take risks and see failure as a learning opportunity

We are determined and always look for solutions to move forward

Responsibilities and Tech Stacks:

Data Processing

• Maintain and improve data collection pipelines, which are the backbone of our product
• Create new integrations
• Maintain DWH architecture
• Support and improve Kubernetes infrastructure on GCP
• Developer infrastructure for ML pipelines

Tech stack:

Redis, Argo

Python 3, Pandas, Numpy, Sqlalchemy

PostgreSQL, Google BigQuery

Docker, Kubernetes, Google Cloud Platform

Flask

Backend Development

• Support and further develop the product backend API
• Developer infrastructure for ML pipelines

Tech stack:

Python, Flask

PostgreSQL

Docker, Kubernetes, Google Cloud Platform

Other Requirements:

Work closely with the founders on product development and improvement

Actively contribute ideas to product innovation including AI implementation

Customer focused 

An excellent / good working English is a must as you will deal with the English speaking customers

Personal Qualities:

A self-starter who has the experience and confidence to act independently

Have a positive mindset and attitude to solve problems and represent NeuCurrent with our customers

A strong team player who is keen to learn and share her/his knowledge

Enthusiasm and desire to progress professionally to a managerial role quickly

Be structured and a good communicator 

What we offer:

You will have a lot of freedom in your field to create direct impact on our clients and NeuCurrent business and product development

We have a young, highly motivated, and talented team

You will become an integral part of new product development

You will have plenty of opportunities to make your own decisions

You will have unlimited opportunities to progress to managerial roles in our fast growing company

Options / shares are available for a proven candidate

Are you passionate about safeguarding countries, societies, and businesses from hybrid threats?

Look no further! Our rapidly scaling company is committed to this crucial mission, collaborating with some of the world’s most influential organizations, including NATO and the EU.

If you’re ready to join a high-impact team shaping the future of intelligence and security, we invite you to become part of our journey. Join us and help defend the digital world with purpose and innovation!

Role Overview

We are looking for a Senior Data Engineer to work on the data architecture behind our intelligence platform, across both our production data infrastructure and our work with multimodal data.

We operate on text, social media, geospatial, and time-series data at scale. A large part of the engineering challenge is in how that data is represented and connected: the same real-world entity can be modelled on a map, as a node in a graph, and as a pattern over time, and the value comes from being able to query and reason across those views together.

You'll bring hands-on experience with geospatial data, knowledge graphs, or entity-relationship systems across multiple data types. We care more about how you reason through architectural and technology choices than about any specific tool.

This is a hands-on, senior role with real ownership over architecture, execution, and technical direction.
 

Key Responsibilities

Multimodal Data & Entity Modelling

• Extract entities and relationships from text, social media, geospatial, and time-series data, and model them across map, graph, and temporal representations.
• Evolve our knowledge graph layer and make connected data usable for analytics, visualisation, and AI agents.
• Evaluate and prototype technologies for geospatial and graph workloads at scale

Core Data Platform

• Migrate production workloads from PostgreSQL to StarRocks.
• Improve ingestion and processing performance, and redesign product storage architecture for billion-row scale.
• Own the infrastructure your pipelines run on, including deployment, performance, and cost.

Requirements

• Hands-on experience building data systems for geospatial data, graph data, or knowledge graphs, ideally combining several data types.
• Experience with GIS tooling (e.g. PostGIS), graph databases (e.g. Neo4j, TigerGraph), or modelling graph data on relational or columnar storage.
• Strong architectural thinking and the ability to justify technology choices, run experiments, and change course based on results.
• Proven experience moving and processing data at large scale, with resumable, idempotent, and memory-safe pipelines.
• Experience working with data infrastructure beneath the managed layer, not only inside a ready-made SaaS platform.
• Ability to read query plans and change physical data layout to fix performance issues.
• Experience building and shipping production backend services in Python.
• Comfort owning infrastructure, including Kubernetes, Helm, Terraform, or similar tools.
• Fluent English.

Nice to Have

• Production experience with a columnar or MPP engine such as StarRocks, Doris, ClickHouse, BigQuery, or Snowflake, including partitioning, distribution, and indexing.
• Experience with Elasticsearch or OpenSearch at scale, including mappings, ILM, tiering, and reindexing.
• Experience with a long-running migration involving dual-write, reconciliation, and cutover.
• Knowledge of PostgreSQL internals.
• Experience with event-driven ingestion using Kafka or RabbitMQ.
• Experience with observability tools (Prometheus, Grafana, Loki), lakehouse formats (Parquet, Iceberg), and orchestrators such as Airflow.
• Background in threat intelligence, OSINT, geospatial intelligence, or defence.

We Offer

• Opportunity to contribute to a mission-driven startup and support prestigious clients, including governments worldwide, enterprise clients, and leading NGOs in tackling global security challenges.
• Autonomy and freedom to drive experiments and bring your own ideas to life.
• Flexibility of fully remote work.
• Flexible, unlimited time-off policy.

ZentixSoft is looking for two Senior Data Ingestion Engineers! 🚀

Format: Direct Contract Engagement (1-Year Contract, 100% Full-Time Remote). 
 

💡 Why ZentixSoft Partner Network?
 💸 Transparent Compensation: Direct payroll arrangement with $40 USD/hour base compensation. 
⚖️ Work-Life Balance: Sustainable engineering workflows with predictable long-term deliverables. 
🦾 Trust & Transparency: Zero micromanagement and full autonomy over your technical pipeline architecture. 
🎁 Culture & Growth: Long-term project stability within large-scale financial and insurance data domains.
 

🧩 Responsibilities:

• Pipeline Design & Engineering: Design and deploy scalable data ingestion pipelines for high-volume structured/unstructured documents (PDFs, scans, emails, Word, Excel, PowerPoint).
• Document Extraction & OCR: Implement OCR and document processing workflows using AWS Textract (or equivalent) for text extraction, cleaning, normalization, and metadata tagging.
• RAG & Vector Storage Prep: Execute semantic chunking, metadata extraction, vector storage schema design, and retrieval mechanism preparation for downstream AI models.
• Integrations & Connectors: Build robust connectors with enterprise sources, including SharePoint, email servers, and public cloud repositories.
• Validation & Observability: Implement automated error monitoring, OCR extraction validation, and CI/CD automated testing using AWS Step Functions and CloudWatch.
   

🎓 Our Perfect Match (Requirements):

• Senior Data Engineering: 7+ years of commercial Data Engineering experience, with strong proficiency in Python and SQL.
• AWS Expertise: 5+ years of hands-on experience in AWS environments, specifically AWS S3, Step Functions, CloudWatch, and public cloud data processing.
• Unstructured Data & OCR: Proven background in building document extraction pipelines (handling PDFs, scanned images, emails, Office documents) and utilizing OCR technology (AWS Textract or similar).
• Source Connectors & Pipeline Operations: Practical experience integrating sources like SharePoint and email, performing text normalization, metadata tagging, and maintaining CI/CD/Git testing standards.
• EU Residency & Location: Candidates MUST reside in an EU member country (with active legal residency; citizenship can be non-EU/global).
• Language & Communication: C1 Advanced English (verbal and written) for direct technical collaboration.
   

➕ Good to Have:

• Experience with Vector Databases, RAG architectures, semantic chunking, and vector retrieval mechanisms.
• Background in Insurance or Financial Services industries handling enterprise security standards.
• Exposure to Azure or Databricks platform components.
   

📍 Project Details:

⏳ Duration: 1 Year (Full-time contract).

🚀 Target Start: End of October.

📍 Location: 100% Remote (Must be physically located in an EU member state with valid residency).

About the role:

Our digital business depends on reliable financial and marketing data. We are looking for a highly reliable, detail-oriented Analytics Engineer who will take ownership of the core data architecture behind our Finance and Marketing operations.

In this role, you will be responsible for the daily data operations that support our Payment Service Providers (PSP), Tax reporting, Controlling, Finance reporting, and marketing campaign cost tracking.

You will ensure that our Pentaho ETL pipelines run smoothly through Jenkins, our MariaDB databases remain performant, and our Qlik Sense dashboards consistently deliver accurate, business-critical numbers.

This is a hands-on, high-impact role where precision, stability, and reliability are essential.

What you’ll do:

ETL & Data Architecture

• Own, maintain, and optimize core ETL pipelines built with Pentaho Kettle 9.3 Community.
• Ensure reliable and accurate data transformations across critical business processes.
• Monitor and improve the stability and performance of existing data flows.

Database & Automation

• Work with complex data transformations and queries in MariaDB.
• Optimize database performance and ensure data processes run efficiently.
• Manage and monitor automated jobs orchestrated through Jenkins.

Finance & Marketing Data

• Work daily with transactional and financial data.
• Support data processes related to Payment Service Providers (PSP), Tax reporting, and Controlling reconciliations.
• Manage the ingestion and processing of marketing campaign costs and ad spend from different advertising platforms.

BI & Reporting

• Design, develop, and maintain critical Qlik Sense dashboards and reports.
• Ensure stakeholders receive accurate and consistent financial and operational data.
• Analyze data and translate it into logical conclusions and actionable insights.

Production Support & Troubleshooting

• Investigate and resolve data issues in a live production environment.
• Identify root causes and implement reliable fixes.

• Ensure that daily reporting and month-end financial closing processes are not delayed due to data issues.

   

What we’re looking for:

Experience

• 4+ years of experience working with ETL processes, databases, and BI solutions.

Core technical expertise

• Strong hands-on experience with Pentaho Kettle or similar legacy ETL tools such as Talend or Informatica.
• Solid experience with relational databases: MariaDB, MySQL, PostgreSQL or similar.
• Experience with Jenkins and automated job orchestration.
• Advanced SQL skills.
• Strong understanding of production data processes and error handling.

BI

• Strong experience with Qlik Sense.
• Experience with Power BI or Tableau is also relevant.

Domain experience

• Practical experience working with Finance data, including:
  • Payment Service Providers (PSP)
  • Tax reporting
  • Controlling and financial reconciliations
• Experience working with Marketing data, particularly campaign cost and advertising spend tracking.
   

Soft skills

• Comfortable working under time constraints, especially during month-end financial closing.
• Strong communication and collaboration skills.
• A structured, methodical, and diagnostic approach to problem-solving.
• High attention to detail and a strong sense of ownership.
   

Tech stack

Pentaho Kettle 9.3 Community · MariaDB · Jenkins · Qlik Sense · SQL

Why Join Us? 

    🌍 Remote-first & flexible work

• Fully remote working model.
• Access to an office if there is one available in your location.
• Monday–Friday, 40-hour working week.

• Flexible working hours: schedule options between 08:00 and 19:00, including a 1-hour lunch break (the only requirement is a minimum of 5 hours of overlap with the team)

  🏖️ Vacation & sick leave

• 20 paid working days of vacation during the first year.
• 10 paid sick days per year (for childcare would also be covered), compensated at 100% of salary.

• Public holidays are days off by default.

  ❤️ Health & family benefits

• Medical insurance or medical insurance compensation.
• Company gifts for:
  • Birthday
  • Wedding
  • Birth of a child
• Paid lunches at the office.
• Family days etc.
• 📑 FOP / Individual Entrepreneur Support if needed
  🔄 Paid Bench period between projects
  🤝 No time-tracking software — transparent and trust-based time reporting.
  🎁 Referral Program

About Zoral

Zoral is an IT product and professional services company serving banks, insurers, wealth managers and fintechs. Alongside its flagship product, Zoral fOS (Financial Operating System), Zoral delivers data, analytics and AI programmes for regulated financial-services firms in Europe, the UK, the US and Asia.

About the role

We are looking for a Data Architect / Senior Data Engineer to define the target architecture and actively design, develop and deliver enterprise data platforms on Azure Databricks for regulated financial-services clients. Typical platforms integrate tens of source systems and millions of customer and policy records, support several hundred regulatory, management and operational reports, and are delivered by growing multidisciplinary teams of data engineers, architects, analysts and client specialists.

This is a hands-on architecture and engineering role within a collaborative delivery team. You will assess the client’s existing data landscape, help refine the target architecture, establish modelling and engineering standards, and act as a senior technical authority throughout delivery. You will work closely with other architects and engineers, providing direction, reviewing designs and supporting implementation rather than operating as a standalone architect.

At the same time, you will remain close to the engineering work: developing transformation logic and data models, profiling and cleansing data, implementing data-quality controls, and contributing to production engineering where your experience has the greatest impact. The successful candidate should be comfortable moving between architecture, technical leadership and hands-on implementation while enabling the wider team to deliver effectively.

Responsibilities

• Assess existing data lakes, warehouses, pipelines, models, jobs, notebooks and tooling; work with client architects and the wider delivery team to confirm or refine the target architecture and document decisions, options, trade-offs and rationale.
• Design the layered data architecture (landing, bronze, silver and gold) and guide its implementation across the engineering team, covering ingestion, standardisation, transformation, integration, conformance, data products and reporting marts.
• Develop and review production-grade data transformations using SQL and Python/PySpark, including reusable and metadata-driven pipelines that are idempotent, re-runnable, testable and observable.
• Define data-modelling standards and contribute to the development of integrated and consumption-layer models, including Data Vault 2.0, Kimball dimensional models, slowly changing dimensions, historical and as-at reporting, and model-generation approaches.
• Define the data-quality and data-cleansing approach together with the engineering team: establish profiling and baseline measures; define rule catalogues, quality gates and scorecards; design quarantine, remediation, reconciliation, audit, balance and control processes; and ensure these are implemented consistently.
• Guide and contribute to data cleansing, standardisation, de-duplication, customer matching and entity-resolution approaches, including match rules, survivorship and persistent identifiers.
• Design how the new platform is separated from and coexists with legacy platforms, including catalogue and environment layout, access boundaries, dependencies, migration, reconciliation and decommissioning.
• Define practical standards for layering, naming, ownership, classification, lineage, security, masking, resilience and retention, and work with engineers to ensure these standards are applied consistently in delivery.
• Evaluate tools for ingestion, transformation, orchestration, data quality, catalogue, lineage, master data and reporting, working with other technical stakeholders to present recommendations and trade-offs.
• Review engineering designs and code, contribute directly to complex or critical implementation areas, troubleshoot difficult data and pipeline issues, and support automated testing and CI/CD practices.
• Provide technical leadership to engineers and client staff through design reviews, pairing, mentoring and collaborative problem solving.
• Produce architecture documents, decision records, technical standards, runbooks and handover material in collaboration with the wider project team.

Requirements

• 8+ years in data architecture and/or senior data engineering, with substantial hands-on delivery responsibility, including at least three years designing and engineering Databricks platforms in production.
• Demonstrated ability to operate as both an architect and a senior engineer within a delivery team: define target-state designs and standards, guide other engineers, and implement or materially contribute to pipelines, transformations and data models.
• Experience providing technical leadership within multidisciplinary data teams, including design reviews, mentoring, engineering standards and collaborative delivery.
• Expert SQL and strong Python/PySpark skills, with experience developing, testing, optimising and supporting production data transformations.
• Hands-on knowledge of Azure Databricks, including Unity Catalog governance (grants, row filters, column masks, tags and lineage), Delta Lake, Lakeflow Declarative Pipelines, Lakeflow Jobs and Lakeflow Connect.
• Proven experience designing and developing layered lakehouse or data-warehouse architectures using Data Vault 2.0 and Kimball dimensional modelling, including slowly changing history and as-at data.
• Experience defining and implementing data-quality and cleansing strategies, including profiling, rule design, quality gates, monitoring, scorecards, quarantine, remediation and reconciliation controls.
• Experience with master data management, customer matching, de-duplication or entity resolution.
• Experience migrating data, pipelines and models from legacy warehouses or lakes, including coexistence, reconciliation, parallel running and decommissioning.
• Practical experience with Git, code review, automated testing, CI/CD and production support for data pipelines.
• Ability to write clear architecture and engineering documentation and present technical decisions to senior stakeholders.
• Excellent written and spoken English; willingness to work on site at client premises, including in the UK, for periods of several weeks.

• Provide governed data, enterprise knowledge, document intelligence, knowledge graph, and retrieval capabilities that ground AI agents in trusted evidence

   

• Responsibilities:

  Design RAG and enterprise-search architectures.
  Build ingestion, transformation, enrichment, document extraction, embedding, and indexing pipelines in Databricks.
  Create and maintain Azure AI Search, vector search, and access-controlled retrieval indexes.
  Implement Azure Document Intelligence pipelines for extracting structured data from PDFs, forms, engineering documents, reports, and scanned content.
  Design and maintain knowledge graph models to represent entities, relationships, metadata, lineage, and domain context for agent reasoning and search enrichment.
  Implement metadata, filtering, citations, lineage, data freshness, and security-trimmed access controls.
  Measure and optimize retrieval relevance and knowledge quality.
  Support structured data products used by agents and deterministic services.

   

• Mandatory Skills Description:

  Bachelor's degree in Computer Science, Data Engineering, Data Science, Information Systems, or a related field.
  5+ years of data engineering, search engineering, analytics engineering, or knowledge-platform experience.
  Strong SQL skills and experience building production data pipelines in Databricks.
  Experience with data quality, metadata, lineage, and governance practices.
  Understanding of structured and unstructured enterprise information retrieval, including document extraction and knowledge graph patterns.
  Hands-on Azure AI Search, Databricks, RAG, or vector-search experience.
  Experience with Azure Document Intelligence, document processing, chunking, embeddings, ranking, metadata, and citation design.
  Hands-on knowledge graph experience using Neo4j, RDF, graph databases, semantic modeling, ontology design, or entity-relationship mapping.

   

• Nice-to-Have Skills Description:

  Azure Data Engineer or Databricks certification.

   

• Languages:
  • English: B2 Upper Intermediate