Lead Data Engineer

Pine Software Technology Ltd is a software company specializing in trading technology solutions for regulated brokers and trading firms. We build SaaS products that support electronic trading, brokerage operations, risk management, and financial data analysis.
‍

We are looking for a Lead Data Engineer for the Market Data Platform.
‍

Responsibilities

• Lead the design and development of the Data Platform, focusing on data pipelines, storage, and processing layers.
• Own architectural decisions around data storage, data modeling, and data access patterns.
• Design and build scalable data pipelines for ingestion, processing, and transformation (batch and streaming).
• Develop and maintain high-performance Python-based data processing components and services.
• Work with PostgreSQL and other storage systems to ensure correct schema design, partitioning, and performance optimization.
• Design and maintain streaming data pipelines for real-time market data processing (Kafka or similar).
• Implement data orchestration workflows (Airflow, Dagster, or similar).
• Ensure data reliability and quality: validation, monitoring, observability, and alerting.
• Optimize data systems for high throughput and low latency.
• Establish and drive best practices in data engineering, code quality, testing, and CI/CD.
• Define and maintain data contracts, schemas, and governance practices across teams.
• Mentor engineers and drive the technical growth of the data engineering team.
• Collaborate with platform, backend, and product teams to align data domains and interfaces.
  ‍

Job requirements

• 6+ years of professional experience in Data Engineering (with Python) and 1+ year in a Lead role.
• Strong experience building data-intensive systems and data platforms.
• Advanced proficiency in Python as a primary language for data engineering (data processing, pipelines, internal tooling).
• Strong experience with ETL/ELT pipelines (batch and streaming).
• Experience with distributed data processing frameworks (Apache Spark, Flink, or similar).
• Experience with streaming platforms and event-driven data pipelines (Apache Kafka or similar).
• Hands-on experience with data orchestration tools (Airflow, Dagster, or similar).
•  Strong practical experience with PostgreSQL and other data storage systems: schema design, query optimization, indexing, partitioning
•  Experience working with analytical databases/data warehouses/data lakes.
•  Strong knowledge of data modeling (OLTP, OLAP, dimensional modeling, time-series).
•  Experience designing and building high-throughput data ingestion and processing systems.
•  Experience implementing data quality, validation, and observability practices.
•  Experience with containerization and deployment practices (Docker, CI/CD).
•  Strong engineering discipline: clean architecture, testing, maintainability.
•  English level: B1-B2
  ‍

Nice to Have

• ClickHouse or other analytical / time-series databases, NoSQL solutions
• Kubernetes
• Experience with lakehouse technologies (Delta Lake, Iceberg, Hudi)
• Experience with real-time analytics systems
• Domain experience in trading/market data systems
• Familiarity with event sourcing
• Patterns: CQRS, Materialized Views, Snapshotting (in data context)
• Experience with cloud services (AWS S3, IAM, etc.)
  ‍

What We Offer

• Care from Day One — medical insurance from your first day, including dental care, massage, and professional psychological support.
• Work-Life Balance — 25 days of paid vacation + 30 days of paid sick leave.
• Investment in Your Energy — partial reimbursement for sports and fitness activities.
• Growth & Learning — coverage for English or Ukrainian language courses and a dedicated professional development budget.
• Knowledge Library — office books and access to the Kuka online library.
• Island Relaxation — up to 14 days per year at the company villa in Cyprus.
  ‍

 Please note, that only shortlisted candidates will be contacted. Thank you for considering Pine as your next career move!