Senior Data Engineer

The Client is the UK’s leading fee-free mortgage broker. They are seeking experienced Data Engineer to join client’s data team. This role focuses heavily on core data engineering, architecture, and pipeline delivery within established frameworks rather than the presentation layer. The ideal candidate will be responsible for designing, building and optimizing scalable ingestion-to-consumption patterns and modern data warehousing principles.

Requirements

Must-Have Skills:

• Python / PySpark for Data Engineering: Strong experience writing PySpark notebooks in Fabric, building data transformation/cleansing logic, and notebook-based pipeline development.
• Medallion Architecture: Proven expertise in designing and implementing Bronze/Silver/Gold layered ingestion-to-consumption patterns, incremental loading, and transformation logic.
• Dimensional Modelling: Strong knowledge of Kimball-style fact/dimension design, SCD (Slowly Changing Dimension) handling, and Gold-layer conformance.
• T-SQL & DDL Proficiency: Expertise in warehouse schema design, writing stored procedures, and performance tuning.
• Pipeline Development & Orchestration: Practical experience with Dataflows Gen2, Fabric pipelines, notebook orchestration, scheduling, and dependency management.
• Good written and verbal English communication skills.

Nice-to-Have Skills:

• Governance & Security Implementation: Exposure to Microsoft Purview integration, Entra ID mapping, and workspace/access control architecture.
• Power BI & Semantic Modelling: Basic knowledge of DAX, semantic model design, and row/column/object-level security (RLS/CLS/OLS).

Job responsibilities

• Data Pipeline & Framework Development: Design, build, and orchestrate robust end-to-end data pipelines using Dataflows Gen2, PySpark notebooks and native orchestration tools.
• Architecture & Storage Layer Management: Implement and maintain Medallion Architecture (Bronze, Silver, Gold layers), ensuring proper incremental loading strategies and layer-specific transformation logic.
• Data Warehousing & Dimensional Modelling: Build Kimball-style fact and dimension models, manage Slowly Changing Dimensions (SCDs), and ensure Gold-layer data conformance for analytics consumption.
• Data Transformation & Optimization: Write clean, modular Python/PySpark code and T-SQL/DDL logic for data cleansing, schema design, stored procedures, and query performance tuning.