Data Architect (Azure, Databricks)

Project overview

The project focuses on building a governed conversational BI solution on Databricks using Genie. It includes designing scalable data architecture, implementing strong data governance with Unity Catalog, and enabling reliable SQL and AI-driven data consumption through well-defined semantic models.

Responsibilities

• Define and oversee the governed conversational BI architecture on Databricks using Genie
• Ensure governance from day zero, including compliance with Unity Catalog policies for all data assets
• Implement RBAC, row-level security (RLS), lineage, and audit logging
• Design and validate the Medallion architecture (Bronze / Silver / Gold layers)
• Define CI/CD processes in Azure Pipelines for semantic model management
• Oversee the setup of two Genie Spaces and ensure guardrail responses for out-of-scope questions
• Drive quality gates, benchmark cycles, and SME validation processes
• Build and maintain semantic models and knowledge structures
• Define data models for SQL and AI consumption

Requirements

• Experience with Databricks (on Azure), including Lakeflow pipelines
• Strong knowledge of Unity Catalog, including data governance, RBAC, and lineage
• Experience working with Genie (Databricks AI-powered analytics interface)
• Experience with CI/CD processes using Azure Pipelines for semantic model version control
• Strong SQL experience, including SQL Warehouse, Delta Lake, and Gold schema management
• Knowledge of PII masking, encryption, and audit/compliance frameworks
• Experience with template-based deterministic SQL design