Solution Architect (Databricks/Azure Data Platform)

Summary

We are seeking a highly experienced Solution Architect with 15+ years of experience in designing and delivering large-scale, cloud-native data platforms. The ideal candidate will have deep expertise in Azure Databricks and strong capability in architecting highly scalable, compute-intensive solutions processing datasets exceeding 100 million records.

Job Description

The Solution Architect will be responsible for designing end-to-end cloud-native data platforms with a strong focus on big data processing, analytics, and distributed computing on Microsoft Azure.

This role requires deep hands-on expertise in Azure Databricks and modern data engineering practices, combined with the ability to integrate .NET-based microservices and design robust, event-driven architectures.

The architect will work closely with engineering and business teams to define scalable, resilient, and cost-optimized enterprise solutions that support high-volume data processing and real-time or batch analytics workloads.

Responsibilities

• Design and architect large-scale data processing solutions using Azure Databricks
• Define scalable data pipelines capable of processing 100M+ records and high-throughput workloads
• Architect cloud-native data platforms on Microsoft Azure with focus on performance and cost optimization
• Design integration patterns between data platforms and .NET-based microservices
• Work with event-driven and asynchronous architectures using Azure messaging services
• Ensure proper use of Azure PaaS services, including Cosmos DB, Service Bus, Key Vault, and Storage
• Define data architecture standards, best practices, and governance principles
• Collaborate with engineering teams to implement robust ETL/ELT pipelines and streaming solutions
• Ensure system scalability, resilience, fault tolerance, and security
• Support architectural decision-making and provide guidance on trade-offs for data platform design

Technologies, Tools, and Methodologies

Data Platform (Primary - 70%)

• Azure Databricks (Apache Spark, Delta Lake)
• Big Data processing and distributed computing
• ETL / ELT pipelines
• Batch and streaming data architectures
• Data modeling and transformation at scale

Cloud and Azure Services

• Azure Cosmos DB
• Azure Service Bus
• Azure Key Vault
• Azure Storage (Blob, Data Lake)
• Event-driven and asynchronous architectures

Backend / Integration (20%)

• .NET-based microservices
• REST / messaging-based integrations
• MongoDB
• Microsoft SQL Server

Architecture (10%)

• Cloud-native architecture design
• Scalability, resilience, and cost optimization principles
• Distributed system design
• Architectural and integration patterns

Professional Skills

• Fluent English communication skills, both verbal and written
• Strong ability to design and explain complex distributed data systems
• Experience working with large-scale enterprise environments
• Strong analytical and problem-solving mindset
• Ability to collaborate with cross-functional engineering teams
• Strong ownership and ability to drive architectural decisions

Certifications and Additional Experience (Desirable)

• Microsoft Azure certifications (Data Engineer / Architect tracks)
• Experience with Apache Spark ecosystem
• Experience in large-scale data platform modernization projects
• Experience in real-time data processing systems
• Knowledge of DevOps and CI/CD for data pipelines