Databricks Architect Offline

Job Summary:

We are seeking a highly skilled Databricks Architect to design, implement, and optimize data solutions leveraging Databricks and the Lakehouse architecture. This role will be responsible for architecting scalable data pipelines, ensuring efficient big data processing, and driving best practices for data engineering, analytics, and machine learning.

The ideal candidate has deep expertise in Apache Spark, Delta Lake, and cloud platforms (AWS, Azure, or GCP), along with a strong understanding of data warehousing, ETL, and real-time streaming architectures.

Key Responsibilities:

• Architect and optimize scalable, high-performance Databricks-based data solutions.
• Design Lakehouse architectures that unify data lakes and data warehouses using Delta Lake.
• Develop ETL/ELT pipelines using Apache Spark, Python, Scala, and SQL.
• Implement data governance, security, and compliance best practices.
• Lead cloud migration efforts from traditional data warehouses to Databricks on AWS/Azure/GCP.
• Design real-time streaming architectures leveraging Structured Streaming, Kafka, and Kinesis.
• Optimize query performance and ensure cost-efficient cloud compute usage.
• Work closely with data engineers, data scientists, and BI teams to support analytics and AI/ML workloads.
• Implement MLFlow for machine learning model lifecycle management.

• Drive automation and CI/CD pipelines for data workflows and infrastructure as code (IaC).

   

Required Qualifications:

• 5+ years of experience in big data architecture, data engineering, or analytics.
• Expertise in Databricks, Apache Spark, and Delta Lake.
• Strong programming skills in Python and/or Scala.
• Hands-on experience with SQL-based transformations and query optimization.
• Deep understanding of ETL/ELT architectures and data modeling.
• Experience with cloud platforms (AWS, Azure, or GCP) and serverless compute.
• Experience implementing data security, governance, and access control (Unity Catalog, IAM, RBAC).
• Familiarity with MLFlow, machine learning pipelines, and AI/ML workloads.
• Strong problem-solving and communication skills, with the ability to lead technical teams.
•  

Preferred Qualifications:

• Databricks certifications (Databricks Certified Data Engineer, Databricks Certified Associate Architect, etc.).
• Experience with orchestration tools (Airflow, Azure Data Factory, AWS Glue).
• Proficiency in Terraform or CloudFormation for infrastructure as code.
• Background in data warehousing (Snowflake, Redshift, BigQuery).
•  

Why Join Us?

• Work on cutting-edge big data and AI projects.
• Opportunity to lead Databricks implementations for enterprise-scale solutions.
• Competitive salary, benefits, and career growth opportunities.
• Collaborative and innovative work environment.