Senior Data Engineer, Databricks

We are looking for a senior data engineer with deep Databricks experience to own the data platform behind one of our largest enterprise accounts and lead its migration from AWS to Databricks.

About the role

Data infrastructure is the foundation of everything we deliver. Our models, dashboards, and the decisions our clients make on top of them are only as reliable as the pipelines underneath, which is why we put the platform first.

You will join a small senior team working inside the client's environment. Through the rest of 2026 you will take ownership of the existing pipelines and the architecture around them. From 2027 you will lead the migration to Databricks: designing the target architecture, planning the cutover, and doing the implementation. This is a hands-on engineering role with real architectural authority. You decide how the platform gets built, and then you build it.

You will be the person the team turns to on anything Databricks, working alongside senior machine learning engineers and data scientists who depend on what you deliver.

What you will do:

• Design and build the target Databricks architecture, and plan the migration from the current AWS stack
• Build and maintain production data pipelines: ingestion, transformation, and orchestration
• Own data quality, including the checks and alerts that catch problems before the client does
• Design the data models the analytics and machine learning work sits on
• Support the pipelines you build: monitoring, alerting, and the on-call reality of production data
• Work directly with the client's engineers, and explain technical decisions to people who are not engineers

Requirements:

• Five or more years in data engineering or software development
• At least three years of production Databricks work
• Deep experience with Spark, Delta Lake, and Unity Catalog
• Has planned and executed at least one migration onto Databricks from another platform, and can walk through the challenges
• Expert Python: clean modular code, comfortable with optimization and profiling
• Advanced SQL and relational database work: raw SQL, data modeling, scripted migrations
• Systems design: how storage and compute fit together, end to end pipelines, containerized applications
• Cloud operations on AWS or equivalent: networking, identity and access management, monitoring and alerting, and the domain knowledge for why things break
• Comfortable working inside a client's environment, with their data and their constraints
• Strong written English, and spoken English good enough to present your work
• Available for some overlap with United States working hours

Nice to have:

• Databricks certification
• Deep AWS experience specifically, since that is the platform we are migrating from
• dbt, Airflow, or a similar transformation and orchestration framework
• Machine learning pipeline experience, since models run on top of what you build
• Background in utilities, energy, telecoms, or another regulated industry
• Experience being the senior engineer a client relies on directly

You are:

• Careful with data quality, and you ask when something looks wrong instead of guessing
• A self starter who can turn a general requirement into a specific estimate and a deliverable
• Someone who documents. We write things down, and every number we deliver carries its source
• Comfortable in a small remote team where nobody is checking your hours

About Bennett Data Science

We have spent ten years building predictive analytics and data science for enterprise clients, from some of the largest companies in the world to startups in Silicon Valley. Our engineers sit inside the client's systems and build critical systems. The work ships to production and lands at the bottom line.

What we offer:

• Competitive compensation, reviewed as you grow
• Work from anywhere, with a flexible schedule
• A small senior team, so your work is visible and your decisions are important
• Direct exposure to enterprise clients and the problems they actually have
• Mentorship from senior staff, and the space to grow and mentor others