Lead Data Engineer (offline) $6000-7000

Skills
• 8+ year experience with ETL and software development
• 2+ years managed small teams
• 8+ years SQL professional experience
• Structured programming language experience: python, Java, .Net, etc….
• Azure experience / familiarity, specifically Azure Data Factory
• Data pipeline/ETL experience
• Analytic, critical thinking skills
• Familiar working within an Agile and Scrum project environment
• Nice to Haves
o Spark experience
o Azure Synapse experience
o Python experience

Duties
Our stack is an ELT process inside of Azure Synapse. Day to day we work on SQL queries in spark
notebooks and manage the metadata controlling the data pipeline flow.

• This is a hands-on role that will also require assisting junior team members and interfacing with
the client to understand work items, prioritize work items, and resolve problems. So
communication is key!
• Responsible for creating dimension and fact tables in sql using definitions provided by the
business
• Ingest new data source objects into the ETL framework using metadata. Requires the ability to
understand different schemas, adapt existing data processing steps to handle the new data
object using SQL in a spark notebook environment
• Troubleshooting data discrepancies in existing dimension and fact tables by tracing the data
flow through various SQL transformations
• Maintaining metadata that defines the ETL processing steps and troubleshooting metadata
inconsistencies
• Manage team of 4 data engineers
• Daily interface with client data engineers to review work progress, identify and resolve blockers