Senior Data Engineer (IRC292480) $$$$

Job Description

• Proven experience designing, building, and operating data-oriented solutions in a high-volume, transactional, global, industry. Experience with advertising technology (AdTech) is highly desired.
• Proven experience and a passion for developing and operating data-oriented solutions using Python, Airflow/Composer, Kafka, Snowflake, BigQuery, and a mix of data platforms such as Spark, AWS Athena, Postgres and Redis. Excellent SQL development, query optimization and data pipeline development skills required. Strong experience using public cloud platforms including AWS and GCP is required; experience with docker and Kubernetes strongly preferred. Experience in the use of multiple programming languages is highly desirable. Experience using Functional programming is ideal.
• Proven experience in developing data structures and algorithms. Experience supporting ML/AI highly desirable.
• Proven experience contributing to product strategy development with Product Management teams. Experience in SaaS product engineering and operations highly desirable.
• Proven experience in data engineering and testing practices and implementing these in teams, with a willingness to share, partner and coach engineers, product people, and operations through pragmatic and effective development and delivery practices.
• Proven experience shaping and evolving the principles, practices, and tooling supporting agile, testing/QA, DevSecOps, automation and SRE in a data-focussed team. Experience in Trunk Based Development, XP, & implementing CI/CD as used in data teams is highly desirable.
• A focus on continuous learning and improving, both technically and professionally, in your industry, for you and your teams.
• Demonstrated resilience, with experience working in ambiguous situations.
• Strong English, excellent influencing and communication skills, and excellent documentation skills.

Job Responsibilities

• Work with product, product engineering, data engineering, and data science peers to build and support our AdTech platform.
• Architect and build data-oriented solutions that are simple, scalable, reliable, secure, maintainable, and make a measurable impact.
• Provide our teams with the data they need to build, sell, and manage our platform. Develop, deliver and maintain batch and real-time data pipelines, analysis services, workflows and orchestrations, and create and manage the platforms and data infrastructure that hold, secure, cleanse and validate, govern, and manage our data.
• Manage our data platform, incorporating services using Airflow/Composer, BigQuery, Snowflake, Kafka, and Redis running on Kubernetes, GCP, and AWS.
• Support our Data Science teams with access to data, performing code reviews, aiding model evaluation and testing, deploying models, and supporting their execution.
• Employ modern pragmatic engineering principles, practices, and tooling, including TDD/BDD/ATDD, XP, QA Engineering, Trunk Based Development, Continuous Delivery, automation, DevSecOps, and Site Reliability Engineering.
• Contribute to driving ongoing improvements to our engineering principles, practices, and tooling. Provide support & mentorship to junior engineers.
• Develop and maintain a contemporary understanding of AdTech developments, industry standards, partner and competitor platform developments, and commercial models, from an engineering perspective.

Department/Project Description

Who is our client:
Our client is a global data products and technology company. They are on a mission to transform marketing by building the fastest, most connected data platform that bridges marketing strategy to scaled activation.
They work with agencies and clients to transform the value of data by bringing together technology, data and analytics capabilities. Delivering this through the AI-enabled media and data platform for the next era of advertising.
The client is endlessly curious. Their team of thinkers, builders, creators and problem solvers are over 1,000 strong, across 20 markets around the world. Our client's culture is based on mutual trust, sharing, building, and learning together. They value simplicity, maintainability, automation, and metrics.

About this role:
Client's team consists of 100+ engineers, designers, data, and product people, working in small inter-disciplinary teams closely with creative agencies, media agencies, and with their customers, to develop and scale their platform, a leading digital advertising optimization suite that delivers amazing outcomes for brands and audiences.
As a Data Engineer you will work with the client's Product and Engineering team, as well as other feature development teams, to build, deliver and operate their data platform. The role is a hands on data engineering role, analyzing and sourcing data for their teams to use, building and maintaining pipelines and automation to wrangle, cleanse, secure, govern and provide that data to teams, building automations, owning tests, supporting junior engineers, and contributing to the wider team principles and practices, and the tools they use. The role has no line management responsibilities.
Their data platform is built with Python and Airflow, deployed using CI/CD, heavily exploits automations, and runs on AWS, GCP, k8s, Snowflake and more. Their efforts in data engineering support their adserving platform which serves 9 petabytes and 77 billion objects annually, which amounts to 20 billion ad impressions across the globe. You’ll play a leading role in significantly scaling this further.