Experimentation / Product Analytics Engineer

N.B! - Preferrable locations: remote from Latvia, Lithuania, Spain, Poland, Slovakia, Romania, Hungary 

In partnership with one of the global consulting enterprises, we are seeking Experimentation / Product Analytics Engineers at multiple levels (see level-specific requirements below). These roles are part of the client's World Service App and wider product teams, helping to enhance engagement across iPlayer, Sounds, News, and Sport apps by designing, running, and analyzing experiments at scale.

Overview

Experimentation is central to making the right decisions for our users. As part of the Product Analytics & Experimentation team, you will design and manage A/B tests, calculate statistical thresholds, validate data quality, and deliver actionable insights to product teams. Your work will directly shape product decisions, ensuring that new features and user experiences are guided by evidence, data, and user behavior.

This role is highly collaborative, working closely with product managers, data scientists, engineers, and analytics teams.

Core Tools & Skills (All Levels)

• SQL (DataGrip, Workbench, VS Code, AWS Console, etc.)
• Statistical tools: Excel, Python, or R
• Experimentation platforms: Optimizely
• Digital analytics: Piano Analytics, Tag Inspector
• Data dictionaries & metadata: Alation or similar
• Collaboration tools: Jira, Confluence, GitHub
• Visualization: Tableau (desirable at higher levels)

Key Responsibilities (All Levels)

• Work closely with product and analytics teams to identify and solve problems through experimentation.
• Design and manage experiments (A/B testing, RCTs), including calculating and communicating baseline conversion rates, sample sizes, MDE, confidence levels, and statistical power.
• Monitor and validate data quality in running experiments.
• Analyze performance of experiments, visualize findings, and communicate them clearly to product teams and wider stakeholder groups.
• Conduct deep-dive analyses of results to provide actionable insights and suggest future iterations.
• Collaborate with the platforms team to improve the experimentation platform and analytics infrastructure.
• Ensure findings are accessible to both technical and non-technical stakeholders, supporting data-driven decision-making.

Level-Specific Requirements

Level 3 (Entry/Mid-Level)

• Skills & Tools: Proficient in SQL and Excel. Familiarity with Optimizely, Piano Analytics, and Tag Inspector. Exposure to Python or R for basic statistical analysis. Basic understanding of experimental design and statistical concepts.
• Typical Experience: 1–2 years in a data or analytics role. Works under general direction, escalating complex issues.

Desirable (All Levels)

• Experience running end-to-end A/B testing.
• Working knowledge of data visualization tools (e.g., Tableau).
• Familiarity with digital analytics tracking, specs validation, and debugging.
• Strong facilitation skills for workshops and stakeholder alignment.