Sr. Manager, Experimentation & Product Insights - HBO Max

This is a Full Time Job

Location: Remote, California, United States of America

Company Overview

WarnerMedia is a powerful portfolio of iconic entertainment, news, and sports brands. We bring people, technology, and the world's best storytellers together to drive culture and meaningful connection. We believe the enduring power of stories can open our eyes to the world, to each other, and to new and different perspectives.

HBO Max is where storytelling takes center stage and where creatives find a home with the support and resources to do their best work, no matter the genre or format. Whatever the viewer wants to watch is front and center and more of what they crave is easily discovered. It is where our exclusive HBO Max Originals and iconic entertainment brands thrive, with HBO, Warner Bros., DC, Turner Classic Movies, Cartoon Network and more delivering the greatest array of series, movies and specials for audiences of all ages. HBO Max launched in the US in May 2020 and will launch in Latin America and Europe during 2021.

Join us. Shape your story here.

The Job

The Senior Manager of Product Insights & Experimentation will help evolve HBO Max Product testing capabilities that drive quantifiable growth in user engagement and retention through feature enhancements. This will include working with near-term capabilities while developing long-term organizational and process requirements to create a scalable Product experimentation pipeline. In addition, it will be essential to collaborate with the Product and Product Analytics teams to define and execute a meaningful test and learn agenda, comprised of both small and big loop tests that help to significantly grow the business.

The ideal candidate will have experience with A/B tests, working with testing platforms, offline evaluation and simulations, and can collaboratively work with product and engineering leaders to meaningfully translate business requirements into solutions. In addition, this team will help analyze user behavioral data to identify patterns, uncover opportunities, and create a common understanding of how people are interacting with the product.

The Daily

Leadership:
• Assess current capabilities and develop the roadmap for necessary growth/process improvement for Product Experimentation.
• Manage existing Product Experimentation resources and assess future hiring needs.

Experimentation:
• Lead analysis and execution of experiments in the areas of customer acquisition, streaming quality, application performance, and commerce.
• Share and promote experimentation best practices.
• Work with Product, Engineering and Product Analytics teams to ensure required event tracking is in place.

Analysis automation:
• Effectively scale analysis from a single A/B test to standardize metric definitions, statistical tests and results visualizations.
• Leverage 3rd party tools (e.g. Optimizely) and identify necessary internal development or external vendors to improve scale through self-serve models and automation.

Partnership:
• Partner closely with product managers, the experimentation platform team, data engineers to improve platform capabilities, drive development of new features and testing strategies.

Deep analysis:
• Work closely with the Product Analytics team to deliver reporting and analyses aimed at developing a deep understanding of in-app user engagement.
• Perform ad hoc analysis and build compelling presentations and propose data-driven decisions to shape retention strategies – supported by data from experiments.
• Synthesize large volumes of data and complex concepts into clear accessible learnings for a broad audience

The Essentials
• 8+ years of hands-on experience designing and executing A/B and MV tests. Experience with Optimizely preferred.
• 5+ years of experience using SQL.
• 5+ years of Python or other statistical programming language.
• 4+ years leadership experience (includes people management, mentoring interns, technical leadership, and project management).
• Deep understanding of statistical concepts like hypothesis testing, Bayesian decision-making, etc.
• Experience with bandit optimization, adaptive experimentation, and large-scale inference methods preferred.
• Knowledge of observational causal inference and design (regression adjustment, propensity matching) or quasi-experimental design preferred.
• Hands-on experience in advanced data analysis
• Strong communication skills, for both technical and non-technical audiences.
• Have a demonstrated history of translating complex data sets into clear and actionable narratives and communicate opportunities and challenges to stakeholders
• Experience and aptitude to deliver high-quality solutions on short timelines

Nice to Haves
• Experience thinking strategically to interpret market and consumer information, preferably about a subscription service or large e-commerce business.
• Familiarity with data exploration and data visualization tools like Tableau or Looker.
• Advanced degree in an analytical field or equivalent experience.

The Perks
• Exclusive WarnerMedia events and advance screenings
• Paid time off every year to volunteer for eligible employees
• Access to well-being tools, resources, and freebies
• Access to in-house learning and development resources
• Part of the WarnerMedia family of powerhouse brands

Warner Media, LLC and its subsidiaries are equal opportunity employers. Qualified candidates will receive consideration for employment without regard to race, color, religion, national origin, gender, sexual orientation, gender identity or expression, age, mental or physical disability, and genetic information, marital status, citizenship status, military status, protected veteran status or any other category protected by law.