warner bros games analytics
date:
apr 2022
timeline:
2 months
industry:
gaming
role:
lead data analyst, designer
team:
ashley takayama, rylan kanae, tiffany ng
tools:
tableau, ms suite, xl-miner, canva
this business analytics capstone project was sponsored by the warner bros games.
i led a team in analyzing a beta puzzle game to determine whether we should launch it globally. through market benchmarking, revenue and retention modeling and forecasting, we recommended a conditional greenlight: a hybrid action/puzzle game with marketing and pricing adjustments.
this project won first place and was later featured at the university’s founder’s day.
problem
given 31 days of beta performance data and first-quarter competitor data, we were tasked to determine whether it is worth launching the new mobile puzzle game at a global scale.
executives hoped that it could match the success of four other competing games: bricks ‘n balls, diamond diaries saga, hello neighbor, and simon's cat.
however, early monetization, retention, and market-fit signals were uncertain: is our puzzle game positioned to succeed as scale?
goals & metrics
what does success look like?
key metrics for evaluating a game's long-term viability are lifetime value (ltv), retention, churn, and return on investment (roi).
in the mobile gaming world, strong performance are defined by high ltv, high retention, high roi, and a low churn rate. these benchmarks guide how we analyze the beta data, forecast performance, and make comparisons.
role
lead data analyst, designer
i led the analytical workflow by running all predictive and forecasting models in xl miner, and visualizing key finding in tableau.
i also applied ux storytelling skills to design the final pitch deck, shaping a clear narrative around our insights and recommendations.
during the final presentation, warner bros games managers recognized my individual contribution in both analysis and communication.
approach
how are we tackling this problem?
1) market understanding
with the goal of understanding the market landscape, we start by conducting specific competitor and genre analysis.
2) game understanding
next, we evaluated the beta game's performance by examining revenue per user, retention curves, churn indicators and roi to establish a baseline of player behavior and monetization.
3) revenue and user count forecasting
using the beta metrics, we applied multiple linear regression models to forecast projected revenue and user count at d90 and d180, estimating the longer-term potential for our game after the beta period.
4) churn classification
we then built a classification tree model to segment users by churn risk, identifying which behaviors and early-game patterns were most predictive.
5) model comparison
to ensure accuracies, we compared several time-series models and ultimately selected the holt-winters multiplicative model for its stability and strong performance on our data.
6) final recommendations
finally, compiling our quantitative analysis with additional market research, we delivered a conditional greenlight recommendation and outlined suggestions for a successful global launch.
analysis
1) market understanding: competitor and genre analysis
[competitor analysis: total downloads by date]
among the four benchmark games, bricks 'n balls led decisively with over 6 million downloads in its first quarter, potentially driven by its hyper-casual, universally appealing gameplay and strong organic exposure through social media platforms.
[competitor analysis: revenue per user by date]
however, download volume didn't translate directly to monetization strength: diamond diaries saga generated the highest revenue per user, followed by simon's cat and bricks 'n balls.
this is potentially supported by its wider in-app purchase range based on our additional research. hence, we hypothesize that games with higher purchase ceilings tended to achieve higher revenue per user.
[genre analysis: average revenue per user colored by genre]
genre-level analysis showed that action games delivered the highest revenue per download, followed by casino, puzzle, and arcade titles.
in fact, puzzle–action hybrids, such as the long-running puzzle & dragons, demonstrated strong, sustained engagement and profitability over many years.
[genre analysis: absolute revenue to absolute download ratio]
we believe that puzzle games showed the healthiest balance of reach and revenue potential, indicating a genre with steady long-term growth rather than short-lived spikes.
by blending the appeal of puzzle mechanics with action-driven engagement, a hybrid approach could reach a broader demographic and improve monetization potential.
therefore, based on our findings, we recommended steering the beta game toward an action/puzzle hybrid to maximize its chances of global success.
2) game understanding: how is our game performing during beta?
[beta game: roi by event date]
[beta game: revenue by event date]
across the 31-day beta, both revenue per user and roi showed a steady upward trend: from lows of $0.05 rpu and 0.019 roi on day 2 to highs of $0.62 rpu and 0.65 roi by day 25.
this pattern suggests that players were more likely to spend after becoming familiar with the game, indicating a healthy late-cycle monetization potential during the beta period.
[beta game: retention rate by event date colored by sessions]
as for retention, although it gradually declined (as expected for mobile games), the total number of sessions in a day grew from 1,405 on day 1 to 9,271 by day 31, indicating a strong engagement among players who stayed.
compare to industry averages, our puzzle game seems to be doing well!
period | top 25% | average | our game |
|---|---|---|---|
day 1 | 37% | 30% | 40.85% |
day 7 | >15% | 10% | 10.51% |
day 28 | <8% | 4% | 5.60% |
we might have established a solid user fanbase already in just 31 days!
3) user count and revenue forecasting: what does d90 and d180 look like?
to estimate long-term performance, we used time-series multiple linear regression on 31 days of beta data, modeling both user count and revenue through day 90 and day 180. we applied time-series partitioning and validated linearity before running regional and network-level models.
[user count forecast]
for user count, the overall model performed well (R² ≈ 69%, p ≤ 0.05), forecasting growth from roughly 4,276 users at day 90 to 7,525 by day 180, with region 1 and the organic network consistently outperforming others.
[revenue forecast]
for revenue, only the overall model, region 1, and network 2 produced reliable fits (R² ≈ 70%), projecting revenue rising from about $3,459 at day 90 to $6,967 at day 180.
lower R² values for several regions and networks indicated those forecasts were not reliable, but the valid models showed strong upward trends pointing to meaningful long-term growth potential.
we suggest moving advertising efforts towards region 1, network 2, and organic networks!
4) churn classification: what story can we infer from our players' early behavior?
we revised our logistic classification model by redefining retention, where an active player is someone who played at least 3x during the 31-day period.
using cumulative event date, total sessions, region, and network as predictors, we generated logit, odds, and probability equations to forecast churn for d90 and d180.
however, despite a perfect roc curve and a solid lift chart, the model’s error and accuracy rates were both 0% and all p-values were ≥ 0.05, indicating severe overfitting and no statistical significance.
since the model could not reliably generalize, we ultimately did not use it for forecasting and turned to classification tree and regression tree.
[classification tree for churn]
to better segment churn behavior, we built a classification tree with a 36/38/26 training-validation-test split.
the model achieved 96.97% accuracy and showed strong generalization: test-set ROC was near-perfect, lift was >1, and decile analysis demonstrated meaningful separation of churners vs. non-churners.
the pruned tree produced a simple rule: players with fewer than 2.5 cumulative days of activity are highly likely to churn.
this explains why 9,412 users (89%) dropped off.
yet, compared to the industry average 95% churn by day 30, our retention is actually 6% stronger for the genre!
[regression tree for revenue]
we also built a regression tree to identify revenue drivers, again using a 36/38/26 data split.
model diagnostics showed stable mse across training, pruned, and test sets, indicating low overfitting and reliable predictive performance.
the resulting rule highlighted session count as the strongest determinant of in-app purchase revenue: players with ≥ 5.5 sessions generated roughly $4.79, and revenue decreased progressively for users with fewer sessions.
around 1,000 users met this threshold, illustrating a clear behavioral segment for monetization!
based on both our churn and revenue forecasts, we see strong potential for this beta puzzle game to perform well in a full global launch!
despite high industry-standard churn rates, our models suggest that we retain our users better than average, and players who stay engaged tend to contribute meaningfully to revenue.
with targeted improvements and expanded market reach, we believe the game is well-positioned for success once released worldwide.
impact
how confident are we with our findings?
after testing out various time-series models, we decided to highlight the holt-winters multiplicative model as our primary forecasting method due to its consistently low error rate, despite a small increase in mape and no signs of overfitting.
[holt-winters multiplicative model]
while we were only able to forecast through day 131 (not day 180), the model provided clear directional insights across key metrics, including retention, user count, revenue, roi, and cumulative installs.
across all comparisons, retention appeared to be declining overall—but the holt-winters output revealed a subtle but meaningful upward trend over time, rising from 4.37% on day 32 to 5.46% by day 90.
given its stability and accuracy relative to other models, holt-winters offered the most reliable snapshot of longer-term performance.
conclusion
a conditional greenlight
1) expand and refine marketing strategy
our strong early download growth justifies expanding into new marketing channels, but installs lag behind ad spend, indicating potential inefficiency.
we believe focusing on organic growth and network 2 (our strongest sources of loyal, high-value users) will yield better returns and strengthen long-term performance.
2) a hybrid puzzle–adventure game
market data shows pure puzzle games tend to fade quickly, while story-driven titles sustain engagement, with prominent example being puzzle & dragons (p&d).
given our rapid 31-day growth, the core puzzle mechanics already brings initial appeal. adding narrative depth could turn further leverage this strength and turn short-term traction into a lasting, high-value franchise.
3) increase the in-app purchase range
revenue per user rises initially but dips late in beta, likely due to low activity and limited purchase options.
following high-performing models like diamond diaries saga, expanding price ceiling may sustain a late-stage monetization and boost overall ltv.
we understand that these changes carry development, marketing, and financial risks including higher costs, potential beta retesting, and uncertain returns from new channels.
however, from our additional analysis on competitor success suggests strong potential roi if the game captures breakout traction.
takeaway
what are our next steps?
besides iterating toward a hybrid action-puzzle design, refining marketing and expanding in-app purchase option, we recommend a staged expansion to minimize risk: starting with the united kingdom, germany, japan, then finally the united states (based on total downloads and total revenue by country).
overall, we believe our beta puzzle game has the potential to succeed in a global launch!
endorsement
"Tiffany Ng has done a stellar job with the Warner Brothers’ case competition project. She has done really nice work in studying the business problem, developing both the descriptive and predictive models, and suggesting recommendations which closely resembled the best real-world solutions. She led the team with competence and care and was instrumental in their team winning the first position in the case competition. In addition, Tiffany is endowed with some special skills in user interface design, which will be a huge plus to any firm wishing to employ Tiffany."
- Dr Gudigantala, Operation and Technology Management Professor at the University of Portland