Video Game Analytics 101, Part 3: The Slightly-More-Advanced Bits

Posted by Dmitri Williams

Find me on:

Oct 9, 2013 6:59:00 AM

Video Game Analytics 101-3 Ninja Metrics

Now that you have the basics from the last post, you’re ready for the next level where we get away from the aggregate metrics and into areas where you can take actions, test, and see results.

Here we introduce the following concepts:

 - Individual-level metrics

 - Getting past averages

 - Segmentation

 - Predictive analytics

 - LTV and Social Value

 - Individual-level metrics

The basic metrics tell you averages, but you can’t act on an average unless you want to risk doing too much for some users and not enough for others. In this age of big data, there is no excuse to send out a one-size-fits-all anything. The marketing tools are simply too easy now and should be built into any analytics package.

Every user has a value and you should see it, understand it and act on it. Let’s take something as basic as ARPPU, the average spending of your paying users. ARPPU may be $17, but there’s a huge difference between the following two possibilities: A) one person spent $100 and everyone else spent $12 or less; B) half of the users spent $15 and the other half spent $19. If you want to treat the users differently based on their behavior, you simply have to look at them and interact with them as individuals, or at least in sensible buckets. For example here, you could have a bucket for minnows and one for whales.

Likewise, if you have predictive values, LTVs or anything else, you must see them on a user-by-user basis so you can take intelligent actions.

Getting past averages

Averages are just not that useful, and can lead to bad decisions. So, rather than trust that one number, you want to know what the full range of values looks like (the fancy word is “distribution”). The best way is to graph them. Say we have those same spenders from the previous section where the ARPPU was $17. Here are the 10 players who spent money:

Right away you can tell that an average is going to be misleading. Of course, you don’t want to graph out every metric you see. So, there are some simple KPI metrics that will tell you how to think about that ARPPU (or any value).

The standard deviation is the easiest. It tells you how wildly that average value swings around. If the ARPPU is $17 and the standard deviation is $5, then that tells you that just about everyone is clustered around $17. In other words, that $17 is a pretty trustworthy way of knowing what most people spent. But what’s the standard deviation for this graph? It’s $29. That’s a lot more than $17, and because it’s so big in relation to the average, you now know that the average wasn’t a great indicator.

The other metric I like to use is called a Gini coefficient. It tells you how much of the behavior is due to one person vs. everybody. It’s a favorite of economists who look at income, and if you imagine that graph above was income you’d see why--one rich guy and a bunch of poor people. In our case, we’re looking at spending, and clearly the spending is being driven by a minority, not the group--in this case, just one player. A gini always goes from 0 to 1, where a 1 means one person is doing everything and a 0 means that everyone is doing exactly the same amount. The gini for this group? It’s .52, telling you that it’s fairly skewed, but that there is still some spending value among the smaller frys here.

So, see how we went from an ARPPU of $17 and then got a lot smarter by also seeing the standard deviation of $29 and the gini of .52? If you also see a minimum and maximum value ($5 and $100 in this case), you really get a good sense for the entire distribution.

Segmentation

It’s always smart to think about your users as being in different buckets and groups. You might guess that men will behave differently than women, that the French will spend differently than the Australians, or that the older users will send fewer messages than the younger ones. However you want to slice and dice it, segmentation lets you layer those group values on to other metrics.

Let’s stick with ARPPU for a minute. If the whole population is $25, but you find that men are $45 and women are $5, you’ve just diagnosed a problem. Segmenting can be done based on start date, which gives you cohort analysis. Check out our cohort analysis blog post for more detail. Or, you may segment based on one group you had see version 1 of the app and one group that saw version 2. Presto, you have AB testing.

More complex variations of segmentation allow multiple types. For example, show me the female Germans using iOS vs. the male Americans using Android. As always, the key is thinking about the right questions first, then picking the tool second.

Predictive Analytics

We do a lot of this, so you can find much more detail on these topics here: "Predictive Analytics for Games" or "9 Predictive Analytics You Should Use Now". The basics are exactly what you think, though. Predictive analytics tell you the likelihood that something will happen. Again, you can think about this in the aggregate, or by individuals. Let’s say you want to know the likelihood of churn. You can see that value overall, or for each person. And, you should want to see their spending for context. The player who is 80% likely to leave and has spent $10 should not be treated like the player who is 70% likely to leave who has spent $2,000.

Predictive analytics can be done by a smart team of PhDs by building a model with your data. We did that for many years, and then figured out how to automate it, so fresh results now appear daily and require no team.

LTV and Social Value

LTV stands for Lifetime Value. For some, this is how much the user has spent to date, but if you have predictive analytics this should be: how much they will spend before they churn out. The way we calculate it, LTV is an asocial value. That is, it’s really only money that is driven by the user and their relationship with the developer.

In contrast, Social Value is revenue created by the users interacting with each other. Seen in the aggregate, it’s how much revenue the community generates. Seen on the individual level, it’s how much the user generates among his/her friends. Some people call this influence, but since there are “influence” metrics on the market that do something very different (i.e. measure Twitter activity and call it influence), we use the Social Value term instead.

Topics: Game Analytics, Video Game Analytics, Game Metric Definitions

    

Subscribe to Email Updates

Ninja Metrics Katana Demo

Ready for a live Demo of Katana?

If you have a social game or application, we can help you learn who matters most in your network and what features are affecting the health of your community