Your metrics may be more arbitrary (and less useful) than you think.

In this era of data analytics and data science driven by machine learning (ML) and artificial intelligence (AI), there’s often a temptation to jump straight to solving the most complex business problems because these new techniques allow for it. But for most businesses, a more immediate and tangible opportunity exists.

The metrics you use to evaluate performance may be calculated using arbitrary factors that are based more on historical business practices than on current customer behavior. Using statistical methods to recalibrate your key metrics based on actual customer behavior will result in insights that are more consistent, actionable and representative of your business, while also setting you up to implement ML and AI successfully.

The quick approach

Our brains are programed to make things look nice and to create order. We’ll often round to the nearest 5, 10 or 100 for the sake of neatness. We’ll use intervals that are familiar (weeks, months, quarters, years), design ranges that look even (1–10, 11–20, 21–30) or follow a pattern (1–2, 3–5, 6–10) because that makes the bands themselves easier to understand, rather than the distribution they show naturally. We often begin to view the business and to take action based upon these values or bands, without ever questioning their origin or utility.

A Stable Analysis Period

The time period over which you run your analysis or reporting is incredibly important. You need to find a time period that ensures that a representative volume of your customers — and their behavior — will be included in the analysis. However, especially in the case of high frequency businesses, the period shouldn’t be so long that you lose efficiency due to the high volume of data that needs to be processed.

A view across multiple stable periods

The best way to ensure consistency in your analyses is to take your inferences over multiple time periods. Doing this means you can remove factors that may skew results if just viewed in a limited snapshot. For example:

Pragmatic, thoughtful modeling

It’s important to note that this isn’t just an algorithm spitting out some clusters and the business blindly using the output. As with any machine learning or artificial intelligence technique, the results must be vetted by an analyst to ensure that they’re appropriate given the business context. For example, it may be that the highest average spend/frequency band identified in the clustering only accounts for 0.1% of your customer base. This is likely too small to be particularly useful. By running the algorithm multiple times with varying numbers of clusters outputted, you can see where clusters naturally split. You can then use those splits to adjust an earlier solution to make the resulting solution more applicable.

Selecting useful cluster sizes
RFM segmentation



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aviary Analytics

Aviary Analytics

We help clients achieve transformative growth using advanced analytics, data modeling, and AI.