Call us: +1 (855) 514-8470Available 24/7 · [email protected]

← All case studies

Research · open data

Customer retention model on 500,000+ transactions

Which clients come back? A retention model on a public dataset that beats the benchmark by 9 points of AUC.

The task

Predict which clients will buy again, and turn the answer into budget advice: how much to spend on winning new clients versus keeping current ones. The same question applies to a fund’s backers, a firm’s clients or a practice’s patients.

The data

UCI Online Retail II: 500,000+ real transactions of a UK online seller, clients in 38 countries, December 2010 – December 2011. Public dataset, CC BY 4.0, used because client data under NDA cannot be published.

The approach

  • Cleaned cancellations, returns and service lines; built average transaction value and RFM features (recency, frequency, monetary) per client.
  • Compared order values of UK and international clients: Welch's t-test, t = 15.42, p < 0.001, a significant difference.
  • Trained a Random Forest classifier for retention and compared it with a 0.70 ROC-AUC benchmark.
  • Turned model output into recommendations on acquisition budget and retention offers.

Results in numbers

0.79ROC-AUC, against a 0.70 benchmark
77%model accuracy
500K+transactions, 38 countries
t = 15.42UK vs international order value, Welch's t-test, p < 0.001
Share of revenue by RFM segment, %Champions are 14.6% of customers and bring 49% of revenue.
Champions49.0%
Loyal26.0%
Can't lose2.0%
At risk7.3%
Potential loyalists5.6%
Need attention1.8%
New0.2%
Promising0.3%
About to sleep1.9%
Hibernating5.9%

Source: UCI Online Retail II (Chen, 2019), sheet Year 2010-2011, CC BY 4.0; our calculation

The chart is recalculated on the same public dataset; the model metrics above are from the original project.

Dataset and model

The source is Online Retail II by Daqing Chen on the UCI Machine Learning Repository (doi.org/10.24432/C5CG6D): 541,910 transaction lines of a UK online retailer, 2010-12-01 to 2011-12-09, licensed CC BY 4.0. The raw log stays at the source. Below are the per-customer features we derived from it, a summary by segment and the script that rebuilds both. Our files carry the same CC BY 4.0 license: credit Daqing Chen (UCI) and DataSense8 when you reuse them.

How the dataset is built

  • Sales only: cancellations, zero or negative lines, postage and fees, duplicates and lines without a customer ID are removed.
  • Features use transactions before 2011-09-01: days since last purchase, number of purchases, total and average spend, days since first purchase, distinct products, UK or not, RFM scores and segment.
  • Label: did the customer buy again in the next 90 days. 3,314 customers, 56.1% came back.
  • Model: Random Forest, five-fold cross-validation, so every probability in the file comes from a model that did not see that customer.
0.734ROC-AUC of this public rerun
0.684ROC-AUC of recency alone, no model
66.6%accuracy at a 0.5 threshold
3,314customers in the file

The results above the dataset come from the original project; this rerun uses a fixed cut-off and a 90-day window so anyone can check it, which is why its numbers differ.

RFM segment on 2011-09-01Customers% of revenueBought again in 90 days, %Model estimate, %
Champions48245.6%84.6%84.1%
Loyal66730.5%77.7%78.1%
Can't lose411.4%61.0%65.4%
At risk4137.0%48.7%48.4%
Potential loyalists3163.7%52.5%53.6%
Need attention1842.3%50.0%52.0%
New490.4%44.9%48.6%
Promising780.7%52.6%45.1%
About to sleep2121.4%42.9%42.9%
Hibernating8727.1%33.8%33.8%
Source: UCI Online Retail II (Chen, 2012), sheet Year 2010-2011, CC BY 4.0; our calculation

Download

CSV files use a semicolon separator, a decimal comma and UTF-8. To rerun: pip install pandas openpyxl scikit-learn, then python customer-retention-model.py; the script downloads the source file from UCI.