Research · open data
Customer retention model on 500,000+ transactions
Which clients come back? A retention model on a public dataset that beats the benchmark by 9 points of AUC.
The task
Predict which clients will buy again, and turn the answer into budget advice: how much to spend on winning new clients versus keeping current ones. The same question applies to a fund’s backers, a firm’s clients or a practice’s patients.
The data
UCI Online Retail II: 500,000+ real transactions of a UK online seller, clients in 38 countries, December 2010 – December 2011. Public dataset, CC BY 4.0, used because client data under NDA cannot be published.
The approach
- Cleaned cancellations, returns and service lines; built average transaction value and RFM features (recency, frequency, monetary) per client.
- Compared order values of UK and international clients: Welch's t-test, t = 15.42, p < 0.001, a significant difference.
- Trained a Random Forest classifier for retention and compared it with a 0.70 ROC-AUC benchmark.
- Turned model output into recommendations on acquisition budget and retention offers.
Results in numbers
Source: UCI Online Retail II (Chen, 2019), sheet Year 2010-2011, CC BY 4.0; our calculation
The chart is recalculated on the same public dataset; the model metrics above are from the original project.
Dataset and model
The source is Online Retail II by Daqing Chen on the UCI Machine Learning Repository (doi.org/10.24432/C5CG6D): 541,910 transaction lines of a UK online retailer, 2010-12-01 to 2011-12-09, licensed CC BY 4.0. The raw log stays at the source. Below are the per-customer features we derived from it, a summary by segment and the script that rebuilds both. Our files carry the same CC BY 4.0 license: credit Daqing Chen (UCI) and DataSense8 when you reuse them.
How the dataset is built
- Sales only: cancellations, zero or negative lines, postage and fees, duplicates and lines without a customer ID are removed.
- Features use transactions before 2011-09-01: days since last purchase, number of purchases, total and average spend, days since first purchase, distinct products, UK or not, RFM scores and segment.
- Label: did the customer buy again in the next 90 days. 3,314 customers, 56.1% came back.
- Model: Random Forest, five-fold cross-validation, so every probability in the file comes from a model that did not see that customer.
The results above the dataset come from the original project; this rerun uses a fixed cut-off and a 90-day window so anyone can check it, which is why its numbers differ.
| RFM segment on 2011-09-01 | Customers | % of revenue | Bought again in 90 days, % | Model estimate, % |
|---|---|---|---|---|
| Champions | 482 | 45.6% | 84.6% | 84.1% |
| Loyal | 667 | 30.5% | 77.7% | 78.1% |
| Can't lose | 41 | 1.4% | 61.0% | 65.4% |
| At risk | 413 | 7.0% | 48.7% | 48.4% |
| Potential loyalists | 316 | 3.7% | 52.5% | 53.6% |
| Need attention | 184 | 2.3% | 50.0% | 52.0% |
| New | 49 | 0.4% | 44.9% | 48.6% |
| Promising | 78 | 0.7% | 52.6% | 45.1% |
| About to sleep | 212 | 1.4% | 42.9% | 42.9% |
| Hibernating | 872 | 7.1% | 33.8% | 33.8% |
Download
- Per-customer features and labels (CSV)
- Summary by RFM segment (CSV)
- Python script that rebuilds both files (Python)
- Model metrics (JSON)
CSV files use a semicolon separator, a decimal comma and UTF-8. To rerun: pip install pandas openpyxl scikit-learn, then python customer-retention-model.py; the script downloads the source file from UCI.
Have a similar question about your data?
More case studies: Staking fund portfolio: the range of outcomes, not one forecast · Running a small business on numbers: 12 years of pricing, purchasing and costs