Insights / Checklist / Data foundations and advanced analytics
ChecklistPredictive analytics: is your problem ready?
Prediction needs history, consistency and a decision that will change as a result. Use this checklist before you commission a forecasting or machine learning model.
Predictive analytics is one of the most requested and most misunderstood data projects. Leaders hear that a model could forecast demand, flag customers likely to leave or anticipate equipment failure, and the case seems obvious. Months later, the model works on paper but nothing in the business has changed.
Prediction needs three things: enough history, consistent data and a decision that will change as a result. This checklist helps you test all three before committing budget. It takes an afternoon with the right people and can save a quarter of wasted effort.
Is there enough history?
A model learns from past examples of the thing you want to predict. If you want to forecast which customers will cancel, you need a meaningful number of customers who have actually cancelled, along with what you knew about them beforehand. Rare events are hard to predict because there is little to learn from.
Seasonality matters too. A demand forecast built on fourteen months of data has seen each season barely once. Two or three full cycles give a model a far better chance of separating a pattern from a coincidence.
Is the data consistent?
Models assume that the past looks like the future in how it was recorded. When a new ERP went live, a product hierarchy was reorganised or a reason code was renamed, the data changes in ways a model cannot see. It will treat the change as a real shift in behaviour.
A typical example: a manufacturer changed its definition of a late order from one day to three days part way through the year. Lateness appeared to fall sharply. A model trained across that change would learn something that never happened.
Will anyone act on it?
This is the question most often skipped. A churn model that identifies at-risk customers is only valuable if someone will call them, offer something or change the service. A maintenance prediction matters only if engineers can schedule work around it.
Name the decision, the person who makes it and what they will do differently. If no one can answer, stop. Accuracy is irrelevant when the output changes nothing.
The readiness checklist
Work through these questions with the business owner, someone who knows the data and someone who will use the output. If more than two answers are no, fix those gaps before building a model.
- We can state, in one sentence, what we want to predict and over what time horizon.
- We have at least several hundred past examples of the outcome, or two to three full seasonal cycles for a forecast.
- We know what information was available at the moment the prediction would have been made, not just afterwards.
- We have listed system changes, reorganisations and definition changes in the history and know their dates.
- Key fields are consistently filled in; gaps and default values are understood.
- A named person will receive the prediction and has agreed what they will do with it.
- The action is possible in the time the prediction allows, such as calling a customer before renewal.
- We have a simple baseline, such as last year’s figure or a rule of thumb, to compare a model against.
- We have agreed how we will measure whether the prediction improved the decision.
- Someone will own the model after launch and monitor it as conditions change.
Start with a baseline
Before building anything sophisticated, measure how well a simple approach performs. For demand, that might be the same week last year adjusted for growth. For churn, it might be customers whose usage has dropped by half. If a model cannot clearly beat the baseline, the extra cost and complexity are not justified.
Baselines also set realistic expectations. Leaders who see that a simple rule already catches most of the cases are better placed to judge the value of the remaining improvement.
What to do with the answers
If the problem passes, scope a short proof of value with a clear success measure and a stop rule. If it fails on history, start collecting the right data now and revisit in a year. If it fails on consistency, document the breaks and decide whether a shorter, cleaner window is enough. If it fails on action, the work belongs with the business owner, not the data team.
Source: Intelliversal analysis.
