Evaluating promotions after they run

The question worth asking

Evaluating promotions is the practice of deciding, after the fact, whether a promotion was worth running and whether to run it again. It sounds straightforward and is routinely done in a way that guarantees the wrong answer.

The failure is almost always the same: the evaluation measures units sold during the promotion. Units sold during a promotion is not the outcome. It is the one number certain to look good, because the whole point of cutting the price was to move more units. A promotion that sold 3,000 units against a normal 1,000 is not a success until you know how many of those 2,000 extra units were genuinely incremental, what they cost to generate, and what they took from elsewhere.

This page covers the baseline problem, the costs that hide, the four ways a promotion can succeed on units and fail on contribution, and how to build a ranking that tells you what to repeat.

Everything depends on the baseline

Incremental units are actual units minus the baseline: what would have sold without the promotion. The baseline is counterfactual and therefore estimated, and every argument about promotional performance is at bottom an argument about the baseline, usually without anyone saying so.

Three approaches, in ascending order of trustworthiness:

The modelled baseline is the right answer where the data supports it, and the practical obstacle is usually that there are not enough non-promoted weeks to fit against. An item promoted in fourteen of the last twenty-six weeks has no clean baseline to recover, and the honest response is to say the baseline is unrecoverable rather than to produce one anyway. This is developed further in post-promo lift against a SPINS baseline.

One correction that is nearly always missing: the weeks after the promotion belong in the evaluation. A price promotion pulls purchases forward. Buyers who would have bought in week 5 bought in week 3 at a discount, and week 5 is therefore depressed. Measuring only the promoted weeks counts the pull-forward as incremental and never subtracts the trough. The evaluation window has to run until sales return to baseline, which is typically two to four weeks past the end of the promotion and is worth measuring rather than assuming.

A worked evaluation

An illustrative promotion for one item at Sunrise Market. All figures are constructed to show the arithmetic.

Promoted weeks show 3,950 incremental units. Include the recovery weeks and the true incremental figure is 3,400 units, 14% lower. An evaluation stopping at week 4 overstates the result by that margin, every time, on every promotion.

Now the economics, at an illustrative $4.00 regular price, $2.80 promoted price and $2.00 cost:

The promotion moved a great deal of product and lost money. Two lines do it.

The larger is the fourth: the discount applied to every unit, including the roughly 2,000 baseline units that would have sold at full price anyway. That subsidy is invisible in a units report and is frequently the largest single cost in the whole exercise.

The third line is the one that gets mis-stated even by people who remember to include the recovery weeks. It is tempting to net the recovery shortfall against the promoted-week lift and value the whole 3,400 at the promoted margin. That is wrong, and it flatters the result by $660. The 550 units that moved from week 5 into week 3 did not fail to earn $0.80. They failed to earn the $2.00 they would have earned at full price, which is what "pulled forward" costs you. Value the promoted-week lift at promo margin and the recovery shortfall at full margin, separately.

The whole thing checks against a simpler number. Actual margin across weeks 3 to 6 is $7,660; four normal weeks would have produced $8,000. The promotion moved $340 of margin backwards before a dollar of trade spend, and $1,840 after.

The costs that hide

Four costs are routinely left out of promotion evaluations, and all four make results look better than they were.

Subsidised baseline units. As above. Every unit that would have sold anyway sells at a lower price. On a shallow discount with modest lift, this alone can exceed the incremental margin.

Cannibalisation within your own range. A promoted 12-pack takes sales from your own 6-pack. Category-level lift is smaller than item-level lift, and sometimes negative, when the brand's own portfolio is the source of the volume. Evaluating at item grain and never at category grain is how this stays hidden.

Forward buy. Where the retailer buys ahead at the promoted cost and sells after the promotion at full retail, the manufacturer funds a discount on units sold at a normal price and gets no lift for it. Shipment data shows a spectacular week. Consumption data shows the truth, which is the practical argument for evaluating on consumption rather than shipments.

The cost of the stockout you caused. A promotion that sells out on day four of a fourteen-day window loses the remaining ten days, and lost sales appear nowhere in any report. The item simply stops selling, which reads as demand ending rather than supply running out.

Planning promotions with fulfillment capacity

That last cost points at the constraint most promotional planning treats as someone else's problem. A promotion is a demand event that has to be served, and the serving capacity is finite: production runs, distributor lead times, DC slots, and shelf space that physically holds a fixed number of units before it needs replenishing mid-week.

Planning promotions with fulfillment capacity in view is the difference between a promotion that runs for its full window and one that sells out early and looks like a triumph in the units report. The check is not complicated:

• Expected peak weekly demand, from the last comparable promotion rather than from the plan
• Units the shelf physically holds between deliveries
• Production or allocation committed by the promotion start date
• Lead time to resupply mid-promotion, which is usually longer than the gap between deliveries

Where peak demand exceeds what the shelf holds between deliveries, the promotion will go out of stock regardless of how much inventory sits in the DC, and no amount of forecast accuracy fixes it. That is a fulfilment constraint, and it is resolved by changing the delivery schedule or the promotion depth, not by ordering more.

The related failure is the reorder immediately after the promotion, which reads promotional volume as baseline demand and over-orders into a falling market. The mechanics are in replenishment planning.

Evaluating promotions at the right grain

The same promotion produces different verdicts at different levels, and all of them can be true at once. This is not a measurement artifact to eliminate; it is the actual structure of the outcome, and collapsing it to one number destroys the information.

An item-level triumph that is flat at category level tells you the promotion moved share within the category rather than growing it, which is a perfectly good outcome if share was the objective and a poor one if the retailer was promised category growth. Those are different conversations with the buyer, and only the category view distinguishes them.

Two practical rules follow. Always evaluate at least two grains, item and category, because the gap between them is the cannibalisation estimate and costs nothing extra to produce. And state the objective before the promotion runs, because a post-hoc choice of grain will reliably find the level at which the promotion looks best.

Who should own the evaluation

A recurring organisational failure is that the team that planned the promotion also evaluates it, with no separate view of the numbers. That is not a question of anyone's integrity; it is that the planner already believes the baseline assumption they set, and the baseline is the entire argument.

What works is separating the baseline from the verdict. The baseline should come from a method agreed before the promotion ran and applied mechanically afterwards, so the one genuinely contestable number is not chosen by the person whose result depends on it. The verdict, and the recommendation about repeating, can then sit with the commercial owner without the analysis being disputed every cycle. Who has the data access to do this at all is its own question, covered in who can analyze promo data.

Ranking what to repeat

The output of evaluating promotions should be a ranked list, not a verdict on each promotion in isolation. The useful ranking is net contribution per unit of the scarce resource, and the scarce resource is usually trade spend or promotional slots rather than units.

Promotion A produced the largest absolute contribution and B is the better use of the next dollar. If slots rather than dollars are the constraint, rank on absolute contribution instead. Being explicit about which resource binds is what makes the ranking actionable, and it changes the answer.

What to keep, so the next evaluation is cheaper

Most of the cost of evaluating promotions is reconstructing what actually happened, months later, from memory and a shared drive. Almost all of that is avoidable by recording six fields at the time the promotion is set:

None of this is analysis. It is bookkeeping, and it takes minutes at plan time against hours at evaluation time. A brand that keeps a clean promotion calendar in this shape can evaluate a year of activity in an afternoon; one that does not will spend a week per quarter rebuilding it and will still be arguing about which stores were included.

Where Scout fits

Scout evaluates promotions against a modelled baseline from your own POS or syndicated data, runs the window through the recovery weeks by default rather than stopping at the last promoted week, and reports net contribution with the baseline subsidy as an explicit line rather than folded into an average. Item and category lift are shown together so cannibalisation is visible instead of inferred.

The boundary: Scout does not plan promotions, commit trade funds, or negotiate with retailers. It measures what the promotion did and what it cost, so the next planning conversation starts from a number rather than a recollection.

The short version

• Units sold during a promotion is not the outcome. Incremental contribution is.
• Everything depends on the baseline, and an item promoted more than half the time has no recoverable baseline. Say so rather than estimating one.
• Run the window through the recovery weeks. Stopping at the last promoted week overstates every promotion by counting pull-forward as incremental.
• The discount funded on baseline units is often the largest cost and appears in no units report.
• Cannibalisation, forward buy and self-inflicted stockouts all flatter the result and all hide by default.
• Rank on return per unit of whatever is actually scarce, and be explicit about which resource that is.