The model is the easy part
A chain I looked at had run a pricing engine for eleven months and merchants were overriding roughly 60% of its recommendations. The engine was not broken. It had recommended a 40-cent increase on a gallon of milk and an 18-cent cut on the item beside it, and both were arithmetically correct answers to a question nobody had written down. Price optimization models fail on the objective function and the constraint set far more often than they fail on the math, and those two are the retailer's job rather than the vendor's.
This page is about retail shelf pricing: what to charge for an item in a store in a week. It compares the four model families in use, states the objective each one is actually solving, and lists the constraints that have to be in place before any of them produce a price a merchant will sign.
The architecture that defines those constraints, the zones and roles and floors, is a separate decision covered on retail pricing strategies. A model searches inside that architecture. It cannot substitute for it.
Write the objective function first
Every price optimization model maximizes something. If nobody chose what, the software chose for you, and the default is almost always gross margin dollars.
maximize sum over items i of (p_i - c_i) * q_i(p)
subject to constraints C
Three objectives are common and they give different answers on the same item:
| Objective | Formal target | Who asks for it | What it sacrifices |
|---|---|---|---|
| Gross margin dollars | sum of (p - c) x q | Finance | Units, and eventually trips |
| Revenue | sum of p x q | A growth or share mandate | Margin rate, immediately |
| Margin subject to a floor on units | max margin, q at or above q_min | Merchandising | Some margin, in exchange for not losing the basket |
The third is the one most retailers actually want and the one least often configured. A pure margin objective on an item with a measured elasticity near -1 will keep raising price for as long as you let it, because margin dollars climb on every increase even while units fall away underneath. On a background line that is fine. On an item that brings the trip it is how a chain gives up a basket to gain a dime.
Two more terms belong in the objective and are usually missing. Cross-item effects: units lost on a branded bag partly reappear on the private-label bag beside it, at a different margin rate. Trip value: an item can lose money on its own line and earn it back in the basket, which is the entire economic argument for a key value item. A model with neither term will systematically over-price traffic drivers, and that is what produces the override rate.
The four price optimization models
| Family | What it needs | What it optimizes | Right when | Breaks when |
|---|---|---|---|---|
| Rule-based | Cost, a margin target, a rule table | Compliance with the rules | The file is new, dirty, or has no clean price history | Cost moves in a direction demand does not care about |
| Elasticity-based | A demand curve per item per zone | Margin or revenue given that curve | You have clean non-promoted price variation | Elasticity was fitted on promoted weeks or a distribution change |
| Competitive-response | A tracked rival price and an index target | Position against that rival | A named competitor drives your shopper's reference price | The rival is running a promotion, or you follow them down |
| Machine learning | Long panel, cross-effects, promo and seasonal flags | Whatever the loss function says | Thousands of items with interacting demand | Nobody can explain a recommendation to the merchant |
Rule-based is cost-plus with structure. Price equals cost divided by one minus the target margin, then rounded to a permitted ending, then checked against families, ladders and floors. It is not a demand model at all, and its honest description is a policy engine. Every chain needs one underneath whatever else it runs, because the rule layer is what makes the other three safe. Its weakness is that it responds to cost, and shoppers do not observe cost. A supplier increase on an inelastic premium item and the same increase on a KVI get identical treatment.
Elasticity-based models fit a demand curve and solve for the price that maximizes the objective on it. The curve is usually log-log, per item per zone, estimated from non-promoted weeks. This is the workhorse and it is only as good as the identification: you need real price variation that was not caused by the thing you are trying to predict. See price elasticity for how the number gets contaminated, because every failure mode described there propagates directly into the optimizer.
Competitive-response models price to a target index against a tracked rival, and are covered as a metric on retail price index. As a model family the distinct property is that the input is somebody else's decision. That makes them fast and makes them followers. Two chains both running index targets against each other will ratchet downward without either intending to.
Machine-learning models drop the parametric demand curve and let a gradient-boosted or hierarchical Bayesian model learn the response surface, including cross-item and seasonal effects a per-item curve cannot see. On a long panel with many interacting items they genuinely beat the alternatives. The cost is explanation. A merchant who cannot be told why an item moved will override it, and an override rate above about 20% means the model is not running your prices no matter what the license says.
The constraint set is the product
Model choice is a smaller decision than the constraint set. These are the constraints that have to be encoded before any family is safe to run.
| Constraint | Form | Without it |
|---|---|---|
| Margin floor | p at or above cost / (1 - m_min) | The model buys units below the floor on elastic lines |
| KVI freeze | p fixed for items on the key value list | Price image moves without anyone approving it |
| Price family | p equal across items in the same family | Six flavors, four prices |
| Pack ladder | price per ounce strictly falling as size rises | The 32 oz costs more per ounce than the 20 oz, on the tag |
| Zone ordering | zone 1 price at or below zone 2 at or below zone 3 | The competitive zone reads as the expensive one |
| Max move per cycle | change no more than 5% of the current price | Shelf-price whiplash and a labor cost nobody budgeted |
| Permitted endings | p ends in .99, .95 or .49 | A $4.22 shelf tag |
| MAP | p at or above the supplier's minimum advertised price | A supplier conversation you do not want |
Two of these are worth extra attention because they are the ones teams treat as cosmetic. The max move per cycle cap is what turns a theoretically optimal price into an implementable one, and it changes the answer materially, as the worked example below shows. And the KVI freeze is what stops a margin objective from doing exactly what a margin objective is supposed to do on the handful of items where the objective is wrong.
There is also a constraint that is not a rule: label capacity. A cycle that recommends 9,000 price changes across 96 stores is a change the store teams cannot execute, and unexecuted price changes are worse than no change at all, because the file and the shelf now disagree. Cap the recommendation count and rank by margin impact.
Worked example: one item, four models, four prices
A 10.5 oz bag of a national-brand tortilla chip. Current shelf price $4.29, unit cost $2.87, base rate of sale 41.0 units per store per week across 96 stores. Measured non-promoted elasticity -1.8. Constraints: 9-endings, a 5% maximum move per cycle, a 22% margin floor. The tracked rival's shelf price is $4.49, and the store brand beside it is $2.99 at a 41% margin rate.
| Model | Recommended price | Basis | Units/store/wk | Item margin $/store/wk | Category margin $/store/wk |
|---|---|---|---|---|---|
| Today | $4.29 | Incumbent | 41.0 | $58.22 | $58.22 |
| Rule-based, 32% margin target | $4.19 | $2.87 / 0.68, rounded | 42.7 | $56.36 | $56.36 |
| Elasticity-based, capped at +5% | $4.49 | Margin max under the cap | 37.6 | $60.91 | $60.91 |
| Competitive-response, index 98 | $4.39 | 0.98 x $4.49 | 39.3 | $59.74 | $59.74 |
| Cross-effect ML | $4.49 | Same price, wider accounting | 37.6 | $60.91 | $61.83 |
The arithmetic, so it can be checked. At $4.19 the price change is -2.33%, so units rise 1.8 x 2.33 = 4.2% to 42.7, and margin per unit is $1.32, giving 42.7 x 1.32 = $56.36. At $4.49 the change is +4.66%, units fall 8.4% to 37.6, margin per unit is $1.62, and 37.6 x 1.62 = $60.91. At $4.39 the change is +2.33%, units fall 4.2% to 39.3, margin per unit is $1.52, and 39.3 x 1.52 = $59.74.
The ML row recommends the same price as the elasticity row and books a different number, which is the point of it. Of the 3.4 units per store per week that leave the branded bag, roughly 22% land on the store brand at $1.23 of margin each, adding $0.92 per store per week that the item-level model treats as lost.
Four defensible models, a 30-cent spread, and $4.55 per store per week between the best and the worst. Across 96 stores and 52 weeks that is about $22,700 a year on a single line. Multiply by the roughly 4,000 lines where a model has any real freedom to move and the model choice is a material number.
Now the caveat that matters more than the ranking. Every unit forecast in that table rests on the same -1.8. If that elasticity was fitted on weeks containing a competitor's endcap or a distribution change, all four rows are wrong by the same amount and the comparison between them is still roughly right, which is the one mercy here. But the absolute recommendation is not, and a model that reports a confident price off a contaminated curve is the most expensive failure in this whole area.
How to tell whether the model is working
Backtesting a pricing model against its own training period proves nothing. The model saw those prices. Three tests actually discriminate.
Holdout stores. Hold 10 to 15 stores out of the recommendation, matched on zone and volume, and run them on the incumbent rules. After a quarter, compare margin per store per week between the treated and held-out groups on the items that actually moved. This is the only test that measures the counterfactual rather than the forecast.
Forecast error on units, not on margin. The model's unit prediction is the falsifiable part. Track weighted MAPE on predicted versus realized units for moved items, split by item class, using the discipline on measuring forecast accuracy. A model whose margin looks right because volume error and price error cancelled is running on luck.
Override rate, by merchant and by reason. An override is a labelled training example. If 60% of recommendations are overridden and 70% of those overrides are on items the merchant considers image-sensitive, you have discovered your KVI list is wrong, not that your merchants are stubborn.
Two things not to use. Do not compare to last year, because last year had different costs, a different competitive set and different weather. And do not grade the model on realized blended margin, which is dominated by mix and will move for reasons the pricing engine had nothing to do with.
Where these models go wrong
Fitting on promoted weeks. Promoted elasticity runs two to three times base elasticity. Feed it to a model choosing an everyday price and the model will recommend a permanent cut that never earns its volume back.
Simultaneity. Prices were cut historically because demand was falling, so a naive regression reads the cut as the cause of the decline and recommends raising price on the very items losing shoppers. This is the classic identification problem in pricing data and it is why deliberate, randomized price variation across matched stores is worth its cost.
Modeling smoothly through a threshold. A fitted curve interpolates through $4.99 to $5.09 and reports a 2% effect. Shoppers do not. Test the point.
Cold start on new items. A new item has no price history, so every family except rule-based has nothing to fit. Borrowing the curve from a "similar" item is defensible only when the similarity is defined by the shopper's substitution behavior rather than by the category hierarchy.
Optimizing items independently. An item-level optimizer run across a category will happily raise every item in it, because each one is optimal alone. Cross-price terms are what stop that, and they are the first thing dropped when the data gets thin.
Ignoring the execution cost. A recommendation set that requires 9,000 tag changes has a real labor cost, and half of it will not happen. Rank and cap.
Where Scout fits
Scout fits base and promoted elasticity separately off the retailer's own POS, per item and per zone, and holds the constraint set as an explicit, version-controlled input rather than as vendor defaults. The rule layer, the KVI freeze list, the pack ladders and the zone ordering are checkable against the live file, so a recommendation that would break one never reaches a merchant.
It runs the comparison above as a standing view: for a given item, what each family recommends and what each implies for units and margin, with the arithmetic visible. A merchant who can see why the elasticity model wants $4.49 and the competitive model wants $4.39 overrides on judgment rather than on distrust.
Scout carries the result through to execution as well. It can hold the item file, maintain cost and retail, and push the approved price file to the POS, which means the price the model recommended and the price on the tag are the same number.
The boundary worth stating: none of these models segment by shopper. POS is transactions, and Scout holds no demographic panel, loyalty-linked profile or footfall data. Every elasticity here is a store-level behavioral read, not a statement about who the shoppers are.
The short version
- Write the objective function before choosing a family. Margin dollars, revenue and margin-subject-to-a-unit-floor give different prices on the same item, and only one of them protects the trip.
- The four families are rule-based, elasticity-based, competitive-response and machine learning. Every chain runs the rule layer underneath whichever of the other three it chooses.
- The constraint set is the product. Margin floors, KVI freezes, families, pack ladders, zone ordering, move caps and endings decide whether a recommendation is implementable.
- On one 10.5 oz bag, four defensible models spread 30 cents and $4.55 per store per week, about $22,700 a year across 96 stores. All four rest on the same elasticity estimate, so the estimate deserves more scrutiny than the model choice.