Three Thermometers

The pack went out on Sunday night. By ten on Monday the board had three numbers for one month, a plan figure of €1,850,000, and a disagreement about whether August had been a good month.

The growth dashboard said August grew +6.42% and beat plan. The investor pack said it shrank 5.10%. The finance close said it grew +3.64% and missed plan by €46,587. Somebody suggested that the data team should “find the right number”, which was the wrong request: the three numbers were €77,076 apart — 4.29% — and every one of them was produced by a correct query over correct data.

Two things had happened in the business. Free shipping ran for the whole of August as a promotion. And two wholesale clearance lots had shipped in July, to a reseller buying end-of-season stock by the pallet.

The shape of this failure. Every failure in Lessons 06–19 had something wrong in it: a stale flag, a duplicated key, a rewritten partition, a merged customer. Here nothing is wrong. Three answers are right, none of them is designated, and a test cannot fail because a test needs one expected value and this company holds three — none of them written down.

A calibration certificate does not just record a reading. It records every source of uncertainty in that reading and what each one contributes. “GMV” is not one instruction — it is nine independent choices, and the company has never written down which way any of them is set.

• 864 defensible definitions, 552 of them giving a distinct answer, spanning €1,460,146 to €2,192,094 — 50.1% wide.
• The board’s plan of €1,850,000 is beaten by 28.6% of them. The three published numbers sit at the 79th, 43th and 47th percentile.
• Four of the nine dials are worth less than half a percent of the month whichever way the others are set; three are worth more than five percent of the audited year. One dial, D1, is in both lists at once.
• The dial the room argued about — time zones (D4) — moved €0.00 in August, and never moved more than 0.22% of any month in two years. The dial nobody mentioned — wholesale lots (D7) — is worth 16.7% of the month.
• The dials are not independent. D6 is worth 1.78% of August on the order-date basis and exactly €0.00 on the dispatch-date basis, because a cancelled order never ships. D1 is worth 8.73% of the audited year and €0.00 of August, because the promotion set every shipping fee to zero. Whether a definitional choice matters is not a property of the choice. It is a property of the month you ask about.

The two definitions differ on eight of the nine dials (D9 only becomes a choice once returns are netted at all, and the dashboard does not net them). Flip them one at a time:

• Gross definitional movement €663,959; net difference €68,962 — a 9.63× cancellation. The two numbers nearly agree because the wholesale dial (+€297,499) and the returns dial (−€294,384) point in opposite directions this month. Two numbers that nearly agree are not evidence of small disagreement.
• Run the same reconciliation in reverse order and D6’s attributed contribution moves by €39,209 — 56.9% of the entire gap. A waterfall over interacting settings is a valid path from one number to the other, not a decomposition of the difference. Publish the order, or somebody will be held responsible for €39,209 that the ordering invented.

Timing dials (which timestamp, whose midnight, which month a credit note lands in) do not create or destroy revenue; they move it between adjacent periods. Their disagreement is a boundary effect, and a boundary is a smaller share of a longer window.

Scope dials (which orders count, which components of the price count) change the level. They disagree by the same proportion however long you look.

• Weekly: timing 4.53%. Monthly: 4.81%. Annual (month-aligned FY): 1.14%. The annual reconciliation passing is not evidence that the definitions agree — it is evidence that you looked over the window where they agree most. The exec team reads the weekly number.
• The timing curve flattens near 0.84% and does not reach zero. A shift cancels over a long window; a lag does not. The dispatch-date basis holds a few days of revenue back permanently, so a roughly constant slice sits outside any window however long. Timing dials that are shifts wash out; timing dials that are lags are a permanent level difference wearing a timing costume.
• The sign flip. Across the 864 definitions, August’s month-over-month change runs from −7.32% to 17.91%, and 24.5% of them say the month shrank.
• Within the scope family the growth exposure splits again. Vouchers move the growth rate by 0.04 pp and FX by 0.18 pp, because what they scope out grows in step with the business and cancels in a ratio. Wholesale lots move it by 10.22 pp and shipping by 11.33 pp, because one is lumpy and the other changed. A scope dial is invisible in a growth rate exactly as long as the component it scopes out grows at the same rate as the rest.

Lesson 06’s band test, on all three series:

All three fail, at roughly the same rate, because all three are growing (Lesson 13: a first crossing is never the finding). The instrument compares a series to its own history, and each definition has a perfectly self-consistent history. Consistency monitoring is definition-blind by construction — it cannot tell you which thermometer is calibrated, because each agrees with itself.

The ratio that is not a ratio of anything

The exec deck’s take rate takes its numerator from the close and its denominator from the dashboard.

• Numerator population: 38,764 orders (dispatched in August). Denominator: 39,896 (placed in August, UTC, wholesale out). In both: 35,862 — 83.79% of the union.
• 7.49% of the numerator’s orders are absent from the denominator, and 10.11% the other way. It is not a slightly-wrong take rate; it is a quotient of two populations.
• Cost: a permanent −1.40 pp level bias and 1.26× the month-to-month noise, so the smallest detectable move grows from 1.16 pp to 1.46 pp.

What August was actually about

Computed with one definition throughout, take rate sits at 21.96% ± 0.58 for twenty-three months and then falls to 12.94% — −9.02 pp in one month. Consumer orders alone: 24.64% → 13.42%, −11.22 pp, on 37.77% less consumer net revenue.

That is a larger number than the entire definitional argument, and it is in none of the three figures in the pack, because all three are GMV and GMV cannot see the cost side. Net revenue does exist in the pack — it is 8.02× smaller, on a different page, and nobody divides across pages.

Honesty note: the consumer take rate’s standard deviation is 0.06 pp, so August scores z = −183.6. That is not evidence of anything — the metric is nearly mechanical (a fixed commission rate plus about a euro of shipping margin per order), so it has almost no historical variance and a z-score on it explodes. The 11.22 points are the finding.

The monitor that does fire

You cannot test which definition is right. You can test that the disagreement is behaving as it always has — but only if you monitor the dials, not the total.

The gap between two definitions is the sum of eight terms, and two of them (wholesale lots, returns) are as lumpy as the business itself; their variance swamps everything. Split the gap into its causes and each cause becomes its own quiet series with its own tight band.

Be clear what the alarm says: not that one number is right, but that the relationship between two definitions changed this month — which is exactly the amount of truth available when several answers are correct.

A semantic layer turns a metric from a habit into a declared object: a name, a measure, an aggregation, the time dimension the measure is aggregated over, and the filters that define its scope — written once, compiled into SQL for whatever asks.

Read from source (dbt-semantic-interfaces, main, 7 Sep 2026) rather than from a blog post:

PydanticSemanticModel:          name, defaults, description, node_relation,
                                primary_entity, entities, measures, dimensions,
                                label, metadata, config
PydanticSemanticModelDefaults:  agg_time_dimension          # exactly one field
PydanticMeasure:                name, agg, description, create_metric, expr,
                                agg_params, metadata, non_additive_dimension,
                                agg_time_dimension, label, config
PydanticNonAdditiveDimensionParameters:
                                name, window_choice, window_groupings
PydanticMetric:                 name, description, type, type_params, filter,
                                metadata, label, config, time_granularity
MetricType:                     simple | ratio | cumulative | derived | conversion
PydanticMetricTypeParams:       measure, numerator, denominator, expr, window,
                                grain_to_date, metrics, ...

The mapping to the budget table is nearly one to one. The date basis (D3) is agg_time_dimension — a declared field, singular, on the model. Scope (D6, D7) is the metric’s filter, so it travels with the metric instead of living in a dashboard’s filter panel. A take rate is type: ratio, and a ratio metric requires a numerator and a denominator — the chimera above is not discouraged, it is unrepresentable.

The dial the layer cannot close: there is no timezone field on either spec. A timezone is a property of the timestamp column, so D4 must be settled upstream in the model that produces ordered_at. It is the one dial a semantic layer will not settle for you — and, here, the only one worth €0.00.

The ladder

What the layer does not do: it does not decide which definition is right. Somebody still has to choose whether wholesale lots are marketplace GMV — a business judgement, not an engineering one. What the layer buys is that the judgement becomes a named object with an owner, a version and a test, instead of a habit inside three tools.

1. “Which timestamp does this number use — and is that timestamp declared in the model, or typed into a dashboard filter?” The second answer means the number has no definition, only a current setting.
2. “If I asked for this same metric weekly instead of monthly, would our two versions disagree more or less?” More, if the difference is timing (4.53% weekly vs 1.14% annually here); the same, if it is scope.
3. “This ratio — where does the numerator come from, and where does the denominator come from?” If the answer names two tools, stop and count the two populations.
4. “Which of our metrics has a person’s name against the definition?” Not a team. A person.
5. “Last time we changed a definition, did we restate history or leave a joint in the series?”
6. “Our monthly close reconciles to the warehouse — does the reconciliation report a total, or a total broken down by cause?” A total that nets to near zero is the least informative possible output, at 9.63×.

The artifact’s appendix carries the deterministic generator (no random seed; standard library only). Save it as gen.py, run python3 gen.py, read figures.json.

1. Reproduce the three readings: €1,872,375, €1,795,299, €1,803,413.
2. Predict before you compute. Pick three dials, write down the sign and rough size of each one’s effect on August, then read the budget table off your own figures.json. Most people get the sign of D7 right and the size of D1 badly wrong.
3. Change the window. Recompute the timing spread for a week and for a single day. Predict the direction first, then explain in one sentence why the curve flattens instead of reaching zero.
4. Break a ratio on purpose. Build a take rate whose numerator and denominator come from different definitions; measure the level bias and the noise ratio. Look for the mechanism — which orders are in one set and not the other — not the magnitude.
5. Write two specs, not one. Express the dashboard’s number and the close’s number as two named metrics (measure, aggregation, time dimension, filter) and give each a name a non-engineer would use. The exercise is not to pick a winner; it is to make the disagreement nameable.
6. Then do it for real. Take the metric your own company quotes most often and list the dials. Stop when you reach nine.

1. A metric name is not a definition. “GMV” left nine choices open here, and the option set spans 50.1% of the month.
2. Timing disagreements shrink with the reporting window (30.2% on a day → 1.14% on a year); scope disagreements never do (43.4% over the audited year). The annual reconciliation passing proves nothing about the weekly deck.
3. Two numbers that nearly agree are not evidence of small disagreement: €663,959 of definitional movement nets to €68,962.
4. A ratio assembled from two tools is a quotient of two populations (83.79% overlap here) and costs both bias and detection sensitivity.
5. You cannot test which definition is right. You can declare one, version it, and monitor the gap to the others dial by dial — the difference between z = +46.2 and z = −0.52.

The eleventh failure axis: the plural

event (06–10) · constant (11) · drift (12) · unwatched (13) · reversible (14) · unreproduced (15) · bundled (16) · transient (17) · extremal (18) · referential (19) · plural (20).

Not one answer that is wrong, but several that are right, with none designated. Every earlier axis was about a test failing to notice. This one is about there being no assertion to write, because an assertion needs a single expected value. Its fix is therefore never a better test. It is a decision, recorded as data, with a name against it.

Vocabulary

• semantic layer — a declaration of metrics (measure, aggregation, time dimension, filters) that compiles to SQL for every consumer, so a metric has one implementation rather than one per tool.
• measure vs metric — a measure is a column plus an aggregation (gmv_eur, sum); a metric is a measure plus a scope, a time grain and a name people say out loud.
• agg_time_dimension — the declared timestamp a measure is aggregated over (D3 here). When it is not declared, it is whatever the last person typed.
• additive / semi-additive / non-additive — GMV is additive over every dimension. Active listings are semi-additive: sum across brands, never across days (what non_additive_dimension with a window_choice exists to express). A rate is non-additive in both directions — store the numerator and denominator, never the quotient.
• timing vs scope disagreement — timing moves revenue between periods and washes out over long windows unless it is a lag; scope changes the level and never washes out.
• the chimera ratio — a quotient whose numerator and denominator come from different definitions, and therefore from different populations.
• uncertainty budget — from metrology: an itemised list of every source of variation in a reading with each one’s contribution. The right artefact to produce the first time two teams disagree about a number.
• definitional version & effective date — Lesson 09’s Type 2 pattern applied to the definition rather than the data.

Threads back to: 05 (grain — a structural disagreement), 06 (the band test, run here on three series at once), 09 (versioning, applied to the definition), 11 (where a cleaning rule belongs), 13 (count equality is not set equality; here, population equality), 15 (the muted WARN, and the grain test as enforcement), 18 (an aggregate hiding an extremal fact), 19 (the same definition, a different idea of the entity underneath it).

594,029 synthetic orders across 730 days (593,433 after the 596 internal test orders are removed — the one filter all nine dials agree on). Every attribute comes from modular arithmetic on the order’s index: no random seed, and the output is identical on any machine.

Modelled rather than derived, and disclosed on purpose: August 2026 is a free-shipping month (fee zero, 11% more orders, return propensity three points higher) and July 2026 contains two one-off wholesale clearance lots. Everything else is arithmetic over that population. Month-level and week-level figures are folded from the same day-grain arrays, and the generator asserts the two agree to the cent for all three team definitions.

Two numbers that are not evidence. The consumer take rate’s z of −183.6 is an artefact of a near-mechanical metric’s tiny variance. And the window curve is noisier below one week because a single weekday dominates a short window — the trend is the finding, not any individual short-window point.

Non-additivity, measured. Averaging twelve monthly average order values gives €50.25 against the true year figure of €50.18 (+0.130%); averaging twelve monthly take rates gives 21.048% against 20.767% (+0.281 pp). Small here because volume and rate barely correlate; unbounded in general, which is why a ratio should be stored as two measures and never as a quotient.

Appendix check. The generator was extracted back out of the rendered artifact HTML, run in a clean directory, and diffed against the published figures key by key: 136 top-level keys, 2,695 scalar values, 0 differences, with an assert that no container path leaked into the shipped code.