Rules
Part of Reading richest rankings with a sceptical eye
Buyer's checklist for richest rankings methodology: what we do instead
Richest rankings methodology statements, audited: how to tell a published method that can be re-run from a page that only describes an intention to be careful.
Most ranking publishers now include a methodology page. That is progress, and it creates a new question: does the stated method let anyone else arrive at the same answer? A method is a set of instructions. A page describing an attitude toward evidence is something else, and the two are easy to confuse.
What to take away
- A method is instructions someone else can follow, not a description of care.
- Published statements usually name source types without naming sources.
- The test is whether a second person could rebuild one entry.
Six things a real method states
The population. Exactly who is eligible: individuals or families, living or including estates, which countries, what threshold. Without this the ordering is not defined.
Six things a real method states
- Populationexactly who is eligible
- Quantityassets or net assets
- Valuation dateone date stated
- Valuation ruleshow each asset priced
- Liabilitieswhat was obtained and assumed
- Uncertaintystated interval per entry
The quantity. Assets, assets minus liabilities, or something else. Named once and applied consistently, not swapped between paragraphs.
The valuation date. One date for everything priced in a market, stated on the page.
The valuation rules. How each asset class is priced, including private companies, property and stakes in partnerships. If a multiple is applied, from which comparables and chosen how.
The treatment of liabilities. What was obtained, what was assumed, and what the assumption does to the result. This section separates a method from a gesture.
The uncertainty. A stated interval for each entry, derived from the assumptions rather than attached decoratively.
A statement carrying all six is a method. A statement carrying two or three is a description of intent.
What published statements usually say instead
| Typical phrasing | What it establishes |
|---|---|
| We consult filings, court records and industry sources | Which categories were consulted, not which documents |
| Figures are reviewed by a team of researchers | That work happened, not what it produced |
| We adjust for debt where known | That liabilities are handled only when visible, which is rarely |
| Private companies are valued using comparable multiples | Not which comparables, nor on what date |
| Estimates are as of publication | Not a valuation date, since publication is a range of days |
| Subjects are given an opportunity to comment | That the input differs between people who engage and people who do not |
Each row describes something a careful person would do. None of them lets a reader rebuild an entry, and rebuilding is what distinguishes a method from a promise.
What statements say vs establish
Typical phrasing
- Consult filings
- We consult filings and court records
- Reviewed by team
- Figures reviewed by researchers
- Adjust for debt
- We adjust for debt where known
- Comparable multiples
- Private companies valued by multiples
- As of publication
- Estimates are as of publication
- Chance to comment
- Subjects may comment
What it establishes
- Consult filings
- Categories consulted, not documents
- Reviewed by team
- Work happened, not what it produced
- Adjust for debt
- Liabilities handled only when visible
- Comparable multiples
- Not which comparables, nor date
- As of publication
- Not a valuation date, a range of days
- Chance to comment
- Input differs by who engages
The reproducibility test, applied to a page
Take any single entry and try to reconstruct it from the methodology statement alone. Which documents were held. Which prices at which date. Which multiple against which comparables. What was assumed about debt.
If you cannot get past the first question, the statement is not a method. The standard is not imported from somewhere alien. It is the ordinary standard in fields that publish numbers about populations. Statistical agencies publish their designs in full. The Bureau of Labor Statistics does so in its Handbook of Methods.
Survey researchers have long-standing conventions for what must be disclosed alongside a result. The American Association for Public Opinion Research sets them out in its disclosure standards. Those documents show what a real methodology looks like. They are long, specific and full of admitted limitations.
The comment loop, which methods rarely address
One item deserves separate attention because it is a methodological problem rather than an editorial nicety. When subjects are invited to comment, some do and some do not, and the two groups end up measured by different processes.
A person who engages can correct errors, supply context and argue for a treatment of their assets. A person who ignores the request is estimated from public material alone. Whatever the compiler's care, the input quality now varies with a personal decision that has nothing to do with wealth. The published statement almost never says what was done about it.
What we do instead
Publish mechanism, and name the bodies that hold the real records. That approach is set out on estimate methodology. Why the format itself resists uncertainty is on richest rankings, what the strongest available documents actually contain is on executive net worth, and why the lifetime version of the question is no easier is on career earnings.
Common questions
Is a detailed methodology page a good sign?
It is a better sign than nothing, and it should be read against the six items above rather than admired for its length. Detail about process is not the same as detail about inputs.
Do any published rankings state an uncertainty interval?
Some acknowledge uncertainty in prose. Almost none attaches an interval to an entry, because an interval wide enough to be honest would make the ordering untenable.
Could a ranking be made reproducible?
For a narrow population with full disclosure obligations, partly. For anyone whose assets and debts are private, no, and no method statement can fix a missing input.
