Editorial & data standards

We publish two kinds of material: our own explanations and analysis, and facts about other people’s work — papers, models, benchmarks, tools and their results. The second kind follows these rules.

• Primary sources. A fact is sourced to whoever published it first-hand: the paper for a method or a result; the vendor’s own announcement, documentation or model card for a model’s specifications; the benchmark’s paper or official site for what it measures; the evaluator’s own publication for a third-party score. When an evaluator has not published a score itself and a vendor republishes it, we cite the vendor’s page, name the evaluator and still label the result third-party. News coverage, social posts and aggregators can point us to a fact, but we do not cite them as its source.
• Secure links only. Every source link uses https.
• Every fact is dated. Each record carries the date it was last checked against its source, so you can judge how fresh it is.
• The source’s own wording is kept. Where a vendor states a context window or an output limit in its own terms, we keep that wording next to the normalised number so the two can be compared.
• Settings travel with scores. A benchmark result keeps the settings exactly as its source states them — or says “Conditions not stated” when the source doesn’t say.
• Our words, not theirs. Summaries are written fresh. Direct quotes are short (under 15 words) and attributed.

These rules are not only editorial intentions. Every dataset on this site is validated against a typed schema when the site is built and by a content check we run before publishing. A record that breaks a rule stops the build, so it cannot go live.

What the schema cannot do. It checks that a source is present, well-formed and dated. It cannot check that the source says what we claim; that takes reading the source. The launch content was drafted with AI assistance and checked against its primary sources; a human editorial sign-off of each piece by HARBAN is still in progress. The corrections process below exists for what a check misses.

Every benchmark result carries one of three labels saying who produced the number, shown next to the number with a link to its source. A vendor’s own figure and an independent measurement are different kinds of evidence, and the label keeps that visible.

A HARBAN internal result must link to its complete run logs; the schema rejects one that doesn’t. Where a count above is zero, we have not published any results of that kind yet.

A benchmark can be famous and still be a weak signal. Each benchmark carries a health label, reviewed on a stated date, with a note and a source for the judgement. Read a high score on a saturated or flawed benchmark with that in mind.

Unknown is not zero. When we could not confirm a fact on an official source, we show Not verified in its place — never a blank, a dash, a zero or an estimate. A missing value tells you something true; a guessed one does not.

The same applies to benchmark references we could not confirm, such as a publication year: they are marked as unconfirmed in our data rather than silently filled in.

• Tell us. Content pages end with a “Report an issue” form. Use it for a wrong number, a broken or outdated source, or a misleading summary. The page reference is attached for you; an email address is optional and only needed if you want a reply.
• We check it against the source. If we got something wrong, we fix it.
• Corrections are public. Paper breakdowns and articles keep a corrections log: a correction adds a dated entry saying what changed, and a page with no entries says so.
• Data records show when they were checked. Models, benchmarks, results and tools each show the date they were last checked against their source, so a corrected record carries a new date.

How long reports are kept is set out in the privacy notice.

• No paid placement. It is a policy of this site that no one can pay for a listing or for placement in the tools directory, or for a position in any ranking or comparison.
• Vendor numbers are labelled as vendor numbers. A figure published by a model’s maker is marked vendor-reported, whoever else repeats it.
• Services show their real status. Each service line is labelled Planned, By request or Available, and nothing is described as offered until it is. See services.

The sourcing rules and schema checks above apply in full to anything drafted with AI assistance. If you find an error, please report it.

Evaluation numbers have rules of their own — how results are sourced, labelled and compared, and when two numbers can sensibly sit side by side. Those are set out in the benchmark methodology.