How we grade the evidence

The system

Two marks, not one

A single score can't describe research honestly. "Nobody has studied this" and "four large trials found nothing" are opposite situations, and a one-dimensional scale collapses them into the same low number. So we grade on two independent axes.

Evidence strength — four dots — measures how good the research base is. It rises with the quantity and quality of the studies, regardless of what those studies found. A well-refuted claim scores high on strength.

Verdict — one word — states what that research concluded: supported, mixed, or not supported.

A third element joins them when it applies. Some compounds carry a documented safety concern that has nothing to do with whether they work, and the two axes above cannot express it — so a safety flag sits alongside them:

Axis one

Evidence strength

Assessed against the specific claim, not the compound in general. The same compound can hold different strength grades for different outcomes.

Axis two

Verdict

What the qualifying trials concluded. Assigned only where strength is Emerging or above — at Preliminary there is no human evidence to draw a verdict from, and the verdict reads Untested in humans.

Scope

What the scale does not grade

Both marks describe evidence about outcomes in humans. The strength axis is built from counts of human trials, participants, and independent replications. Point it at anything else and it returns a number that looks meaningful and is not.

So two categories of claim are reported in our breakdowns but never carry marks:

• Mechanism claims — statements about how a compound works: receptor binding, enzyme activation, a biochemical pathway. These are settled in laboratories, not trials, and have no participants to count.
• Preclinical claims — findings in cell culture or animal models, however extensive. A large animal literature raises a question worth answering; it does not answer it.

Where such a claim matters to a compound's story we describe it plainly and say what the laboratory evidence shows — refuted in vitro, demonstrated in rodents, disputed — without dots and without a verdict pill.

When trials disagree

Conflicting results are the normal state of this literature, not an exception. We resolve them in a fixed order, and we never settle a disagreement by counting how many papers landed on each side — a large, well-run trial and a tiny flawed one are not one vote each.

1. A meta-analysis decides. If a published systematic review or meta-analysis addresses the claim, its conclusion sets the verdict. If several exist and disagree, we prefer the most recent that includes the largest trial set.
2. Absent one, weight by size and quality. Larger trials outweigh smaller. Placebo-controlled outweighs open-label. Independently funded outweighs manufacturer-funded. Pre-registered outweighs not.
3. Look for the variable that explains the split. If the disagreement tracks dose, formulation, population, or duration, we say so explicitly and grade the narrower claim that the evidence actually supports — not the broad one it doesn't.
4. If it still won't resolve, the verdict is Mixed and strength is capped at Moderate. We say plainly that the question is open rather than picking the answer we find more interesting.

Confidence is not magnitude

This distinction causes more confusion than anything else in health reporting, so we separate the two explicitly in every breakdown.

Every breakdown states the effect in the units the trials used, alongside the smallest change a person would plausibly notice. Percentages without baselines, and relative risks without absolute ones, are not reporting — they're decoration.

Third element

The safety flag

Both marks describe evidence about whether something works. Neither says anything about whether it is safe, and for some compounds that is the more important question by a wide margin.

This produced a real failure. A compound whose development was terminated over animal carcinogenicity, and which one national regulator prohibits from sale outright, still displayed as Preliminary · Untested in humans — marks a reader could reasonably take to mean nobody knows, proceed carefully. The marks were accurate. The impression was wrong.

So a safety flag may appear beside the two marks. It is not a third grade. It records what has been documented and who documented it, nothing more.

Where more than one applies, the breakdown shows the one carrying the greatest institutional weight, and the safety section describes the rest. Where none applies, no flag is shown — and the absence of a flag is not a claim of safety, only the absence of a documented concern meeting these criteria.

Safety is reported, never graded

The flag points at something; this section is where it gets explained. Grading safety would imply a judgment about whether a person should take something, which is not a judgment this publication makes. Every breakdown carries a required Safety and Unknowns section covering five points.

• Adverse events reported in the trials we cite, with the rates they occurred at
• Longest human exposure studied — the duration of the longest qualifying trial, stated plainly
• Known interactions with medications or conditions where the literature documents them
• Regulatory status — approved, unapproved, sold as a research chemical, or banned in sport
• What has not been characterized — stated explicitly rather than left as white space

What counts as a source

Marks are assigned from primary literature. We read the studies themselves, not coverage of them.

• We use: peer-reviewed papers indexed on PubMed and PMC, registered trial records with posted results, systematic reviews and meta-analyses, and regulatory assessments.
• We use with caveats, always labelled: preprints that have not cleared peer review, conference abstracts, and trials with manufacturer or undisclosed funding. These can support a verdict but never lift strength above Emerging on their own.
• We do not use: vendor product pages, press releases, podcast claims, influencer protocols, or forum reports as evidence for any mark.

Every factual claim links to the paper it came from. If a claim carries no citation, treat it as our opinion and weigh it accordingly.

Marks change

Both marks are a snapshot of the literature on a date, not a permanent verdict. Every breakdown carries the date its marks were last reviewed.

Changes are logged publicly. We would rather be visibly wrong and corrected than quietly wrong.

What we don't publish

These limits are permanent and are not commercial decisions. They keep this a publication about research rather than a guide to self-experimentation.

• No dosing protocols for human use. We report the doses used in published trials, in the course of describing those trials. We do not tell readers what to take.
• No vendor affiliate links. We take no commission on the sale of any compound we cover, ever. Affiliate revenue and honest grading cannot coexist.
• No medical advice. Nothing here substitutes for a clinician who knows your history.
• No sponsored breakdowns. No company can pay for coverage, for a mark, or for a revision to one.

Corrections

When we get something wrong, the correction appears at the top of the affected breakdown with the date and a plain description of what changed. We don't silently edit. If you spot an error — a misread result, a bad citation, a mark you think is indefensible — write to corrections@compoundcodex.org and we will look at it. If you want the argument on the public record instead, file it as a formal challenge.

Revisions to this rubric

The rubric is versioned and its changes are logged here. A publication that grades other people's evidence should be willing to show its own working.