The National Institute of Standards and Technology unveiled AI Technology Evaluation, or AITE, on 27 July — a testbed run by its Information Technology Laboratory in which models are scored on data their developers have never seen and cannot obtain.

The problem it addresses

Public benchmarks leak. Once a test set exists on the internet it eventually appears in training data, at which point a high score measures memorisation rather than capability — and there is no way to tell the two apart from the outside. Sequestered evaluation is the only structural fix: data providers must supply original, publicly inaccessible datasets, and the model meets them once, blind.

The initial scope

Three tasks to start, all image analysis with large vision-language models: quantum science, genomics and public safety. NIST lists a July 2026 kickoff and evaluation plan release, with the evaluation period beginning in August 2026. Participant login is listed as coming soon.

What it is not

Framings suggesting NIST will now certify AI models, or that this constitutes a federal AI safety standard, are wrong on every count. AITE is voluntary, confers no certification, and has no regulatory effect. It covers three narrow image-analysis tasks and does not evaluate frontier-model safety at all — no cyber capability, no biological risk, no alignment. Its scope is closer to a scientific instrument-calibration exercise than to a safety regime.

Why it matters anyway

Because in the absence of a federal AI statute, evaluation infrastructure is where American policy is actually being built. Every regulatory proposal in circulation — including the mandatory pre-release testing Anthropic's chief executive asked for the same day — presumes the existence of tests that vendors cannot game. Right now those tests largely do not exist, and vendor self-reported benchmarks are the default currency. A government body holding blind data is the precondition for making any of those claims contestable.

Dating caveat

NIST's own page gives month-level granularity only; the 27 July launch date rests on a single trade publication's report.