AI benchmarks have a cheating problem that contracts never fully fixed. Either the lab sees the questions, or the evaluator sees the model.
On August 27, Google DeepMind said it ran what it calls the world’s first double-blind evaluation of a proprietary, frontier-class model. Partners: Singapore’s AI Safety Institute, OpenMined, AVERI, and MLCommons. The model under test was a Gemini Flash Lite variant. TechRepublic dated the wider desk writeup to August 28.
Neither side peeks
DeepMind’s framing is simple. Historically you traded one secret for another. Hand the lab your prompts and you risk contamination. Hand the evaluator the weights and you risk the IP. Double-blind, in their design, uses Google Cloud Confidential Space so the evaluator cannot see Gemini’s weights and Google cannot see the evaluator’s test prompts.
TechRepublic adds the hardware line I am keeping: a Google Cloud A3 Confidential VM with Intel TDX host-memory encryption and an NVIDIA H100 Confidential GPU, plus remote attestation to check the software environment. AVERI used reserved prompts from MLCommons’ AILuminate safety family. Singapore AISI ran its own confidential prompts aimed at harmful content in a Singapore context.
The method is public. The scores are not.
This is the part a lot of headlines will skip. DeepMind’s post and the desk coverage describe the architecture. They do not publish how Gemini Flash Lite scored. I am not inventing a leaderboard from silence.
TechRepublic also flags the residual trust DeepMind’s own technical report admits: some proprietary inference code could not be fully inspected, individual Confidential Space builds were not independently reproducible, and Google services signed and verified the attestation report — so Google stays in the verification path. MLCommons cautioned that technical secrecy is not enough without legal protections and careful benchmark stewardship.
Why I care about the plumbing
Benchmark contamination is not a vibe. If the model already saw the test, the score is partly memory. DeepMind’s student-exam metaphor on the blog is the right one. Zero-logging promises and contracts help. Cryptographic boxes are a harder claim — if they hold, and if they scale past a single GPU.
Independent writeups note the current pilot fits models in the Flash Lite class on one H100 Confidential GPU. Bigger frontier stacks need multi-chip confidential setups that are not production-ready yet. So this is a method story, not “Gemini aced a secret test.”
If you buy AI systems on leaderboard screenshots, ask the boring questions: who owned the prompts, who graded the outputs, what was disclosed, and which parts still required trust in the lab. DeepMind just made those questions sharper. It did not retire them.
