Google launches double-blind evaluation to prevent AI benchmark contamination

Google has announced the first double-blind evaluation mechanism for cutting-edge proprietary AI models. It keeps external assessments inside an encrypted 'black box' to stop models from seeing test questions in advance and optimizing their performance. This helps make sure the evaluation results are honest. The method solves a big problem in high-risk evaluations: either the evaluator shares test prompts or the model owner shares model weights, both risking privacy leaks.

Google worked with Singapore's AI Safety Institute, OpenMined, AVERI, and MLCommons to run confidential benchmark tests on Gemini Flash Lite in a privacy-protected environment. They used Confidential Space from Google Cloud's confidential computing products. This uses encrypted verification to keep evaluation data and the model's weights private from each other. So the evaluator can't see Gemini's weights, and Google can't see the test prompts. This stops benchmark contamination and keeps sensitive data safe.

This is especially important for high-sensitivity evaluations by cybersecurity or government agencies. It lets independent groups test advanced AI models without giving up data control or security. Google hopes this trial opens a new direction for model oversight and helps build AI systems that are safer, more reliable, and more trusted by the public.