OpenAI Launches CSAM Detection Tool with 48% Fewer False Positives

OpenAI has launched a child sexual abuse material (CSAM) detection tool, available free to vetted organizations through its API. The tool includes a new image classifier that spots visual signs of abuse, helping review teams find new images and prioritize high-risk content. The National Center for Missing & Exploited Children supported training, evaluation, and deployment.

The tool cuts false-flagged images by 48% compared to existing measures, and flagged images are about twice as likely to actually contain abuse. Tests show one in four flagged cases ends up banned.

OpenAI will also add warnings in ChatGPT. When users try to seek abuse material, the system interrupts the action and directs them to help, with the account suspended for 24 hours. Repeated violations may lead to longer bans, permanent suspension, or reports to authorities. OpenAI has partnered with Anthropic and Google to commission new guidelines on handling child sexual exploitation disclosures in AI systems, recently published by Tech Coalition.

SponsoredDomoAI
AI Tools Mentioned

Comments

Loading comments…