Thorn and ten AI firms adopted child-safety design rules

The design

In April 2024, the child-safety nonprofit Thorn and the responsible-technology coalition All Tech Is Human published 'Safety by Design for Generative AI: Preventing Child Sexual Abuse,' a set of principles developed with, and publicly committed to by, Amazon, Anthropic, Civitai, Google, Meta, Metaphysic, Microsoft, Mistral AI, OpenAI and Stability AI. Thorn's own announcement, whose live page carries a July 2024 timestamp but is marked 'Originally published April 2024' in its own text, and All Tech Is Human's own blog post, dated 23 April 2024 and written by its founder David Polgar, both name the same ten companies as signatories.

What the evidence says

Thorn's own post states the companies 'have publicly committed to Safety by Design principles' and, separately, 'agreed to transparently publish and share documentation of their progress acting on the principles,' which is a stated commitment to future reporting, not evidence that such reporting has since occurred; this entry did not verify any individual company's subsequent progress disclosure. Thorn also states, citing its own sourcing, that 'more than 104 million files of suspected CSAM were reported in the US' in 2023, a figure presented as context for the scale of the underlying problem rather than as a measurement of AI-generated content specifically.

What it asks of people

The principles ask signatory companies to take design-stage, deployment-stage and post-deployment steps against AI-generated CSAM, including the kinds of dataset curation and detection tooling the two organizations' posts describe in general terms; neither post published the full technical text of the underlying paper, so this entry cites the organizations' own public summaries rather than the complete principles document, which was not separately opened. The commitment structure asks the public to trust self-reported 'progress updates' from named companies rather than an independent audit process, a design choice both organizations state plainly rather than obscure.

Privacy and safeguards

The initiative is specifically scoped to preventing the creation and spread of AI-generated child sexual abuse material and other sexual harms against children, not to companion-app privacy practices generally; All Tech Is Human's post frames the harm context as including 'harmful image creation and sharing across middle and high schools,' naming a specific category of harm the principles target. Neither organization's post states an enforcement mechanism beyond the companies' own pledged transparency, which this entry notes as the boundary of what the commitment currently guarantees.

• Which of the ten named companies has published a progress-update document since April 2024, and what does it cover?
• Does the 104-million-file figure Thorn cites include any AI-generated content specifically, or is it the broader annual total?
• Has any signatory's later AI companion or persona product been evaluated against these principles by an outside party?

The April 2024 announcement documents a named, dated commitment by ten companies to a shared set of design principles against a specific, serious harm; it is a pledge with a self-reporting structure, and its record here should not be read as evidence that any signatory's product has since been independently confirmed to meet the principles it agreed to.