Hugging Face, an open-source platform hosting thousands of AI models and datasets, is under scrutiny after a study revealed its safeguards do not stop the creation of non-consensual intimate imagery. The findings, released by European nonprofit AI Forensics, show the platform’s weak moderation allows tools that generate sexual deepfakes, including those targeting minors.
Models complied with explicit requests
AI Forensics tested the nine most popular image-editing models on Hugging Face’s Spaces, a cloud feature where users can host and demo AI tools. Using an AI-generated image of a woman, it asked the models to recreate the same pose and face but topless. Seven of the nine models produced an undressed version without refusal.
The nonprofit then ran its own image-editing model on Spaces, logging user prompts without generating images. Over a week, it recorded 1,081 submissions. 73 percent were sexual in nature, even though the model was not tagged for NSFW or adult tasks. A total of 83 percent of the requests wanted to undress the person in the image uploaded with them, with 95 percent of the image subjects being women. Worse still, 6.7 percent of the sexual requests targeted a minor.
Related: iPhone SOS mode fix explained simply
Nearly no moderation in place
AI Forensics discovered that only 3 percent of the Spaces it had audited had any output moderation. While Hugging Face has policies that prohibit non-consensual sexual images, the report states it “found virtually no safeguards” preventing tools that can generate them from being uploaded to Spaces and from being used for that purpose.
The study comes as the EU and UK prepare to ban “nudification” apps—tools designed to digitally remove clothing from images. Hugging Face’s position as a hub for open-source AI development places it at the heart of discussions about balancing innovation with ethical limits. Its community-driven approach lets users share and improve tools, but that openness also creates risks when oversight is minimal.
For victims of deepfake abuse, the damage goes beyond the initial violation. Once an image is created, it can spread across forums, social media, and messaging apps, making removal difficult. The report indicates that without stronger controls, platforms like Hugging Face may unintentionally speed up this harm.
The company has not addressed the study’s findings publicly. The nonprofit conducted its research as of June 25, 2026, but did not say whether Hugging Face has made changes since then. While its terms of service prohibit illegal content, the study argues those rules are rarely enforced.
Related: NVIDIA Forms Open Secure AI Alliance for Cyber Defense
The situation reflects a challenge in open-source AI: keeping tools accessible while preventing misuse. Unlike commercial platforms, Hugging Face depends on community reporting and automated filters, which the report shows are easy to bypass. The absence of real-time moderation means harmful tools can stay active for long periods, even after being reported.
AI Forensics suggests Hugging Face add stricter pre-screening for models in sensitive categories and require output filters for image-editing tools. It also urges clearer reporting options and faster takedowns. Whether the platform will act on these recommendations is uncertain.
Security concerns like these highlight the need for better safeguards in AI development.
