Manhattan DA takes down 12 AI deepfake porn sites

Twelve AI deepfake sites were taken offline this week after a New York district attorney seized their domain names for violating laws around disseminating non-consensual videos. 

Manhattan District Attorney Alvin Bragg held a press conference on Monday touting the takedown of the sites, which hosted AI-generated videos of more than 1,200 people. The sites allowed users to use the faces and bodies of real people to create illegal pornography.

Bragg noted during the press conference that while many of those who appeared on the site are celebrities, they know of domestic violence cases where partners threatened to release deepfake images of victims. 

He added that they have spoken to victims who did not know anything could be done about the sites and he urged more victims to come forward. He declined to say whether arrests or charges will be pursued, only telling reporters that the investigation is ongoing. 

Bragg said several people under investigation used real photos and videos to create “hyper-realistic images and videos of those individuals engaging in sexual conduct.”

“These horrific violations of privacy follow victims into their careers and personal lives and take an immense toll on emotional and mental wellbeing,” Bragg said. “Let me be clear – my office does not tolerate sexual exploitation in any form and will not stand by when businesses profit from the publication and dissemination of non-consensual sexual images and videos, AI-generated or not.”

The sites, the names of which were not released, explicitly marketed themselves as deepfake pornography platforms. 

Bragg said they would continue to monitor the sites and try to take down any attempts to revive the platforms on new domains. 

Governments and tech giants like Microsoft have sought to address the proliferation of AI deepfake sites over the last two years. Microsoft filed a lawsuit last year against six developers who used the company’s Azure OpenAI services to create “non-consensual intimate images of celebrities and other sexually explicit content.”

Earlier this year, the European Commission announced an inquiry into social media platform X after its Grok AI tool was used to create sexual images of a minor.