Elon Musk’s X Corp. sues California over election deepfake law

Elon Musk’s X Corp. is suing the California attorney general over a recently signed law requiring large online platforms to delete or label audio or video deepfakes related to elections. 

The legislation, AB 2655, was part of a package of measures signed into law by Governor Gavin Newsom in the lead-up to Election Day meant to regulate deceptive material in advertising and on social media platforms. 

A companion law prohibiting content creators from sharing election-related advertising or other “election communication” with artificially manipulated audio or video was blocked in October by U.S. District Judge John A. Mendez, who wrote the legislation “acts as a hammer instead of a scalpel.”

Musk took to X to praise that decision, and now the billionaire is seeking to strike down California’s other attempts to reign in deepfakes, saying in the lawsuit the “system will inevitably result in the censorship of wide swaths of valuable political speech and commentary.” 

California’s unconstitutional law infringing on your freedom of speech has been blocked by the court. Yay! https://t.co/dRihErTsJ9

— Elon Musk (@elonmusk) October 3, 2024

The law specifically requires large online platforms, defined as having more than 1 million users in California, to create a mechanism through which users could report suspected deepfakes related to election officials, the electoral process or political candidates. The platforms are required to respond to such requests within 72 hours, and if the content is indeed found to be digitally manipulated they are required to either remove it or label it as deceptive. 

The law exempts material that is explicitly parodic. 

Candidates, elected officials, the attorney general and other state attorneys are also allowed to seek injunctive relief if they deem a platform has failed to respond adequately to reports of “materially deceptive content.”

According to X, such a law violates both the First and Fourteenth Amendments and “has the effect of impermissibly replacing the judgments of covered platforms about what content belongs on their platforms with the judgments of the State.”

The legislation, they claim, incentivizes platforms to remove or label suspect content but does not provide recourse to users or content creators who believe they have been censored unfairly. 

“The government is involved in every step of that system: it dictates the rules for reporting, defining, and identifying the speech targeted for removal; it authorizes state officials (including Defendants here) to bring actions seeking removal; and, through the courts, it makes the ultimate determination of what speech is permissible,” X said in the lawsuit.

“It is difficult to imagine a statute more in conflict with core First Amendment principles.” 

Before their passage, free speech advocates like the American Civil Liberties Union voiced their opposition to the laws, saying that concerns around artificial intelligence’s ability to create manipulative content don’t justify encroaching on freedoms of expression. 

“Speech that is false, confusing, or which presents content that some find abhorrent, nevertheless maintains its constitutional protections as a driver of free discourse,” the group said. 

“This remains so no matter what the technology used to speak. Unfortunately, the provisions of AB 2655 as currently drafted threaten to intrude on those rights and deter that vital speech.”    

In the lead-up to the bill’s passage, Gov. Newsom specifically pointed to a video created by X user “Mr. Reagan” that used cloned audio of Vice President Kamala Harris saying she was a “diversity hire.” 

“Manipulating a voice in an ‘ad’ like this one should be illegal,” Newsom said, pledging to sign a bill “in a matter of weeks to make sure it is.”

Manipulating a voice in an “ad” like this one should be illegal.

I’ll be signing a bill in a matter of weeks to make sure it is. pic.twitter.com/NuqOETkwTI

— Gavin Newsom (@GavinNewsom) July 29, 2024

That X user, Christopher Kohls, filed a lawsuit against the state on the day they were passed on the grounds that his account is satirical in nature and the laws criminalize “computer-generated parody” and “political satire.”

In his ruling deciding in Kohls’ favor, Judge Mendez wrote: “YouTube videos, Facebook posts, and X tweets are the newspaper advertisements and political cartoons of today, and the First Amendment protects an individual's right to speak regardless of the new medium these critiques may take.” 

U.S. intelligence agencies warned in the lead-up to the presidential election that Russian influence operations were spreading deepfakes — sometimes made with AI — to denigrate the Harris campaign.