Case Summary and Outcome

An American District Court granted a preliminary injunction against a piece of legislation which prevented the sharing of “deepfake” videos without a prominent textual disclaimer during election periods. A content creator, who had published a “digitally altered” video of a presidential candidate, sought to prevent the enforcement of the law on the grounds that it infringed the right to freedom of expression. The Court found that the law was an overly broad, content-based regulation that unconstitutionally restricted protected political speech, including parody and satire. It emphasized that even deliberately false statements about government officials are constitutionally protected, and found the law’s disclaimer requirements unduly burdensome compelled speech. The Court acknowledged that the risks of artificial intelligence and deepfakes were significant, but held that the law’s broad restrictions acted as a “hammer instead of a scalpel,” unconstitutionally hindering free expression and democratic debate. [p. 20]

---

Facts

On July 26, 2024, Christopher Kohls, an American content creator who operates under the name “Mr Reagan” on various social media platforms including YouTube, Facebook, and X (formerly Twitter), posted a parody video of US presidential candidate Kamala Harris’s campaign ad. Kohls creates humorous content that comments on and satirizes political figures, maintaining a significant following with approximately 80,000 followers on X and 360,000 subscribers on YouTube. The Harris video was clearly labeled as a parody and included a disclaimer noting that “Sound or visuals were significantly edited or digitally generated” and featured AI-generated content that mimicked Harris’s voice, combining it with clips from her actual campaign videos. This form of video is colloquially referred to as a “deepfake”, where a person’s image has been digitally altered. In the parody, the AI-generated Harris voice made satirical statements, including claiming she was selected as the Democratic Party candidate because President Joe Biden “finally exposed his senility” at a presidential debate with Republican Party candidate Donald Trump on June 27, 2024. It also included satirical commentary about Harris being a “diversity hire” and mocked her speaking style, particularly her statements about “significance.”

The AI-generated Harris said: I, Kamala Harris, am your Democrat candidate for president because Joe Biden finally exposed his senility at the debate. Thanks Joe. I was selected because I am the ultimate diversity hire. I’m both a woman and a person of color. So if you criticize anything I say, you’re both sexist and racist. I may not know the first thing about running the country, but remember, that’s a good thing if you’re a deep-state puppet. I had four years under the tutelage of the ultimate deep state puppet; a wonderful mentor, Joe Biden. Joe taught me rule number one: carefully hide your total incompetence. I take insignificant things and I discuss them as if they’re significant. And I believe that exploring the significance of the insignificant is itself significant.

The video gained significant traction when Elon Musk shared it on X, describing it as “amazing.” Musk’s repost garnered over 100 million views. In response, on July 28, 2024, the Governor of California Gavin Newsom posted a photo of a news story about Musk’s retweet on X and said that “[m]anipulating a voice in an ‘ad’” like Kohls had done “should be illegal”. [p. 4]

Subsequently, on September 17, 2024, the California legislature passed two bills, including AB 2839, “Elections: deceptive media in advertisements”. AB 2839’s focus was on election-related content that was “materially deceptive” – defined as “content that has been ‘digitally created or modified’ such that it ‘would falsely appear to a reasonably person to be an authentic record of the content depicted in the media’  [p. 5] – and stated that “a person, committee or other entity shall not … with malice, knowingly distribute an advertisement or other election communication containing material deceptive content” of a candidate for office “portrayed as doing something that the candidate did not do or say if the content is reasonably likely to harm the reputation or electoral prospects of a candidate.” [p. 3-4] The legislation also prohibited the distribution of materially deceptive content if that would be “reasonably likely to falsely undermine confidence in the outcome of one or more election contests”. [p. 5] The legislation’s restrictions applied for 120 days before an election and (except for depictions of candidates) for 60 days after the election. The Act did include a “safe harbor for candidates portraying themselves as long as these videos are labelled with a disclosure ‘no smaller than the largest font size of other text appearing in the visual media’.” [p. 4] The “safe harbor” also exempted satire and parody with the same requirement of a disclosure “no smaller than the largest font size of other text appearing in the visual media”.  [p. 5]

On September 17, 2024, Kohls filed an application for an injunction preventing the State of California from enforcing AB 2839. He submitted that the Act violated his right to freedom of expression as it “discriminates against political speech based on content and is overbroad”. [p. 7]

The First Amendment to the US Constitution states: Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.

Article 1, section 2(a) of the California Constitution states: “[e]very person may freely speak, write and publish his or her sentiments on all subjects” and “a law may not restrain or abridge liberty of speech”.

---

Decision Overview

John A. Mendez of the United States District Court for the Eastern District of California heard the application for a preliminary injunction. The central issue for the Court’s determination was whether Kohls had satisfied the requirements for the injunction, namely that he was “likely to succeed on the merits”, was “likely to suffer irreparable harm” if the injunction was not granted, that the “balance of equities tips in [his] favor”, and that the injunction is in the public interest”. [p. 6]. 

Kohls argued that AB 2839 violated his First Amendment rights. He argued that the law would allow any political candidate, election official, the Secretary of State, and anyone who viewed his Artificial Intelligence (AI)-generated videos to sue him for damages and injunctive relief during an election period. He submitted that the disclosure requirements for parody videos were unduly burdensome, as they required text to be displayed for the entire duration of a video in a size no smaller than the largest font used in the video, which he argued would make his videos almost unviewable by obstructing the entire frame. He submitted that the law’s restrictions on “materially deceptive content” were overly broad and would infringe on constitutionally protected speech, particularly in the realm of political satire and parody. Kohls maintained that the statute relied on various “subjective terms and awkwardly phrased mens rea,” which had the effect of implicating vast amounts of political and constitutionally protected speech. [p. 9]

The Attorney General of California, Rob Bonta, argued that AB 2839 was constitutional under the First Amendment as a “restriction on knowing falsehoods that fall outside of the category of false speech protected by the First Amendment” as established in the case of United States v. Alvarez. He submitted that the legislation’s restrictions fell within the possible exceptions recognized in Alvarez for “lies that involved legally cognizable harm” as it addressed tangible harms to electoral integrity. Bonta maintained that the law was justified because the California Legislature had a compelling interest in protecting free and fair elections and that the statute met the strict scrutiny standard through its safe harbor provisions for parody and satire. Bonta argued that the burden on plaintiffs like Kohls was minimal and that the balance of equities and public interest factors would only weigh in favor of injunctive relief if Kohls could show a constitutional violation. When faced with the possibility that portions of the law might be struck down, Bonta pointed to AB 2839’s severability clause, arguing that it would allow the Court to salvage portions of the statute that were constitutionally sound. He defended the disclosure requirements for parody videos as reasonable measures to protect electoral integrity, rather than viewing them as unduly burdensome restrictions on speech as Kohls had claimed.

The Court examined the requirements for a preliminary injunction. On the question of whether Kohls was likely to succeed in demonstrating that AB 2839 facially violates the First Amendment, the Court referred to the Moody v. Netchoice case where the Supreme Court had “substituted a less demanding though still rigorous standard for facial challenges”. [p. 7] It added that a facial challenge to a law has two parts and a Court has to determine the law’s scope and then which elements of the law violate the First Amendment; if the unconstitutional elements outweigh the constitutional elements the Court may strike the law down. The Court rejected the Bonta’s argument that the law was similar to restrictions on the publication of defamatory content, noting that it prohibits content that is “reasonably likely” to harm the “reputation or electoral prospects of a candidate” (emphasis in original). [p. 8] The Court found that AB 2839 went beyond defamation because it did not require actual harm and described the phrase “electoral prospects” as “amorphous”. [p. 8] It also found that the definition of “materially deceptive content” encompassed all “deepfakes” and so any “deepfake” could be prohibited even if it did not impact on a candidate’s electoral prospects. The Court described the terms “elections official”, “voting machine, ballot, voting site or other property or equipment”, “reasonably likely” and “undermine confidence” as “lacking any objective merit and being difficult to ascertain” and commented that there are various types of conduct that could undermine confidence in an election. [p. 8] It added that “any digitally altered content” could be deemed harmful under the law but that much of this information may not alter the election result or undermine confidence. The Court accepted Kohls’s description of the law as relying on “‘various subjective terms and awkwardly-phrased mens rea,’ which has the effect of implicating vast amounts of political and constitutionally protected speech”. [p. 9]

The Court rejected Bonta’s argument that the speech prohibited by AB 2839 fell within the category of “legally cognizable harm” as set out in the Alvarez case and held that the law “unconstitutionally suppresses broader areas of false but protected speech”. [p. 9-10] It stressed that the case of New York Times v. Sullivan had confirmed that false statements – “deliberate lies (said with ‘actual malice’)”- are protected under the First Amendment and that civil penalties for criticism of the government “have no place in our system of governance”. [p. 10] The Court specified that these principles apply to a “new technological age when media may be digitally altered”. [p. 10] 

In examining whether the law would pass strict scrutiny and was narrowly-tailored, the Court found that AB 2839 was a content-based restriction on speech as it related to political or electoral content. With reference to the cases of Reed v. Gilbert and Nat’l Inst. of Fam. & Life Advocs, the Court noted that content-based restrictions are presumptively unconstitutional, justifiable only if the government can demonstrate that it is narrowly tailored to “serve compelling state interests”. [p. 11] A content-based restriction requires strict scrutiny of its constitutionality, and the Court must determine whether there were no other less restrictive means for the government to achieve its objectives. The Court gave “substantial weight” to the fact that the Californian government had an interest in protecting free and fair elections but, with reference to the cases of McCullen v. Coakley and FCC v. League of Women Voters of Cal., the Court emphasized the need to protect the “marketplace of ideas” and to ensure robust public debate. It quoted the direction given by the Supreme Court in Whitney v. California that to expose falsehoods and promote truth “the remedy to be applied is more speech, not enforced silence”. [p. 12]

In acknowledging the justification of a “well-founded fear of a digitally manipulated media landscape”, the Court stressed that this does not “give legislators unbridled license to bulldoze over the longstanding tradition of critique, parody and satire protected by the First Amendment”. [p. 12] It compared the modern forms of expression on social media to newspaper advertisements and political cartoons of the past and held that the First Amendment “protects an individual’s right to speak regardless of the new medium these critiques may take”. [p. 12] The Court noted that there are existing legal remedies, such as defamation, privacy and copyright actions, that could be used to protect those affected by “artificially altered depictions on the internet”. [p. 12]  Accordingly, the Court held that AB 2839 prohibited content that, although false, is constitutionally protected. 

The Court discussed the danger of the state seeking to regulate truth, and quoted Justices Breyer and Alito in Alvarez who had said that “[t]here are broad areas in which any attempt by the state to penalize purportedly false speech would present a grave and unacceptable danger of suppressing truthful speech”. [p. 13] It added that it is particularly “perilous” for the state to seek to be the “arbiter of truth” in the political sphere, and noted that “[w]hen political speech and electoral politics are at issue, the First Amendment has almost unequivocally dictated that Courts allow speech to flourish rather than uphold the State’s attempt to suffocate it”. [p. 13]

Accordingly, the Court held that AB 2839’s broad prohibitions on speech meant that Kohls would be likely to succeed on a First Amendment facial challenge to the Act. 

In determining whether AB 2839’s requirement that every video contain a disclaimer alerting a viewer to the satire or parody was unduly burdensome, the Court acknowledged Kohls’s concern that the large font would make the video “almost unviewable”. [p. 14] It described the obstructiveness of this requirement as concerning, noting that “parody and satire have relayed creative and important messages in American politics”. [p. 14] It quoted the Supreme Court case of Hustler Magazine v. Falwell, which had remarked that “[d]espite their sometimes caustic nature, from the early cartoon portraying George Washington as an ass down to the present day, graphic depictions and satirical cartoons have played  prominent role in public and political debate.” [p. 14] The Court accepted that the content impacted by AB 2839 was not commercial speech and found that the disclosure requirement would force “parodists and satirists to ‘speak a particular message’ that they would not otherwise speak” and held that this constitutes “compelled speech that dilutes their message”. [p. 15] It added that even if some content affected by the law was commercial speech – which makes it subject to a lower standard of scrutiny – AB 2839 was still overly burdensome because the text requirement for videos would take up much of the screen which is unreasonable.

Accordingly, the Court held that AB 2839’s requirement that parody or satire videos have text (as large as the largest font used in the video) alerting the views to the nature of the video was unduly burdensome and not sufficiently narrowly tailored and was therefore unconstitutional.

In looking at whether AB 2839 infringed the Californian Constitution’s protection of free speech, the Court acknowledged that courts have interpreted the “Liberty of Speech” clause as providing broader protections than the First Amendment. It accepted that the Californian state right “is at least as protective as its federal counterpart” and so if Kohls were likely to succeed on a First Amendment facial challenge then he would likely also succeed on the state claim. Accordingly, the Court held that AB 2839 was unconstitutional in terms of California’s free speech provision and that Kohls was likely to succeed on this claim and so had met the requirement for an injunction.

The Court found that Kohls had proven the other requirements for an injunction, namely that he would suffer irreparable harm without the injunction, the balance of equities favored an injunction, and it would be in the public interest. With reference to the case of Klein v. City of San Clemente, the Court confirmed that as a First Amendment infringement constitutes irreparable harm, and that as Kohls had demonstrated that his right would be infringed, he had satisfied that element of the test. The Court held that as Bonta had not addressed the questions of balance of equities and public interest it was “not persuaded that a balance of equities or public interest analysis does not weigh in favor of a preliminary injunction”. [p. 18] The Court accepted that California may suffer some hardship without AB 2839 and that it had an interest in “preserving election integrity and addressing artificially manipulated content” but held that this was “minimal” when weighed against a First Amendment infringement and stressed that there is a significant public interest in upholding constitutional rights. [p. 18]

The Court held that the only part of AB 2839 that was constitutional and so could be severed was the “audio-only requirement”. This requirement involved a “verbal disclosure at the outset and conclusion of a recording combined with interspersed disclosures in two-minute intervals” and the Court held that this was “on its face reasonable and not unduly burdensome”. [p. 19]

In conclusion, the Court recognized that AI and “deepfakes” pose significant risks and that California has an interest in protecting election integrity; it commented that it “does not enjoin the state statute lightly”. [p. 20] However, it described AB 2839 as a “hammer instead of a scalpel” which “hinders humorous expression and unconstitutionally stifles the free and unfettered exchange of ideas which is so vital to American electoral debate”. [p. 20-21] The Court emphasized that while technology may advance, the basic principles of the First Amendment do not vary, citing Brown v. Entertainment Merchants Assn. Accordingly, the Court granted Kohls’s application for a preliminary injunction and enjoined Bonta and other state officials from enforcing AB 2839, except for the audio-only portion of the legislation it had severed as being constitutional.

---