It is increasingly difficult to determine if content is created or altered by humans or artificial intelligence, undermining confidence in what we read, watch and hear. The issue is particularly painful for the news media.
Trust in news organizations – once regarded as one of the main sources of reliable information – is at an all-time low. The media industry’s chief source of revenue has already been hobbled by the tech sector, which grabbed most of the advertising revenue and normalized the giving away of content for free. Now it is being threatened by AI models that are ingesting content without attribution, consent or compensation.
“For the last two years the content industry has been a crime scene,” says Erik Svilich, CEO of Encypher and Co-Chair of the text working group of the Coalition for Content Provenance and Authenticity (C2PA) initiative, which seeks to add provenance to media.
For example, a lawsuit filed by the New York Times against OpenAI and Microsoft in December 2023 contends that millions of articles published by The Times were used to train automated chatbots that now compete with the news outlet as a source of reliable information. The suit alleges that the AI-generated outputs sometimes produce near-original copies of Times articles, that OpenAI models can generate expressive content of Times articles at no cost, making readers less likely to visit its website; that Large Language Models also allows users to bypass The New York Times’ paywall ,and that the AI model sometimes makes up stories and wrongly says they came from the Times, hurting the paper’s reputation. The suit does not include an exact monetary demand. But it says the defendants should be held responsible for “billions of dollars in statutory and actual damages” related to the “unlawful copying and use of The Times’s uniquely valuable works.”
Proving the provenance of digital text is even more challenging than video or photos, making it difficult and expensive for publishers to prove their case in court, says Svilich. A new CP2A industry standard based on cryptography released on January 8 could change that.
Until now publishers and content providers have had to rely on statistical AI detectors to find proof that AI has siphoned or altered text without authorization, a method that has an accuracy rate of only 26%. It the equivalent, Svilich says, of “dusting for fingerprints when the culprit wore gloves.”
The solution is to change the nature of the evidence, he says, by shifting from statistical evidence to mathematical proof of origin.
Encypher, the Portland, Oregon-based startup that Svilich founded, says it has developed a cryptographic method that will allow content producers to embed what it describes as a unique unforgeable digital fingerprint, not just in every article but in every sentence. EncypherAI’s patent-pending technology, which enables invisible metadata embedding in text using zero-width characters, combines technologies that have existed for a long time in new ways, says Svilich.
Encypher combines Unicode Variation Selectors-zero-width, non-printable characters for invisible embedding, cryptographic signatures compatible with the C2PA trust list, and C2PA manifest structure for provenance metadata, he says “We always embed a full C2PA manifest at the end of content, with a streamlined approach for sentence-level tracking throughout. The innovation is binding these together at sentence-level granularity while maintaining full C2PA compliance.”
The Portland-based startup is aiming to help media companies encrypt provenance information in text and establish a framework for licensing the content to AI companies. It also wants to help thousands of smaller media outlets to band together to encrypt their content and license it as a group to increase their negotiating power.
Encypher is open sourcing some of the key elements of its technology and incorporating it into a standard it is co-developing with the members of C2PA, an organization which includes the industry’s largest media and tech companies.
The C2PA standard provides the open baseline, explains Svilich. “Our patent-pending enhancements (sentence-level tracking, formal notice infrastructure) extend it for enterprise use cases,” he says.
Incorporating parts of its technology into the CP2A standard will ensure that its cryptographic approach to provenance scales, he says. “Even if we disappear tomorrow this technology will still be around and supported. By establishing a shared protocol for hard binding provenance directly to text we can create a transparent and more equitable digital economy.”
An Alternative to Generative Watermarking
Enchyper’s technology – and the C2PA standard – aim to strengthen current attempts to determine providence of AI content which rely on so-called watermarking.
The term watermarking comes from a process in which an identifying image or pattern in paper appears when viewed by transmitted light to prove the providence of a document. Generative watermarking techniques for the AI age modify the training process, the inference process, or both so that an artifact of the model – the text, audio, and video they generate – embeds some identifying information of the model from which it originates. This way a model operator, or potentially the consumer of the content themselves, can determine whether an artifact came from by checking for the presence of the watermark.
Over the next decade generative watermarking technologies could evolve from optional technical safeguards to important components of digital trust infrastructure, according to the Dubai Future Foundation, which contributed to a report that covered the topic compiled by the World Economic Forum in collaboration with the open science publisher Frontiers. But for that to happen many hurdles need to be overcome.
Text-based watermark technologies, such as Google DeepMind’s SynthID technology, take advantage of the fact that there are thousands of words in each language that can be randomly substituted by others They work by including a narrow and specific subset of such words throughout AI-generated text that seems natural but is distinct from the more random word choices a human writer might make. This results in an AI-specific textual “fingerprint” that increase detection probability but still produce a confidence score, not a proof.
They can also be hacked by bad actors. If a watermark is embedded in AI-generated content but can be stripped away with little effort, then it becomes impossible to prove whether the data was generated by AI, Hanqing Guo, an assistant professor at University of Hawaii at Mānoa Department of Electrical and Computer Engineering, said in an interview with The Innovator. In practice, there is no perfectly robust watermark, he says. Bad actors might pass the watermarked output through multiple signal-processing steps, feed it into another generative model for re-synthesis, re-record it through an analog channel, or shuffle the content, says Guo, whose recent work includes studies on the robustness of watermarking in generative AI models and analysis of overwriting attacks that compromise neural watermarking. Each of these transformations can gradually erode or destroy the watermark, making detection unreliable, he says.
“Cryptographic provenance is a fundamentally different category,” says Svilich. “Whereas statistical watermarking is like a fingerprint that might smudge, cryptographic provenance is like a notarized signature embedded into the text itself,” he says. “We have binary certainty: either the signature verified or it doesn’t. No false positives or guesswork. That is not to say that encrypted signatures can’t be removed – they could be – but if you are expecting to see a signature from a trusted source and you don’t, you know that it has been tampered with,” he says.
Watermarking expert Sridhar Krishnan, Dean, Faculty of Engineering and Architectural Science, at Toronto Metropolitan University, agrees that cryptography will strengthen watermarks. “Cryptography helps reduce the attack surface, enhancing watermarking by providing keys, authentication, and tamper-resistance,” says Krishnan, who is the topic editor of a new research topic for open science publisher Frontiers in signal processing to explore how watermarking can maintain its integrity across various file formats and promote standardized verification protocols to enhance digital providence.
Convincing AI Companies To Opt In
Another challenge posed by the current watermarking system is that it will only be effective if all the LLM model makers opt in and so far, this is not the case, according to industry experts. While Google has open sourced SynthID it is unclear which LLM providers are actively using watermarking technology.
One of the first encryption-based approaches to generative watermarking was developed in 2022 by Scott Aarosnon for Open AI while he was on leave from the University of Texas in Austin. But Open AI did not deploy it, reportedly out of fear of user blowback. And experts say some LLM providers fear that the ability to race content back to their AI models might leave them open to legal liability.
Without persistent provenance, the plausible deniability of AI foundational model owners is stronger than the proof publishers use, says Svitlich. Content providers must prove:
- The owners of AI foundational models accessed their content
- Ingested it into training data
- They knew it was protected
- They chose to use it anyway
Each step requires expensive forensic analysis, legal discovery, and expert testimony, he says Meanwhile, LLM model makers can simply say: “We didn’t know.”
That argument will no longer cut it once publishers and content owners use the C2PA text provenance standard to embed provenance and licensing terms into their own content because whenever that content appears anywhere, whether for training, use on another website or scraped repositories of data, the cryptographic proof travels with it, says Svilich. “By embedding cryptographic provenance into their content publishers can turn ‘we didn’t know it came from you’ into “provable, willful infringement,” he says, “This alone will help bring AI companies to the negotiation table for content licensing discussions.”
AI foundation model providers are already under regulatory pressure. China has acted to regulate generated content to require watermarking and other regions, such as the European Union, are also developing responses to manage the security and authenticity of digital content.
“Regulatory pressure is one lever, but not the only one to drive adoption,” says Svilich. As publishers start marketing their content at scale, AI providers will need to adopt the technology to interface with the ecosystem, he says. There is a business intelligence driver for them to do so: provenance enables AI companies to understand exactly which content drives model performance, verify quote accuracy (protecting against hallucination liability) and build proper licensing infrastructure with publishers, he says. The framing changes from “you must do this to comply with the EU AI Act’ to ‘this data is valuable to you,” says Svilich.
“We believe AI and content creators can thrive together,” he says. “Our mission is to build the infrastructure that makes this possible—open standards that enable attribution, licensing, and trust at scale.”
