Publishers under increasing threat from Google’s use of AI

You may never have to read another news story in your life, if you have artificial intelligence (AI) that can digest all the web’s information and serve up a summary on demand.

That’s the stuff of nightmares for media barons as Google and others experiment with what’s called generative AI, which creates new content drawing from past data. Since May, Google has begun rolling out a new form of search powered by generative AI, after industry observers questioned the tech giant’s future prominence in providing consumers with information following the rise of OpenAI’s query-answering chatbot, ChatGPT. The product, called Search Generative Experience (SGE), uses AI to create summaries in response to some search queries, which appear on the top of the Google search home page, with links to “dig deeper”, according to Google’s overview of SGE.

If publishers want to prevent their content from being used by Google’s AI to help generate those summaries, they must use the same tool that would also prevent them from appearing in Google search results, rendering them virtually invisible on the web.

Searching for “Who is Jon Fosse” — the recent winner of the Nobel prize in Literature — for instance, generates three paragraphs on the writer and his work. Drop-down buttons provide links to Fosse content on Wikipedia, NPR, The New York Times and other websites; additional links appear to the right of the summary.

Google says that the AI-generated overviews are synthesised from multiple web pages and that the links are designed to be a jumping off point to learn more. It describes SGE as an opt-in experiment for users, to help it evolve and improve the product, while it incorporates feedback from news publishers and others. To publishers, the new search tool is the latest red flag in a decades-long relationship in which they have both struggled to compete against Google for online advertising, and relied on the tech giant for search traffic.

The still-evolving product — now available in the US, India and Japan — has raised concerns among publishers as they try to figure out their place in a world where AI could dominate how users find and pay for information, according to four major publishers who spoke to Reuters on condition of anonymity to avoid complicating ongoing negotiations with Google.

These concerns relate to web traffic, whether publishers will be credited as the source of information that appears in the SGE summaries, and the accuracy of those summaries. Most significantly, publishers want to be compensated for the content on which Google and other AI companies train their AI tools — a major sticking point around AI.

A Google spokesperson said in a statement: “As we bring generative AI into Search, we’re continuing to prioritise approaches that send valuable traffic to a wide range of creators, including news publishers, to support a healthy, open web.”

On compensation, Google says it is working to develop a better understanding of the business model of generative AI applications and get input from publishers and others.

In late September Google announced a new tool, called Google-Extended, that gives publishers the option to block their content from being used by Google to train its AI models.

Giving publishers the option to opt out of being crawled for AI is a “good faith gesture”, said Danielle Coffey, president and  CEO of the News Media Alliance, an industry trade group that has been lobbying Congress over these issues. “Whether payments will follow is a question mark, and to what extent there is openness to having a healthier value exchange.”

The new tool doesn’t allow publishers to block their content from being crawled for SGE, either the summaries or the links that appear with them, without disappearing from the traditional Google search.

Publishers want clicks to secure advertisers, and showing up in Google search is key to their business. The design for SGE has pushed the links that appear in a traditional search further down the page, with potential to reduce traffic to those links by as much as 40%, according to an executive at one of the publishers.

More alarming is the possibility that web surfers will avoid clicking any of the links if the SGE passage fulfils the users’ need for information — satisfied, for example, to learn the best time of year to go to Paris, without having to click on a travel publication’s website.

SGE is “definitely going to decrease publishers’ organic traffic and they’re going to have to think about a different way to measure the value of that content, if not click through rate”, said Forrester Research senior analyst Nikhil Lai. Even so, he believes publishers’ reputations will remain strong as a result of their links appearing in SGE.

While publishers and other industries have spent decades adjusting their websites to show up prominently in traditional Google search, they don’t have enough information to do the same for the new SGE summaries, these publishers say.

“The new AI section is a black box for us,” said an executive at one publisher. “We don’t know how to make sure we’re a part of it or the algorithm behind it.”

When given the option, websites are blocking their content from being used for AI if doing so doesn’t affect a search, according to exclusive data from AI content detector Originality.ai. Since its August 7 release, 27.4% of top websites are blocking ChatGPT’s bot — including The New York Times and Washington Post. That is compared to 6% that are blocking Google-Extended since its September 28 release.

Reuters