Tech companies have an insatiable need for content on which to train their AI models. The creators of all that content would like to be paid for the role they’re playing in developing such a valuable technology. And many resulting copyright lawsuits could threaten the data supply without an agreed-upon compensation scheme.
Enter the co-creator of the RSS feed, Eckart Walther, and Doug Leeds, former CEO of IAC Publishing and Ask.com. They’ve devised such a scheme, and it’s supposed to be Really Simple.
Really Simple Licensing (RSL) aims to provide a standardized and automated way for web publishers to set licensing and compensation terms. It also spans a nonprofit collective rights organization designed to create leverage by numbers, if enough content owners join. Major websites like Reddit, Yahoo, and Medium have already signed on, as have news publishers like Ziff Davis, Adweek, and the Daily Beast.
Having built their careers around the open web, Walther and Leeds said they were spurred to action by the growing cracks they saw in online business models. Other web publishers have been panicked about them, too; Google search traffic has been crashing for months as the search engine centers AI-generated results and competing AI chatbots draw away would-be searchers.
“There was this traditional contract or ecosystem between content and aggregators. For 30 years, we’ve had the click economy, and the click economy is breaking,” Walther told Tech Brew. “There’s a whole bunch of efforts you see out there, like marketplaces and licensing, but…you can’t fix it by throwing some proprietary solution at it—you have to fix the internet.”
Lots of bots: Meanwhile, websites are seeing more traffic of a different kind. AI crawlers and scrapers are swarming them to copy content for AI training and retrieval. You can see websites attempting to swat away some of these in their “robots.txt” files, standardized forms that instruct which crawlers and bots are unwelcome. A quick perusal of one of these pages for any big publisher tends to feature a litany of different blocked bots belonging to big names in the AI industry (Morning Brew currently has such blocks in place).
The RSL standard is designed to allow websites to add licensing and compensation terms to these robots.txt files, among other places, instead of just blocking them outright. It can also be added to digital media like e-books, videos, and datasets.
“[It] aligns the incentives from both their AI company side and from the content creator side,” Leeds said. “If they win, the creators that power them win, and that, we think, is the way to make a sustainable and fair ecosystem.”
Membership in RSL also doesn’t preclude existing deals between websites and AI companies, like Reddit’s licensing deals with OpenAI and Google, according to Leeds.
But AI companies have also been accused of circumventing robots.txt files, and a pay-per-inference fee, like one of the example structures on the site, would likely require AI company buy-in. The collective rights organization, the RSL Collective, is designed to provide the leverage to bring AI companies to the table. It’s modeled after other collective rights organizations like ASCAP, which manages rights on behalf of music artists.
Keep up with the innovative tech transforming business
Tech Brew keeps business leaders up-to-date on the latest innovations, automation advances, policy shifts, and more, so they can make informed decisions about tech.
“I can just read the tea leaves. This will never work unless we band together,” Medium CEO Tony Stubblebine told us. “We start by revoking the rights as broadly as we can and forcing these companies to the negotiating table. And I think this is what I’m hoping that RSL will do.”
Stubblebine said he would distribute any fees collected through a licensing agreement to writers on the platform, potentially minus legal fees.
A bigger effort: RSL is not the only one beating that particular drum for websites right now. Matthew Prince, CEO of internet infrastructure company Cloudflare, has attracted attention in the industry for a new Cloudflare tool that blocks crawlers by default and effectively implements a pay-per-crawl system.
Prince claimed that this was “still a very early step in what will be a long journey,” but that AI companies have become more willing to negotiate compensation in the last few months since it was implemented.
“The conversations around compensating content creators for content have gotten much more fruitful, and the rates that the AI companies are willing to pay have markedly ticked up,” Prince told us.
Prince was critical of the RSL effort: “It feels, unfortunately, a little bit more like an organization that’s good at press releases and not actually good at solving problems,” he said. “I hope the RSL people will officially release a license, because otherwise it’s just talk.”
Leeds stressed that the project is still in its earliest stages and already gaining momentum. “The initial launch of the RSL Standard last week was just that—the first phase for the protocol,” he told us. “Since launching, we have been receiving strong support from major publishers all across the globe, adding to the lineup of leading publishers and platforms that endorsed RSL at launch.”
Legally bound: Having gone public just this month, though, the RSL is still very much early days. The company is using Cloudflare competitor Fastly for technical enforcement against bots. Leeds said he borrowed an analogy from Cloudflare to describe Fastly’s role:
“If you have this license and a crawler comes and they haven’t agreed to those license terms, they’ll be blocked,” Leeds said. “Those guys will be the bouncers at the door of the club, and we’ll be issuing the IDs.”
Leeds said the collective rights organization will help pool legal resources to fight these battles, like those in other media fields.
“The collective rights organization is for millions of publishers, and the most important thing is that every small publisher or creator joins this collective rights organization, because otherwise, you don’t get this negotiation leverage,” Walther said.