This is the second part of a four-part series called “Protecting Digital Content in the AI Age: A Lawyer’s Guide.” Part 1, “AI and the Digital Content Provider’s Dilemma,” examined the effects of AI on digital content providers, including both a drop in web traffic and significantly higher demands on web infrastructure, and called for new business models like micropayments to maintain the open web. Part 3 will examine AI crawlers, robots.txt, the technology behind the web, and new tools for digital content owners. Part 4 will analyze US and EU laws related to text and data mining opt-outs and technologies that may help digital content owners sustain their businesses.
As discussed in Part 1, the open web is already shrinking in response to AI, with more digital content providers moving their content behind logins, removing it from the internet entirely, or attempting to block access to it via both legal and technical mechanisms. AI companies are losing access to data, while the rest of us are losing an open web. Bespoke deals between AI companies and certain content providers are just a bandaid. AI companies can only negotiate so many deals. If nothing changes, both the AIs and the internet as a whole will continue to lose “long tail” niche content, putting content providers out of business, and AI models will exhibit more bias because they will lack data from smaller cultural communities and non-English speaking countries. Micropayments – an automated, frictionless way for AI companies to pay for the content they use – are the right solution for content providers, AI companies, and society as a whole.
A. Micropayments Can Quickly Force AI Companies to Pay for Content to Remain on Even Footing with Their Competitors
Automated means by which content providers could offer free or cheaper licenses to non-profits, small businesses, companies that respect content providers’ preferences, or companies that release open source AI models (or some combination thereof, under whatever definitions content providers prefer), while signaling that they expect the megacorp AI players to pay is the number one most effective way to pressure AI companies into paying for the content they use. Paying or not paying under such a regime completely changes the AI companies’ calculus of paying content providers. Today, the AI companies weigh the costs and benefits of certain licenses or the possibility of certain lawsuits, and they can incrementally choose how much risk they are willing to accept so long as all of their competitors are doing the same math. But a competitor who gets access to content for much less or for free poses a starker choice to the megacorp: pay for the same data or lose to entities whose AIs can provide higher-quality information. The question would become existential in a way that it is not today.
The unwillingness of content providers to privilege certain entities over others, whether out of principle or in search of maximum profits, simply means disarming themselves of the single most powerful tool they have to protect their businesses. In particular, entering into deals with AI companies but leaving non-profits and small businesses with no rights at all (including by blocking all entities from training on their material in robots.txt) only helps to secure an oligopsony of AI companies and make the content providers more and more vulnerable over time.
Many of the major AI companies view their mission as reaching artificial general intelligence or artificial super intelligence (human levels of intelligence or higher), and many in the industry, including Dario Amodei (CEO of Anthropic), Demis Hassabis (Google DeepMind CEO), and Elon Musk have explicitly stated or otherwise implied that they view such an achievement as a winner-take-all proposition. That is their fundamental argument for why the US should do everything in its power to beat China in the AI race. Content providers need to understand this dynamic in order to fully appreciate why price discrimination is such a powerful tool in this circumstance and why AI companies fear one another more than they fear billion dollar lawsuits.
B. Micropayments Keep Power in the Hands of Creators Instead of Intermediaries
Intermediaries can provide indispensable services (especially when content must be provided in physical form) and in some cases intermediaries can use their leverage to get individuals better deals, especially when they take the form of regulated non-profits, as in the EU. But, if AI companies continue to make bespoke deals with large content providers and intermediaries while ignoring everyone else, the natural result will be intermediary growth and then consolidation, giving them more leverage over time over the AI companies, content providers, and the rest of society.
The companies that ignore or neglect the individual content creator in favor of feeding the intermediary, invariably discover, again and again, that they’ve been throwing meat to a baby dragon, not a household pet. This is the story of how Amazon fed publishers (instead of authors) who then created their own e-book and audiobook platforms, Netflix fed movie studios (instead of writers and directors) who then pulled their catalogs to their own streaming services, YouTube fed multi-channel networks (instead of individuals) that were able to move the best content creators to their own video services, and how Apple’s iTunes Store fed the music labels (instead of individual artists and even independent labels) who then backed Spotify and other streaming services that decimated iTunes. Each and every time, these companies eventually offered individuals more, but only after allowing the dragons to fly away with a limb or two.
The flip side, of course, is that these intermediaries also amass more power over creators, especially the ones they already serve, who may or may not be able to renegotiate for a cut of the money. Such a scenario is already playing out in the Anthropic settlement, where to Judge Alsup’s surprise and dismay, a number of publishers are pulling chairs up to the table instead of authors because many authors have already signed away some or all of the rights to their infringed books. Consolidation means that just a few entities wield massive control not just over AI companies, but over what the rest of us have access to (especially in authoritarian countries), the price for such access, who can make a living making content, and on what terms. Publishers like HarperCollins already don’t blush when it comes to telling their authors they’ll be taking 50% of AI training revenues, take it or leave it.
Intermediary consolidation is no less of a threat to content providers than AI. Elsevier is the wildest dream of every corporate intermediary, exploiting consolidation in scientific publishing to its most brutal, logical conclusion. In large part, their business model is to take people’s research for free, have others do peer-review for free, and then sell the research back to the very same institutions that funded it for exorbitant rates. They require copyright assignment from authors. They bundle journal subscriptions, such that institutions pay even for things they don’t want. Historically, they’ve lobbied against the open access movement and more recently they’ve co-opted it, charging authors $2k-$10k to put their papers up on an open-access website. Their profit margin rivals those of oil companies.
Their high fees have forced smaller scientific publishers out of the market because libraries don’t have the budget for much beyond the Elsevier subscription, leaving authors and institutions who’d actually like to be compensated for their work with few, if any, other options. Authors now largely choose between supporting Elsevier and having their works widely read in well-respected journals, or publishing them freely online in total obscurity. Either way, compensation is off the table and scientists have to make money via institutional paychecks; they cannot work as individuals and make money from their articles instead. Consolidation doesn’t just change prices, it often forecloses individual autonomy and reinforces other exploitative systems.
C. Micropayment Will Help Modernize the Open Web and Make AIs Better
Companies receiving compensation from AI companies will have an incentive to make their websites AI-friendly, making changes that allow AI agents to quickly and accurately find information, make purchases, and select multimedia with appropriate use rights. That would potentially allow AI companies to segregate human-created works from AI-generated works to hone in on high quality data, lower the costs of inference involving retrieval-augmented generation (RAG), and cut the cost of cleansing training data.
In particular, knowing that a certain portion of data is coming from trustworthy sources means AI companies working to avoid data poisoning can focus their efforts and enhance security. There has also been work on protocols like IndexNow that allow a website or content delivery network (CDN) to signal to a crawler whether the website has changed since the last time a crawler visited, potentially reducing unnecessary crawls by 53%, saving energy, and helping AIs prioritize fresh information. It’s not difficult to imagine a regime where AI companies pay AI-friendly sites more. At the same time, people using open source AIs, including researchers and non-profits, will also benefit from higher-quality information.
D. Micropayments Will Ensure Healthy Competition and Technological Progress
Without micropayments, the competitive landscape in AI would stagnate. Startups trying to find their footing in this space simply can’t ink thousands of content deals before they sign a single customer. Payments to the content provider might be back-ended (“we’ll pay you at the end of the year”), but payments to lawyers and salespeople are not.
In the absence of competition, the incumbent AI companies will do what oligopolies always do: stall disruptive innovation, conduct regulatory capture, keep prices high, have an outsized influence on companies and industries upstream and downstream of them, and proceed with the enshittification of their services, as Cory Doctorow puts it: slowly shifting value from the user to the AI company because the user has nowhere else to go now (everybody wave at Comcast!).
It’s nice to be an oligopolist I imagine, but this position wouldn’t necessarily translate to more profits for the AI companies given the fact that squelching innovation – both for their competitors and their users – means that at some point their share of the pie might grow, but the pie itself won’t. Satya Nadella’s goal of increasing worldwide GDP to 10% per year through AI is explicitly contingent on AI itself becoming a commodity, which isn’t possible with billions of dollars worth of barriers to entry. He’s manifestly correct on this front. The success of the Silicon Valley titans over the decades has hinged on technologies where the aggregate value of the technology to its users is many orders of magnitude higher than the amount that can be charged for it, and the right play isn’t charging more for it, it’s spreading it far and wide and finding a way to harvest some of that value downstream. 1
A simple example of this is Apple apps. Apple could have decided that the only apps on its phone would be Apple apps and they’d make 100% of the money off of all software on their phones. Great, they’d have a monopoly there. But it’s clear that Apple has made more money off the Apple Store from third-party apps than it ever would have from its own apps. Third-party apps had huge knock-on effects. Here’s a non-exhaustive list: the phone now had appeal both to consumers and businesses, other companies became more valuable because their products could be accessed on the go (so they hired more people, who needed more phones, who wrote more apps, who invested in new companies…), an entire industry of mobile-first apps like Uber and Shazam sprung up, and everyone online got slightly richer for the added security of two-factor authentication, enabled by third-party apps. All of this made the phone that much more valuable, as well as all the other products and services in the Apple ecosystem. By allowing their share of the app pie to shrink, the entire universe of potential Apple customers expanded.
Stay Tuned!
Stay tuned for Part 3 of this series, which will do a deep dive into the technologies which can enable micropayments, and will provide a refresher of many fundamental web protocols that can be used and extended for AI.
- This is the rationale for much of commercial open source and for free products like Gmail or Bing. Sometimes the company practices vertical integration and becomes a user itself in a particular domain to capture some of those downstream benefits. GitHub Copilot is an example.
↩︎
