Because IP attorneys are often arguing cases in front of non-technical judges and juries, lawyers often rely on analogies to help them explain how a particular technology works. Choosing the right analogy can make the difference between winning or losing a case. Lawyers have used several different analogies to explain how ML/AI models (I’ll often just call them “models” here) are trained and how they function in service of arguments that the training process does or doesn’t infringe US copyright law. This post will only examine whether training a model is copyright infringement. Whether the output is infringing is out of scope for this discussion.
The Suggested Analogies
In the class action accusing Stable Diffusion and Midjourney of copyright infringement, the plaintiffs argue that the generative AI models in question are essentially sophisticated collage tools, with the output representing nothing more than a mash-up of the training data, which is itself stored in the models as compressed copies.
Among those taking issue with this analogy is my colleague Van Lindberg, a distinguished IP attorney who recently published a law review article in Rutgers Business Law Review titled “Building and Using Generative Models Under US Copyright Law,” which provides a great introduction to generative AI to a non-technical audience. Lindberg argues that current generative AI models do not store compressed copies of the training data, but that “the model training process records facts about the work.” He continues: “Think of the analogy of the art inspector taking every measurement possible – brushstrokes per square inch, correlations between colors six inches apart, and the number of syllables in the artist’s name.” On the strength of this analogy, Lindberg argues that model training either isn’t governed by copyright law or constitutes fair use.
Lindberg is right that the model is not literally storing copies of copyrighted works like an archive or a database. It’s not possible just to click around and open up various works that were part of the training data. That implication by the plaintiffs is at best misleading and in any case literally and technically untrue. The model contains a set of numbers and equations that have a relationship to the training data and enable the model to produce responsive output appropriate to the input.
But by focusing on the technical inaccuracy of the plaintiffs’ characterization of the training process, Lindberg misses an opportunity. If we engage with it as an analogy, the plaintiffs’ characterization ultimately undermines their arguments for infringement and against fair use.
Evaluating the Suggested Analogies
Generative AI models are often described as “black boxes,” because even AI experts don’t really know why a model produces a specific output for a given input; they do not have insight into a model’s decision-making process or all the factors it takes into consideration when producing a particular output. We can say how the model is designed to analyze training data, and how the results of that analysis are encoded (in this case, as parameters and biases in numerical format), but not what information the model ultimately records about the training data.
This process is so opaque that researchers are using one AI model to guess what data another model uses in its decision-making and how it weighs one piece of data versus anothers in producing its output. One researcher spent months reverse-engineering a tiny model just to figure out how it does addition, and this was headline-grabbing news in the AI world.
So, it’s important to keep in mind that neither side of this argument actually has a good understanding of what’s actually in any particular model; when Lindberg analogizes what Stable Diffusion does to an art inspector tracking the number of syllables in an artist’s name, we don’t know that Stable Diffusion is actually doing anything like that. To some extent, we’re all guessing, and will be until AI researchers better understand what they’ve built. I will graciously eat my hat if some of my arguments become outdated or debunked in this quickly moving field.
Lindberg asserts that the model records facts about a work but not the work itself.1 But a certain level of fact gathering is nearly indistinguishable from reproducing the work, or creating a derivative work of it.2 If a summary of a book is as long as the book itself, there’s a strong argument there that the summary is really a derivative work of the book. Sheet music is a very detailed factual account of a recorded song, but we recognize both as eligible for copyright. Likewise, scripts with stage directions are a very detailed factual account of a movie, but we recognize both as eligible for copyright. Images don’t have a non-digital analog to describe the copyrighted work the way that movies or recorded songs do, but they do have digital analogs like PNG and JPEG.
If we create new ways of conveying substantially similar creative expression in different formats, why should that creative expression lose protection simply because it is conveyed via a new format? Particularly where, as with ML/AI models, not only can the work be conveyed via the new format, but it can be effortlessly and instantaneously converted back into the original format (or a very similar derivative work) during the output phase – no need to send anyone to the sound stage again! The courts have long recognized that merely changing formats is a creation of a derivative work of the original.
Lindberg argues that “there is no way in which an ML model could be mistaken for any of its training inputs. The mass of statistical probabilities that make up a generative ML model are so different from the training material that there is no question that it is “different in purpose, character, expression, meaning, and message” from any (or all) of the works that were used as input.” But this oversimplifies. After all, images and song recordings don’t lose their copyright protection just because we express them via 1’s and 0’s in JPEGs and MP3s. Simply digitizing a copyrighted work, or in this case encoding it in one digital manner versus another (like switching from WAV to MP3), doesn’t strip the work of copyright protection.
At present, we cannot say for certain whether an ML/AI model does so much “fact gathering” that it’s a derivative work of the training data. We can only deduce certain things about the model by looking at the output. But when a model’s output is identical (or very similar) to the training data, that’s a strong indication that the model has captured enough information about that training data to be considered a derivative work.
Lindberg focuses on one study where researchers were able to extract only a few identical copies of training data from image-generating diffusion models. From these results, he argues that this is a very rare occurrence that mostly happens when the training data isn’t properly deduplicated or the model is overtrained – in other words, when someone messes up.
But Lindberg overlooks the many AI outputs which, while not identical to the training data, may be similar enough to be derivative works. Granted, we can’t accurately quantify how much of a model’s potential output might be derivative of its training data, because the question is subjective and case-specific, but that doesn’t mean we can just avoid the question. Courts make those judgements even if researchers can’t, and our legal system is designed to protect copyright holders from unauthorized derivatives.
Developers can set up certain guard rails to prevent output that is too derivative (and therefore perhaps avoid liability for the output), but the fact that even a competently trained model is capable of producing obviously derivative output strongly suggests that the model is fairly closely encoding the training data. And while Lindberg argues that his reasoning extends to all types of generative models, there is evidence that the performance of large language models is improved when the models memorize more (regardless of overfitting). In other words, memorization is a feature, not a bug.
Some may argue that because a trained model encodes millions or billions of pieces of training data together and not just one, that it can’t really be said to be an encoding of any one particular work; it’s impossible to actually pinpoint data about any one piece of training data within the model because the model generalizes from the training data and “forgets” some of the training data in the process. But while this may be true under a particular set of circumstances, it is not universally true of all AI/ML models because:
- When models are trained on a relatively small set of training data, they are more likely to produce output that is identical or very similar to the training data. They simply don’t generalize.
- Models whose training data contain many duplicates are less likely to generalize and more likely to create verbatim or very similar output to the training data.
- Models that are overtrained are more likely to create verbatim or very similar output to the training data.
- The fact that the encoding for any particular piece of training data is difficult to extract doesn’t mean it’s not there. If the training data can be output by the model, then it is in some way encoded in the model during training. In a mile-wide photo collage, any one picture may be difficult to locate, but it’s still there, and the collage is still a compilation requiring permission from the photo’s copyright holder.
- While current AI/ML model architecture makes it difficult to pinpoint where a model has encoded information about a piece of training data, that is likely to change in the future. Other architectures may prove more useful and researchers are actively working on making AI models more transparent, making it easier for people to understand what AI models learn from training data and how they use that information to make decisions. Specifically, it is desirable for a model to be able to indicate what specific training data it relied on to make a decision, including by showing that training data when asked.
- In certain contexts, it’s extremely desirable for models to have perfect memory of certain portions of their training data or data received after training. The arguments discussed above all relate to generative AI, which is designed to produce new work based on fairly limited prompts. However, other functions related to problem-solving, analysis, planning, and education will require the model to maintain perfect memory of a portion of its training data. For example, no one wants an AI scientific research assistant with imperfect memory of the periodic table of elements or which combination of medicines can kill people. One of the biggest benefits of AI assistance is that it is, in fact, capable of perfect memory in a way humans are not. Many models can also broaden their functionality by receiving and processing data after training has been completed as part of a specific user query (i.e. “Alexa, read this book you’ve never seen before and tell me if it’s suitable for my 5-year-old.”) and it may not be useful for the model to generalize this data.
- To a certain extent, the level of generalization and forgetting a model does is driven not by the need to optimize the model’s results, but by computational limitations. Those limitations are diminishing every day.
Alternative Analogies
Arguments and analogies that assume models don’t memorize aren’t just technically inaccurate, they are not future-proof: they link the question of fair use solely to the specific state of the art available today. If the fair use defense depends on this assumption, it will fail whenever any amount of memorization can be shown. Fortunately, there’s no need to build on such a fragile foundation: there is plenty of precedent that a use can be transformative even if it depends upon copying and retaining entire copyrighted works.
Argument 1: ML/AI Models Can Be Transformative in the Same Way as the Google Books Project
To create a searchable digital book archive, Google copied and compiled thousands of books verbatim. Even so, the Second Circuit approved of the archive as fair use because its purpose (to allow users to discover facts about specific books and literature more broadly) was transformative (and specifically, it was furthering the goals of the Copyright Act to spur the growth and spread of knowledge), and the way in which the archive was ultimately utilized (in this case, the technical features of the search function which limited the scope of the results it would provide) did not create a substitute for the original work in the market.
Applying that same analysis in the model training context, it’s clear that the analysis would look different depending on the model. “Foundational models” like Stable Diffusion and ChatGPT can do a broad range of things within one or more modality (image, text, audio, etc.). For example, ChatGPT and other large language models (LLMs) are essentially designed to generate text that’s responsive to a prompt, and they can be given prompts on most any topic. These models can be further tuned to become even more useful within a particular domain of knowledge or a particular task and might receive additional data beyond the training data made available to the foundational model. Other models are not foundational models at all; they are trained on a much narrower variety of training data and they only do some limited tasks for specific use cases.
With this lens, foundational models, particularly those that haven’t been heavily fine-tuned, look more transformative because they have broader applications than non-foundational or heavily fine-tuned models with specific use cases (like “we want to display copyrighted artwork that best matches people’s moods”).
Even when a model has an ostensibly transformative purpose (and/or still retains it after fine-tuning), the way a model ultimately operates may still vary. For example, one image-generating model may take great pains to avoid outputting verbatim copies of copyrighted works (Stable Diffusion can no longer be prompted by specific artist names, for example), but another might freely allow it or even take steps to make such output easier to prompt. Even though the purpose of both models may be to help people create original art, courts may view them differently based on their implementation.
For this reason, it’s not realistic to classify the training of all ML/AI models as either fair use or infringing. Each instance must be evaluated in the context of the model’s intended use case and whether it is, in practice, offering substitutes in the market for the kinds of works that comprise its training data.
Today, at least, it is unlikely that courts could provide a comprehensive rubric to distinguish fair from infringing models. That would require a degree of technical knowledge unlikely to be relevant to a single trial. And in general, legal rubrics like that tend to emerge only after enough cases have been decided that a court can create a cohesive analysis of all of them.
Argument 2: The Model Is More Like a Means of Reproduction Than a Copy Itself and the Means of Reproduction Are Legal So Long as There Is Some Legitimate Downstream Use
A blanket decision that training models is legal would be desirable and advantageous for the tech sector given the way many models are currently created and shared. Today, it’s not uncommon for entities to create foundational models, to make those models publicly available for download for free, and for other entities to download those models and further fine-tune them for their specific use cases or otherwise experiment with them. Most notably, Meta’s release (first accidental, and then intentional) of LLaMA , a foundational language model, spurred research, experimentation, and improvements to it, and language models in general, from a wide variety of individuals and entities, including universities and start-ups who could not afford to train their own foundational models, but who obviously had a lot to contribute if they had something to work with. It’s likely that some downstream uses of such models don’t raise any eyebrows, but other uses draw a legal firing squad. So, it would be really convenient, and arguably good for the sake of technological progress and invention, for the creators of the models to get legal absolution for creating the model without regard for how downstream users actually use the model.
If the goal is specifically to absolve the creators of foundational models, there is an argument that even if the model is a way of encoding the training data, because the only practical way to access such an “archive” is by having the model create output, it should be treated more like a means to create a copy rather than as a copy in and of itself. I think that AI/ML models can be likened to something like a Star Trek replicator. Since anyone can ask the replicator to create any physical object, be it a bayonet from the 1700s or a James Bond-style martini, the replicator has obviously ingested more or less all of humanity’s collective knowledge. But, this knowledge is entirely useless and inaccessible except to the extent that the replicator actually produces something. Therefore the replicator is treated as nothing more than a means of reproduction (literally, a “replicator” even though it can create things that have never existed before like a hot pink bayonet or a martini with tapioca pearls), rather than an archive or library. It only has value to the extent it can actually produce output. Like a photocopier, tape recorder, or VCR, the replicator’s existence, without actual use by anyone, doesn’t rob any IP holder of an opportunity to monetize their IP; that only comes into play with respect to certain types of output and certain uses of that output, which are controlled solely by the user requesting the output.
Treating the AI/ML model as merely a means to create a copy for all practical purposes would align the fact pattern here with the one in the Supreme Court ruling related to VCRs. Since the user of the VCR was the one producing the tape recordings and not the VCR manufacturer (Sony), direct copyright infringement was off the table, leaving Sony only with claims for contributory and vicarious copyright infringement. In that case, the court closely examined how downstream users used VCRs in making its determination with respect to whether Sony’s sales of them constituted either contributory or vicarious liability. The court did not find contributory or vicarious liability because they could reasonably identify at least one downstream use case that met the fair use standard: at home time-shifting of broadcast tv. The court repeatedly emphasized that “the sale of copying equipment, like the sale of other articles of commerce, does not constitute contributory infringement if the product is widely used for legitimate, unobjectionable purposes. Indeed, it need merely be capable of substantial noninfringing use [italics added].” I’m certain that there are many use cases out there of ML/AI output that have legitimate purposes (like quoting certain sources for educational purposes, creating or editing original art, analyzing data from medical trials, etc.). And unlike the photocopier or the VCR, the vast majority of the output is not, in fact, a copy of the original, meaning that it’s even easier to find noninfringing uses with respect to ML/AI models than with any other piece of technology designed to create near exact copies.
Argument 3: The Model Includes the Training Data, But There’s No Substantial Similarity Between the Two
Another way to get to the same place is to argue that at a certain point the model has so much training data, and it so complexly bundled up and interwoven together in the model, that there is no copyright violation because the model and any one piece of training data no longer have substantial similarity. After all, it is literally impossible to pull any one piece of training data from the model itself without creating output. If you really want a specific copyrighted work in its entirety, it is at least an order of magnitude easier to just go get it from the website it was scraped from or to go photocopy a library book than to try to get a properly trained foundational model to faithfully reproduce it. The portion of data in the model about any specific piece of training data is infinitesimally small compared to the totality of the data in the model.
This argument could be aided by the fact that the model doesn’t just contain information about the training data (or the training data itself), the model also contains information about what sort of output is pleasing to humans – in the image context, it contains data not just about the training data, but about how to interpolate those images to create something responsive to the prompt that’s also appealing and not visually confusing or disturbing. A model that only contained information about the training data but not how to evaluate it and transform it for the purposes of creating output would not be very useful.
The use of any particular piece of training data could be deemed de minimis3 in such a case, but there would still be potential claims of contributory and vicarious copyright infringement as well as claims related to the output, since the output could still have substantial similarity to the training data.
This may be subtle, but I think there’s a big difference between trying to argue that the model doesn’t contain the training data at all (which I find unconvincing, not least because you are still asking a court to declare that a certain small percentage of copying can be overlooked for reasons that aren’t entirely clear as I’m not aware of any model guaranteeing a 0% copying rate), and arguing that the particular type of encoding it has undergone transforms the model into something very different from merely the sum of all of its training data.
The Limits of Analogies
While slam-dunk analogies are incredibly powerful in a courtroom setting, the courts are not the only way to get clarity on the issue of model training; there is also a statutory pathway wherein Congress creates a specific copyright exception for model training based on certain policy considerations. The relatively open practice of model development and refinement is good for technological progress – there is no doubt that making these models publicly available has sparked a Cambrian explosion of technological progress in the field. And models that can be transparent about what training data is influencing their decision-making are socially desirable even if that transparency comes at the cost of allowing models to memorize the training data.
There’s certainly precedent for Congress creating copyright exemptions based on policy goals tied to specific new technologies (like the DMCA’s safe harbor for those who host third party content) and other countries have created copyright exceptions specifically for model training for these and many other reasons. If US legislators are to be encouraged to draft a clear exception for model training, they must decide that the appropriate outcome is not likely to be reached by the courts any time soon, i.e. that the laws currently on the books do not adequately address this topic and new ones must be written. They won’t feel that way, though, if the question of fair use with respect to AI/ML training is popularly treated as a foregone conclusion.
To that end, while I believe that some of the alternative arguments I’ve offered are closer to the technical truth at issue, I don’t have any reason to believe that they are easier for a jury or judge to understand than the ones proffered by others. In general, this is a deeply technical discussion that even AI experts have a broad range of opinion about, which makes this a subject better suited for regulatory experts or legislators in consultation with various advisory committees and federal agencies, than 12 people chosen at random or a single judge.
Conclusion
I agree with Lindberg’s general notion that the legal system should find a way to remove copyright liability from at least some ML model developers for a number of policy reasons. But, tying the legality of AI model training to the fact that its memory is imperfect or to the fact that it’s currently challenging to get a model to reproduce training data as output may be a short-sighted approach. To the extent that it’s desirable to clear a legal path forward for ML/AI model training, it doesn’t make sense to rely on arguments that lead courts to create rulings that can only protect somewhat frivolous AIs like image generators from liability, but whose logic cannot be extended to a future set of AIs to be used in scientific and mission-critical contexts. And in fact, whose logic creates precedent for finding those other types of models to be illegal.
- As I have previously written, I don’t think the model just stores information about the training data. I think the model also stores information about what sort of output humans find appealing. That data is what sets models apart even when they’re trained on the same training data corpus. This seems logical to me since models can be improved without training them on new training data.
- Note that while it’s arguable whether or not 17 U.S.C. 117(a)’s incidental copying exception would apply to model training, that section’s exception doesn’t extend to creating derivative works so I’m going to leave a 117(a) discussion for a separate post.
- While the Ninth Circuit has famously said that the de minimis doctrine is about what percentage of a copyrighted work is copied in Bell v. Wilmott Storage Services, LLC, other courts have looked to the newer work to assess what portion of the newer work is made up of the older work and they have taken into account how observable the copy really is in the new work, even if it was copied in full. Aaron Moss writes: “For example, in a case involving the 1995 Brad Pitt crime thriller “Seven”, the Second Circuit found that that [sic] the use of copyrighted photos that appeared fleetingly and out of focus for 35 seconds of the film was de minimis. Another court reached the same result when a pinball machine appeared in the background of a scene from “What Women Want.” And last year, the court in Solid Oak Sketches v. 2K Games held that the use of NBA players’ tattoos in the popular “NBA 2K” video games was de minimis, pointing out that the tattoos appeared only fleetingly, and comprised only 0.000286% to 0.000431% of the total game data.”
