Extracting memorized pieces of books from open-weight language models

The only way to include a book in a training dataset for LLMs without violating copyright law is to contact the rights holder and buy a license to do so. Buying an ebook license off Amazon isn't enough for this, and creating a digital copy from a physical copy for your commercial use is also against the law. A good rule of thumb is if it would be illegal for a company to distribute the digital file to empolyees for training, it's definetally illegal to train an AI the company will own on it.

If the data of a copyrighted work is effectively embedded in the model weights, does that not make the LLM itself an illegal copy? The weights are just a binary file after all.

This is an interesting comment which avoids the issue: when a user uses an LLM to violate copyright, who is liable, and how would you justify your answer?

You remind me to all the shitty times in literature class where I had to rote memorize countless works from a given author (poet), think 40, then take a test identifying which poem each of the given quotes was from. The little WinForms app I wrote to practice for these tests was one of the first programs I've ever written. I guess in that sense it's also a fond memory. I miss WinForms.

Good thing they were public (?) works, wouldn't wanna get sued [0] for possibly being a two legged copyright infringement. Or should I say having been, since naturally I immediately erased all of these works from my mind just days after these tests, even without any legal impetus.

Edit: thinking about it a bit more, you also remind me to our midterm tests from the same class. We had to produce multiple page long essays on the spot, analyzing a select work... from memory. Bonus points for being able to quote from it, of course. Needless to say, not many original thoughts were featured in those essays, not in mine, not in others' - the explicit expectation was that you'd peruse the current and older textbooks to "learn (memorize) the analysis" from, and then you'd "write about it in your own words", but still using technical terms. They were pretty much just tests in jargon use, composition, and memorization, which is definitely a choice of all time for a class on literature. But I think it draws an interesting perspective. At no point did we ever have to actually demonstrate a capability in literary analysis of our own, or was that capability ever graded, for example. But if you only read our essays, you'd think we were great at it. It was mimicry. Doesn't mean we didn't end up developing such a capability though.

[0] https://youtu.be/-JlxuQ7tPgQ

Where in the article do the authors say this puts anything to rest? Here is their conclusion:

> Our results complicate current disputes over copyright infringement, both by rejecting easy claims made by both sides about how models work and by demonstrating that there is no single answer to the question of how much a model memorizes

I wonder if this is the sort of article that people will claim supports their side, even that it ends the debate without a knockout blow to the other side, when the actual article itself isn't making any such claim.

I'm sure you read the entire article before commenting, but I would strongly recommend everyone else does as well.

Size of LLM: <64Gb.

Size of training data: fuck knows, but The Pile alone is 880Gb. Public github's gonna be measured in Tb. A Common Crawl is about 250Tb.

There's physically not enough space in there to store everything it was trained on. The vast majority of the text the chatbot was exposed to cannot be pulled out of it, as this paper makes clear.

I'm guessing that the cases where great lumps of copyright text can be extracted verbatim are down to repetition in the training data? There's probably a simple fix for that.

(I'm only talking about training here. The initial acquisition of the data clearly involved massive copyright infringement).

I think I big part of copyright law is whether the thing created from copyrighted material is a competitor with the original work, in addition to whether it's transformative.

LLMs are OBVIOUSLY not a replacement for the books and works that they're trained on, just like Google books isn't.

I don’t think it’s that simple.

Last I remember, whether it is ‘transformative’ is what’s important.

https://en.m.wikipedia.org/wiki/Transformative_use

Eg. Google got away with ‘transforming’ books with Google books.

https://www.sgrlaw.com/google-books-fair-transformative-use/

The problem is if someone uses a prompt that is clearly Potter-esque, there have been examples of it returning Potter exactly. If it had never had Potter put into it, it would not be able to do that.

I think the exact examples used in the past were Indiana Jones, but the point is the same.

So can humans? I can ask a human to draw Mickey Mouse or Superman, and they can! Or recite a poem. Some humans have much better memories and can do this with a far greater degree of fidelity too, just like an LLM vs an average human.

If you ask OpenAI to generate an image of your dog as Superman, it will often start to do so, and then it will realize it is copyrighted, and stop. This seems sensible to me.

Isn’t it the ultimate creative result that is copyright infringement, and not merely that a model was trained to understand something very well?

Just because an LLM has the ability to infringe copyright doesn’t mean everything it does infringes copyright.