A novel method developed by INESC-ID researchers is at the heart of a headline-making investigation into AI training practices. The technique — DE-COP: Detecting Copyrighted Content in Language Models Training Data— has been used in a recent study by the AI Disclosures Project, co-founded by media figure Tim O’Reilly and economist Ilan Strauss, to examine whether OpenAI’s GPT-4o model was trained on copyrighted, paywalled content without permission.

At the core of the controversy is the possibility that OpenAI, a leading player in generative AI, used proprietary books from O’Reilly Media in the training of its most advanced model to date, GPT-4o. The AI Disclosures Project’s paper points to DE-COP as a critical tool in establishing this likelihood.

Developed in 2024 by INESC-ID researchers André Duarte and Arlindo Oliveira, together with colleagues from University of California and Carnegie Mellon University, DE-COP tackles one of the most relevant and difficult questions in the field of AI ethics and transparency: How can we detect if copyrighted content was used in a model’s training data, when that data is not publicly disclosed?

DE-COP works by probing large language models (LLMs) with multiple-choice questions, where the correct answer is embedded within both exact quotes and paraphrased versions of suspected training content. If a model consistently selects verbatim excerpts over paraphrased ones, it suggests prior exposure — a hallmark of what is known in the field as a “membership inference attack.”

To validate the approach, the researchers behind DE-COP constructed BookTection, a benchmark dataset featuring excerpts and paraphrases from 165 books, both pre- and post-dating the training cutoffs of popular LLMs. The method outperformed previous techniques by a significant margin — a 9.6% improvement in detection performance on models with available logits, and 72% accuracy on fully black-box models, where prior methods hovered around four percent.

The AI Disclosures Project applied DE-COP to a set of 34 books from O’Reilly Media, analysing nearly 14,000 paragraph excerpts. The study found that GPT-4o showed a much higher “recognition” of paywalled content from O’Reilly books compared to OpenAI’s previous model, GPT-3.5 Turbo. As noted in the article published at TechCrunch.

“GPT-4o [likely] recognizes, and so has prior knowledge of, many non-public O’Reilly books published prior to its training cutoff date,” the authors noted. They also found that GPT-3.5 Turbo, in contrast, demonstrated greater recognition of publicly available O’Reilly materials — suggesting a significant shift in training data sources between model generations.