Chapter 3: Data as Raw Material

Chapter 2 gave us a way in: training data shapes the model. Some examples can come back word for word. To work out what went in, follow the records for…

Chapter 2 gave us a way in: training data shapes the model. Some examples can come back word for word. To work out what went in, follow the records for the specific model and training run. The political stakes start with who built the pile and who could say yes. Follow the data back. Who gathered it? Who did the work of sorting it? What did they get paid, and who got a say in whether it could be used? Those questions are easier to answer when you can get hold of the training records. Two words do a lot of work here. A dataset can be called open, and it can contain material scraped from the web. What did the developer have permission to use? Hēmi Whaanga asks what happens when that same question is put to the institutions that collect and reuse knowledge. Keep it close for Chapters 9, 11 and 13. ---

Where the training data actually comes from

Lire cette section en français

View receipt #132

View receipt #133

When OpenAI trained GPT-4, what specifically did they train it on? When Anthropic trained Claude, when Google trained Gemini, when Meta trained Llama, when Cohere trained the Command series — what data was actually in those training corpora?

In most cases, we don't fully know. Developers rarely give readers a detailed inventory of what their models were trained on. Stanford's December 2025 index documents gaps in data acquisition and data properties. If a company names one dataset or one source, that tells you something about the named item. It takes a fuller inventory to tell you what else went into training.

GOBLIN FACTS — opacity clusters around data. In the 2025 Foundation Model Transparency Index, six major developers did not disclose basic training-data composition information. That is why "what was it trained on?" remains the most important unanswered question in the room.

The examples below come from developer reports, dataset documentation, independent audits and court filings. Treat them as a map of possible inputs. Check the model, version and source before carrying an example from one system to another.

The “public internet.” Common Crawl archives material crawled from the open web. Meta’s original LLaMA paper describes filtered Common Crawl and C4 among the ingredients. A crawl, a filtered dataset and a particular training run are three different things. Finding an article in a crawl tells you where to look next.

Books and long-form text. Meta disclosed that the original LLaMA used Project Gutenberg and Books3. In its 2025 order in Bartz v. Anthropic, a US district court described Books3 as a collection assembled from unauthorized copies and distinguished Anthropic’s use of books for training from its acquisition of pirated copies for a permanent library. The Meta paper describes a particular model family. The order turns on its own record and US law.

Code repositories. The original LLaMA report says its GitHub subset came from a public dataset and was filtered for Apache, BSD and MIT licences. Check the licence on the actual code and the use at issue before drawing a conclusion about permission.

Academic literature. The original LLaMA report documents a processed arXiv subset. An open preprint, a paywalled journal article and material supplied through a licence have different routes into a dataset. Ask which route the developer can document.

News content. News can appear in a web crawl. The Canadian publishers suing OpenAI allege that it copied their journalism for training and other uses without authorization. Their filing establishes what they allege. It does not give the reader a verified inventory of the training set.

Other web sources. The original LLaMA paper lists Wikipedia and Stack Exchange. GPT-4’s public report gives broad source categories. Ask for evidence that a platform’s material entered the particular model under discussion.

Images and captions. LAION-5B indexed about 5.85 billion pairs of text and image links. The Stable Diffusion v1.4 model card names specific LAION subsets. In December 2023, Stanford Internet Observatory researchers documented links to child sexual abuse material in the LAION-5B index. LAION withdrew the original release and published filtered replacements in August 2024. LAION says it removed samples known to its partners as of July 2024. A link in the index, inclusion in a training run and an output are separate pieces of evidence.

Synthetic data. Increasingly, AI training data includes data generated by other AI systems: a model trained on outputs from a previous model. The technique allows training-data scaling beyond what human-generated content alone can supply, and is now standard practice for fine-tuning and alignment. The structural concern this raises: if successive AI generations are trained substantially on AI-generated content, errors and biases in earlier generations propagate forward, potentially amplifying. Academic research on "model collapse" (Shumailov et al., 2024 in Nature) suggests this is a real risk, though the consequences are still being characterized.

A training total can count repeated passes over the same material. It gives us a measure of scale and leaves open the number of distinct works, the share of human writing covered and who agreed to be included.

GOBLIN FACTS — tokens are the measuring tape. Tokens are chunks of text. The total tells you how much text a run processed in those units. To count the books on the shelf, you still need an inventory.

The distinction public discussion collapses — "open" versus "freely scraped"

Lire cette section en français

One of the most consequential confusions in AI policy discussion is the collapsing of two very different things: data that is freely accessible and data that is freely usable. The distinction is anything but academic: it underwrites the entire creator-rights conversation in Canadian AI policy and globally.

Freely accessible means the data is available without paywall or login requirement. Most of the public internet is freely accessible in this sense. You can visit a news article, read it, see the photographs, and process its content yourself without paying.

Freely usable is a much narrower category. The Wikipedia article you can read is also freely usable, because Wikipedia is published under Creative Commons Attribution-ShareAlike (CC BY-SA), which explicitly grants permission for reuse with attribution and same-license requirements. The arXiv physics paper you can read is also freely usable, because the authors agreed to its open distribution terms when posting. The MIT-licensed code on GitHub is freely usable, with the conditions specified in the MIT license. But the news article you can read on the Globe and Mail's website is not freely usable. The newspaper has published it for public reading. The newspaper has not licensed it for arbitrary downstream use, including the training of commercial AI systems.

EXAMPLE — the street-parked car. A car parked on a public street is freely visible. That does not make it freely usable. The same distinction is doing work when a website can be read by the public but has not licensed its content for commercial model training.
🧌 GOBLIN CHECK — "Publicly available" is the AI industry's favourite phrase, and the goblin would like you to notice that it means we could reach it, not we were allowed to keep it. Your car is publicly visible. This has historically not settled the question of who gets to drive it away.

The distinction matters because the AI industry's defense of training-without-permission rests substantially on conflating these categories. "We trained on publicly available data" — OpenAI's stated position, the position of multiple defendant companies in the active Canadian and US copyright lawsuits — is a true statement that elides the distinction between publicly accessible and publicly usable. The news articles, the books, the academic papers, the artist portfolios, the photographer's websites are publicly accessible. They are mostly not publicly licensed for AI training.

ALIGNMENT — available for what? "Publicly available" quietly answers the wrong question. A thing can be free to read and still not free to copy into a commercial training set. When a company says it trained on public data, ask: available to look at, or licensed to use? The gap between those two is where most of the copyright fight lives.

The contested legal question, in Canada and globally, is whether existing copyright law permits AI training as a fair-dealing or fair-use exception, or whether AI training requires explicit licensing of the kind the AI industry has mostly not pursued. The contested fact-question (what data was used) is partly hidden by corporate non-disclosure. The contested legal question (whether the use was permissible) depends on facts that are partly hidden. The opacity is structural to the conflict.

Two distinguishing categories of "publicly available" content the guide will track separately throughout:

Open and licensed for reuse: Wikipedia (CC BY-SA), arXiv preprints (typically CC BY or compatible), most open-source code (MIT, Apache, GPL with conditions), Project Gutenberg public-domain books, government works (varying jurisdictional rules), and similar deliberately-released content. Training on this category raises fewer issues, because the rights holders have, explicitly or by license, granted reuse permission. But even here, license conditions matter: CC BY-SA requires the same license to be applied to derivative works, GPL has copyleft provisions, MIT requires attribution. Whether AI models comply with these conditions when they output text resembling training-data material is a complicated downstream question.

Publicly accessible but not licensed for AI training: news articles, copyrighted books, commercial photography, professional artwork, scientific papers behind paywalls (the publishers' position) or in open-access journals (the publishers' more contested position about whether open-access permits AI training), social media posts that platforms have not licensed for downstream use (most of Reddit before 2024, most of Twitter/X before Musk, most of Facebook's content), and the long tail of content posted by individuals without explicit licensing intent. This is the contested category. The AI industry has generally trained on it. The rights holders are increasingly objecting.

A third category worth naming: data that has been proactively licensed for AI training, through deals between AI companies and content owners. Reddit-to-Google (2024, US$60M/year), the AP-OpenAI deal, the Financial Times-OpenAI deal, the various academic publisher-to-Microsoft agreements, the recent New York Times-Amazon agreement (after the Times sued OpenAI). These deals are growing as content owners realize they have leverage. The deals also confirm, by their existence, that the industry recognizes the underlying material was previously being used without licensing. The licensing market now emerging can reasonably be read as the AI industry hedging its own legal position — you don't usually pay for what you're confident you may take for free. (The industry would call it risk management and input procurement. Both readings fit the same facts; the guide's lean here is declared.)


The human labour layer

Lire cette section en français

Source: Fairwork field study of data labelling workers at Sama

A specific dimension of training data that public discussion routinely misses: most large AI models depend on substantial human labour, often paid at very low wages, often in the Global South, often involving exposure to traumatic content. This is not a peripheral fact about how AI gets built. It is structural to current AI training practice.

Two specific layers:

Data labelling. For supervised learning tasks, including most computer vision and many specialized AI capabilities, humans must label training data. Image labelling tasks like "click on all the pixels showing a car" are familiar to anyone who has completed a CAPTCHA. The same task scaled to professional labelling for AI training is performed by hundreds of thousands of workers globally, primarily through platforms like Amazon Mechanical Turk, Scale AI, Surge AI, and Toloka, with concentration in Kenya, Philippines, India, and other low-wage jurisdictions. The documented wages run low: TIME's 2023 investigation of OpenAI's Sama contracting in Kenya reported take-home pay of roughly US$1.32 to US$2 per hour, and 2023 Washington Post reporting on Scale AI's Remotasks platform in the Philippines documented payments that arrived late, below posted rates, or in some cases not at all. The labour is precarious by design: paid per task, without benefits, and typically without the protections that come with employment status.

Content moderation for alignment. When OpenAI fine-tuned ChatGPT to refuse harmful requests, the training data for that refusal capacity was generated by human workers reviewing extremely disturbing content (descriptions of violence, abuse, child exploitation, terror) and rating model responses as appropriate or inappropriate. **The 2023 TIME investigation** documented OpenAI's contracting through Sama in Kenya, paying workers under US$2/hour to review graphic and traumatic content for the alignment training data that made ChatGPT safer for end users. Workers reported lasting psychological harm. Similar concerns have been documented elsewhere: content-moderation contractors working on Meta's platforms brought litigation in Kenya over trauma exposure and pay, and contract raters working on Google's AI products have publicly disputed their wages and conditions, though the specific arrangements vary by company.

The structural fact: the AI products marketed in Canada as cutting-edge automation are, at their training-data layer, built on substantial human labour. That labour is concentrated in jurisdictions with weaker labour protections, paid at wages substantially below what equivalent work would pay in Canada or the United States, and increasingly involving exposure to traumatic content. The "AI-replaces-workers" framing is partial. AI replaces some kinds of workers in some kinds of jobs while simultaneously depending on other kinds of workers in other kinds of jobs, with the second set largely invisible from the framing.

Chapter 16 puts this work beside union reports of job losses and changes in workplace monitoring. We can ask what the new work pays and what conditions come with it. To find out whether the people who lost jobs ended up doing that work, we would have to follow the same workers.


Whaanga's framing — AI as a new colonizer

Lire cette section en français

The Indigenous Protocol and AI Position Paper (2020), introduced in Chapter 1 and deepened in Chapter 9, contains an essay by Hēmi Whaanga (Ngāti Kahungunu, Ngāi Tahu, Ngāti Mamoe, Waitaha; a Māori linguist and te reo Māori specialist, now Professor and Head of School at Te Pūtahi-a-Toi, Massey University in Aotearoa New Zealand) with a framing that bears specific examination in the data chapter. The essay's title: "AI: a new (r)evolution or the new colonizer for Indigenous peoples?"

Whaanga's argument, distilled: AI systems are "existentially dependent on sucking up vast amounts of data." When that data includes Indigenous knowledge, language, cultural practices, environmental information, and traditional knowledge (and most large foundation models do include this content, because they are trained on the broad internet, including any Indigenous-authored or Indigenous-described material that has been digitized), the result is structurally extractive in ways that map directly onto historical patterns of colonial knowledge extraction.

A non-Indigenous-led guide cannot speak on behalf of this framing. What it can do is present the operational implications. Three are direct:

The training-data layer. The public web can include writing by Indigenous people, material about communities and records published by outsiders. A reference to a web crawl cannot tell communities which materials went in or who authorized the use. OCAP®, the National Inuit Strategy on Research and CARE give us distinct ways to ask who decides, who benefits and what responsibilities follow.

The model-output layer. If an archive gives more space to outside accounts than to the people it describes, there is a risk that a model trained on it amplifies that imbalance. To establish it for a specific system, test the answers and compare them with available evidence about training and retrieval.

The new-knowledge-creation layer. If AI tools are increasingly used in education, research, communication, and creative production, and if those tools have absorbed Indigenous content under conditions that violate Indigenous data sovereignty principles, then the use of those tools to create new content about Indigenous peoples or in dialogue with Indigenous communities perpetuates the underlying violation. Each new piece of content is built on a substrate that did not engage Indigenous frameworks for using Indigenous data.

This is the substantive engagement Whaanga's framing requires. The label "AI as new colonizer" is not rhetorical. It is a structural claim about how AI training data works in relation to Indigenous knowledge, what the alternative would look like (training models that engage the data sovereignty frameworks rather than ignoring them), and what the costs of not engaging are. The guide takes the claim seriously without claiming to speak for it, and asks readers, and Canadian AI policymakers, to do the same.

A specific Canadian connection. The Abundant Intelligences research network introduced in Chapter 1 (co-led from Concordia, with the Lethbridge pod and international partners) is doing concrete technical work on what AI training that did engage Indigenous data sovereignty frameworks might look like: small models trained on data with proper consent, models built using Indigenous epistemological assumptions, prototypes that demonstrate alternatives to the extract-and-scale-up paradigm. The work exists. The federal AI strategy could engage it. AI for All does not currently commit to doing so.


The cost layer — what training data extracts and what it returns

Lire cette section en français

Putting the previous sections together produces a working picture of what AI training data actually represents at an aggregate level. The picture is not the marketing version.

What training data extracts:

The accumulated written, visual, and audio output of human beings over centuries, where digitized and accessible. The professional output of writers, artists, photographers, journalists, programmers, scientists, scholars, and amateur enthusiasts. The cultural and knowledge content of Indigenous peoples globally, accumulated through colonial-era research practices and contemporary online presence. The human labour of hundreds of thousands of low-wage workers, primarily in the Global South, performing labelling and content moderation. The physical-substrate materials (cobalt, lithium, coltan, gallium — Chapter 8's Owen et al. finding) extracted to build the compute infrastructure that processes the training. The electricity, water, and embedded carbon of the training itself.

What training data returns to the people whose work, knowledge, and labour are in it:

For Wikipedia contributors and open-source developers: licences can spell out conditions for reuse. The training disclosures cited here do not establish how often those conditions were met.

For published authors and journalists: some licensing agreements are public. The Canadian news lawsuit documents an allegation of unauthorized use. Those examples give us no industry-wide measure of consent or compensation.

For Indigenous peoples whose knowledge may be in a dataset: ask for evidence of community authorization, benefit-sharing and ongoing governance. The public disclosures discussed here leave the frequency of those arrangements unknown.

For data-labelling and content-moderation workers: in the documented cases (TIME's 2023 reporting on OpenAI's Sama contracting in Kenya, the Washington Post's 2023 reporting on Scale AI's Remotasks), pay as low as roughly US$1.32 to US$2 per hour, typically without benefits or employment-status protections, and in some cases lasting psychological harm.

For the communities living near the extraction sites of the physical-substrate materials: in many cases, environmental degradation, displacement, and disruption of traditional land use.

For the general public: access to AI products, sometimes free at the point of use, that generate value primarily for the corporations that produced the models.

The economics, in compressed form: the AI industry extracts substantial value from data, labour, knowledge, and material substrates contributed by populations who are mostly not compensated, who mostly did not consent, who mostly do not have governance over the resulting models, and who are largely invisible from the marketing narrative. The industry returns substantial value to investors, executives, and (in subset of cases) end users. None of this is concealed. It is simply how current AI gets built, plainly traceable by anyone who follows the supply chain from chip extraction through labour markets through training data through model output to where the revenue lands.

Any account of AI’s costs is thinner if it leaves out the work, the resources and the permissions that went into the training data. The same goes for a policy that starts at the model’s output and ends there.


The working test for evaluating training data claims

Lire cette section en français

Putting the chapter's material together, the practical test the guide asks readers to apply to any AI training-data claim:

Source. What data was specifically used? Has the developer published a comprehensive list, a representative sample, or only vague characterizations? If the latter, why?

Consent. Was the consent of the rights-holders sought before training? If yes, through what mechanism — explicit licensing, opt-out, or assumed under fair-use/fair-dealing? If no, what is the developer's stated legal position on why consent wasn't required?

Compensation. Have rights-holders been compensated for the use of their material? If yes, through what mechanism — direct licensing, collective licensing, or revenue-sharing? If no, what is the developer's position on whether compensation should be required?

Indigenous data. Has Indigenous content been included in training data? If yes, were OCAP/NISR/CARE Principles engaged? If no engagement, what was the developer's reason? Is there a clear separation in the training corpus between Indigenous content used with consent and Indigenous content used without?

Verification. Are the developer's claims about training data composition, consent, and compensation independently verifiable? Has a third party audited the training corpus? If not, the claims are corporate self-disclosure and should be read accordingly.

Output traceability. Can model outputs that resemble specific training-data material be traced back to that material? Current technical capability here is limited but improving. Where output-to-source attribution is possible, the developer's framing of "novel generation" versus "reproduction" can be tested empirically rather than rhetorically.

Most current AI products will fail most of these questions. The guide's recommendation is not that readers refuse to use AI products; readers can make their own choices about that. The guide's recommendation is that readers, and the policy-makers shaping Canadian AI strategy, read the failures as the structural conditions of how current AI is being built, and decide whether those conditions are acceptable or whether they should be changed.


The receipt on the raw material

Lire cette section en français

CHAPTER RECAP — you now have: - A map of documented training sources, from web crawls and books to code and image indexes, and the difference between a source collection and what a particular model was trained on. - The distinction between freely accessible and freely usable that public discussion routinely collapses — and the operational basis of the Canadian creator-rights conversation. - The human-labour layer of training-data preparation, including the low-wage data-labelling workforce in the Global South and the documented Kenyan content moderation contracting for OpenAI's alignment training. - Whaanga's "AI as new colonizer" framing engaged substantively — the training-data-layer, model-output-layer, and new-knowledge-creation-layer implications of training models on Indigenous content without engaging Indigenous data sovereignty frameworks. - The work, knowledge and resources training draws on, and how to ask what contributors received and what remains undisclosed. - The working test for evaluating any AI training-data claim: source, consent, compensation, Indigenous data engagement, verification, output traceability.

The data has to go somewhere. Chapter 4 follows it into the racks, then traces the hardware back through its supply chain. Suzanne Kite’s Lakȟóta protocol asks what we owe the people and places that make the hardware possible.

You can now read any AI training-data claim with the structural equipment to ask the right questions. Most current claims will fail most of those questions. That failure is the structural condition of how current AI is being built, and engaging it honestly is the precondition for any serious Canadian AI policy.


Bias label for this chapter: structural-political analysis of AI training data, with explicit naming of the gap between corporate self-disclosure and operational reality. Author lean: skeptical of "publicly available" framings that elide the freely-accessible-vs-freely-usable distinction; sympathetic to creator-side and Indigenous-led critiques of training-without-consent; willing to name the human-labour layer in the Global South as structural to current AI rather than peripheral; explicit that the manual is not Indigenous-led and cannot speak on behalf of the Whaanga framing while engaging it substantively. Corporate self-disclosure (Stanford FMTI findings, lawsuit defendant positions) labelled and read accordingly. Indigenous-led primary sources (Whaanga essay, OCAP/NISR/CARE frameworks) treated as authoritative within their own frames.

Primary sources cited or relied on in this chapter: Stanford Center for Research on Foundation Models, 2025 Foundation Model Transparency Index; Canadian newspapers v. OpenAI (Ontario Superior Court CV-24-00732231-00CL); Books3 corpus litigation including Silverman et al. v. OpenAI; New York Times v. OpenAI; LAION dataset withdrawal documentation (Stanford Internet Observatory, December 2023); Shumailov et al., "AI models collapse when trained on recursively generated data" (Nature, 2024); TIME magazine investigation of OpenAI's Sama contracting (January 2023); Whaanga, "AI: a new (r)evolution or the new colonizer for Indigenous peoples?" (Indigenous Protocol and AI Position Paper, 2020); First Nations Information Governance Centre, OCAP Principles; Inuit Tapiriit Kanatami, National Inuit Strategy on Research (2018); Global Indigenous Data Alliance, CARE Principles; Reddit-Google licensing deal disclosure (2024). Detailed citations in the Sources appendix.


🧌 GOBLIN CHECK — "Publicly available" is the AI industry's favourite phrase, and the goblin would like you to notice that it means we could reach it, not we were allowed to keep it. Your car is publicly visible. This has historically not settled the question of who gets to drive it away.

Recap

  • A map of documented training sources, from web crawls and books to code and image indexes, and the difference between a source collection and what a particular model was trained on.
  • The distinction between freely accessible and freely usable that public discussion routinely collapses — and the operational basis of the Canadian creator-rights conversation.
  • The human-labour layer of training-data preparation, including the low-wage data-labelling workforce in the Global South and the documented Kenyan content moderation contracting for OpenAI's alignment training.
  • Whaanga's "AI as new colonizer" framing engaged substantively — the training-data-layer, model-output-layer, and new-knowledge-creation-layer implications of training models on Indigenous content without engaging Indigenous data sovereignty frameworks.
  • The work, knowledge and resources training draws on, and how to ask what contributors received and what remains undisclosed.
  • The working test for evaluating any AI training-data claim: source, consent, compensation, Indigenous data engagement, verification, output traceability.

Sources

  • Stanford Center for Research on Foundation Models, 2025 Foundation Model Transparency Index
  • Canadian newspapers v. OpenAI (Ontario Superior Court CV-24-00732231-00CL)
  • Books3 corpus litigation including Silverman et al. v. OpenAI
  • New York Times v. OpenAI
  • LAION dataset withdrawal documentation (Stanford Internet Observatory, December 2023)
  • Shumailov et al., "AI models collapse when trained on recursively generated data" (Nature, 2024)
  • TIME magazine investigation of OpenAI's Sama contracting (January 2023)
  • Whaanga, "AI: a new (r)evolution or the new colonizer for Indigenous peoples?" (Indigenous Protocol and AI Position Paper, 2020)
  • First Nations Information Governance Centre, OCAP Principles
  • Inuit Tapiriit Kanatami, National Inuit Strategy on Research (2018)
  • Global Indigenous Data Alliance, CARE Principles
  • Reddit-Google licensing deal disclosure (2024).