Resources for teachers / sources

Sources

Every number in the talk “Fast Tools, Slow Thinking” and on these pages, with where it comes from and what year. They are grouped by slide, in the order of the talk. If you find a number that is not here, tell us.

The last column says how well a number was checked:

  • Checked against the primary source
  • Secondary report; not checked against the primary
  • My own data or records; no outside check
  • Counted from this site
  • Well-known background; not re-checked

Numbers marked “my own data” come from my own records. The class numbers (finishing times, the questions) are from one graduate course of 12 students: they show what happened in one small class, not what would happen in yours. Found a mistake? Tell us and we will fix it in public.

Teens and AI

Number Source Year How checked
54% of US teens used an AI chatbot for schoolwork (fall 2025) Pew Research Center, teens and AI 2026 (survey fall 2025) Checked against the primary source
About 1 in 10 teens (10%) do all or most of their schoolwork with a chatbot; 21% some; 23% a little Pew Research Center, teens and AI (same report) 2026 Checked against the primary source

Where I’ve landed

No outside numbers. These are my own conclusions.

This is where you work

Number Source Year How checked
Teachers work about 49 hours a week, around 10 over contract RAND, State of the American Teacher 2025 Secondary report; not checked against the primary
Teachers spend about $895 of their own money a year AdoptAClassroom, teacher survey (covers the 2024–25 school year) 2025 Checked against the primary source
22.6% of US students were chronically absent AEI and Chalkbeat reporting on chronic absence (2024–25 school year) 2026 Checked against the primary source

The answer just became free

Number Source Year How checked
Chegg’s revenue fell from $617.6 million to $376.9 million in one year, down 39% Chegg, Form 10-K for fiscal 2025 2025 Checked against the primary source
College students paid $341 a year for course materials in 2024–25, about half the $701 of 2007–08 NACS (National Association of College Stores), Student Watch 2024–25 2025 Checked against the primary source
Publishers’ higher-education revenue was $4.7 billion, up 8.5% Association of American Publishers, StatShot annual report 2025 Checked against the primary source

What we mean by AI

Number Source Year How checked
354 lessons Biochemistrypedia repository history and content (origin/main), counted October 23, 2026 at 6:35 pm ET 2026 Counted from this site

How fast: 17 weeks, then 13 days

Number Source Year How checked
First commit June 12, 2026. The first 30 lessons took about 17 weeks (to October 10) Biochemistrypedia repository history and content (origin/main), counted October 23, 2026 at 6:35 pm ET 2026 Counted from this site
324 more lessons in 13 days (October 11 to 23) Biochemistrypedia repository history and content (origin/main), counted October 23, 2026 at 6:35 pm ET 2026 Counted from this site
354 lessons, all live, in 11 courses, with 464 films (lesson explainers, scientist profile films and methods films) Biochemistrypedia repository history and content (origin/main), counted October 23, 2026 at 6:35 pm ET; films counted from the media manifest 2026 Counted from this site
Grade 7 Life Science: 34 lessons in 8 units (LS.1 to LS.11), planned and released on October 23 (first commit 11:40 am, last lesson merged 6:34 pm ET) Biochemistrypedia repository history and content (origin/main), counted October 23, 2026 at 6:35 pm ET 2026 Counted from this site
High school Biology: 33 lessons (BIO.1 to BIO.8), all released October 20 Biochemistrypedia repository history and content (origin/main), counted October 23, 2026 at 6:35 pm ET 2026 Counted from this site

The 60-second film

Number Source Year How checked
354 lessons in 11 courses, 464 short films (repeated from the film’s narration) Biochemistrypedia repository history and content (origin/main), counted October 23, 2026 at 6:35 pm ET 2026 Counted from this site

500 students wrote a book with me

Number Source Year How checked
Written with more than 500 students of UVA Biochemistry; 133 profiles of 190 scientists; free as PDF, EPUB and audiobook The Molecule Hunters, deppmannlab.com/book (read October 23, 2026) 2026 Checked against the primary source

What building taught me

No new numbers. 354 lessons is counted above.

Checking: 49 wrong, 453 imprecise

Number Source Year How checked
An AI fact-checker with web access read 3,468 claims in one course: 49 wrong, 453 imprecise, 287 it could not verify; 841 edits My fact-check findings for the Cell Signaling course (44 lessons), 2026 2026 My own data or records; no outside check
Wrong claims per lesson: 4.0 in the first batch (9 lessons), 1.5 in the next, 0.0 to 0.6 in Units 3 to 8 (about 0.3 on average). Said in the talk as "four a lesson at first, under one by the end" My fact-check findings for the Cell Signaling course (44 lessons), 2026 2026 My own data or records; no outside check
For Genetics, a second check by a different company’s model found 80 more corrections, for example a trisomy-21 DNA share corrected from 1.5% to 0.8% Genetics course fact-check, second-opinion pass (repository commit bf4fc5b) 2026 My own data or records; no outside check
Expert review takes about 1 to 2 hours per lesson My own estimate while reviewing the lessons; not a timed study 2026 My own data or records; no outside check

Thursday: 2 finished, then 9

Number Source Year How checked
Pre-work module: 36 steps, about 3 to 4 hours for a careful student. Two of 12 students had finished by Thursday morning; by Friday 8:40 am, nine had and three were partwayOne small graduate class. Not a sample of any other group. BIOL 7240 / PFN Foundations of Neuroscience I, University of Virginia graduate course, Fall 2026 (12 students, anonymized) 2026 My own data or records; no outside check
The “Room” slide: 2 of 8 students said yes, 6 said no, to whether blocking dynein proves TrkA is the cargo BIOL 7240 / PFN Foundations of Neuroscience I, University of Virginia graduate course, Fall 2026 (12 students, anonymized) 2026 My own data or records; no outside check
One predict, argue, reveal cycle took 8 to 10 minutes in the room, not the 4 I had planned BIOL 7240 / PFN Foundations of Neuroscience I, University of Virginia graduate course, Fall 2026 (12 students, anonymized) 2026 My own data or records; no outside check
Lineage of the idea: Just-in-Time Teaching (Novak and colleagues, 1999); formative assessment (Black and Wiliam, 1998) Novak et al. 1999; Black and Wiliam 1998 1999; 1998 Well-known background; not re-checked

AI-written exams can be ambiguous

Number Source Year How checked
Spring 2026 Biochemistry: about 735 students per exam, with a question-review request form after each examOne course. It shows what happened there, not what would happen in yours. My own course records (Biochemistry assessments, spring 2026), no student identifiers 2026 My own data or records; no outside check
Review requests per exam: 28 (Exam 1, from 21 students), 44 (Exam 2, from 31 students), 41 (Exam 3, from 30 students) My own course records, spring 2026 2026 My own data or records; no outside check
Two questions were credited to every student: an ambiguous transport question (SGLT1 and GLUT1) on Exam 2, and an "All of the above" question that changed meaning after answer shuffling on Exam 3 (22 of that exam’s 41 requests were about it) My own course records, spring 2026 2026 My own data or records; no outside check
What changed afterward: a red-team review of AI-written questions (answer-length cues, leakage between questions), then "All of the above" banned and distractor lengths balanced from Exam 4 My own course practice, spring 2026 2026 My own data or records; no outside check

Questions nobody read

Number Source Year How checked
About a quarter (20 to 25%) of students’ own questions went beyond the lesson; 4 of 12 students asked about 70% of them BIOL 7240 / PFN Foundations of Neuroscience I, University of Virginia graduate course, Fall 2026 (12 students, anonymized) 2026 My own data or records; no outside check

Shift 1: Meet every student where they are.

Number Source Year How checked
Adaptive software that taught each student at their own level raised math scores 0.36 standard deviations and Hindi 0.22 after about 4.5 months (Mindspark, Delhi). The authors describe "very wide variation in student learning levels within a single grade" Muralidharan, Singh and Ganimian, American Economic Review (NBER working paper w22923, abstract) 2019 Checked against the primary source
Teaching at the Right Level is a "Great Buy" Smart Buys report, Global Education Evidence Advisory Panel 2023 Secondary report; not checked against the primary
A later review finds the effects heterogeneous and often modest Ardington, Menendez and Thunde, review of Teaching at the Right Level 2026 Secondary report; not checked against the primary
A depth ladder with four steps (grade 7 Life Science, Biology, Biology II, College) on every high school Biology lesson, and a three-setting supports switch Biochemistrypedia repository history and content (origin/main), counted October 23, 2026 at 6:35 pm ET 2026 Counted from this site

Shift 2: Let curiosity lead the right tail.

Number Source Year How checked
Interest in science falls from about age 9 to 17 Gottfried et al. 2001 Secondary report; not checked against the primary
Curiosity questions: about 2.4 per two hours in kindergarten, about 0.5 in grade 5 Engel 2011 Secondary report; not checked against the primary

Shift 3: AI carries the volume, so your attention goes where it’s needed.

Number Source Year How checked
Teachers who use AI every week save about 5.9 hours a week, mostly on preparation Gallup and the Walton Family Foundation, teachers and AI 2025 Checked against the primary source
Just-in-Time Teaching, 1999 Novak et al., Just-in-Time Teaching 1999 Well-known background; not re-checked

Shift 4: A tutor the school owns, so you can see the thinking.

Number Source Year How checked
Unguarded GPT-4: practice scores up 48%, later exam scores down 17% Bastani et al., Proceedings of the National Academy of Sciences 2025 Secondary report; not checked against the primary
A guarded tutor did not show that harm; a Harvard tutor with guardrails doubled gains on two lessonsAll AI-tutor evidence is short-term and not yet replicated. Kestin et al., Scientific Reports 2025 Checked against the primary source

Shift 5: How they got there counts more than the answer.

Number Source Year How checked
Chegg’s revenue fell from $617.6 million to $376.9 million in one year, down 39% (also under “The answer just became free”) Chegg, Form 10-K for fiscal 2025 2025 Checked against the primary source
22% of answers were revised after the reveal BIOL 7240 / PFN Foundations of Neuroscience I, University of Virginia graduate course, Fall 2026 (12 students, anonymized) 2026 My own data or records; no outside check

One idea. Every level.

A 40-second film with no numbers. It shows one idea at six levels, from toddler to research lab.

When a measure becomes a target

Number Source Year How checked
Goodhart’s law: a statistical regularity tends to collapse once pressure is placed on it for control (UK monetary policy) Charles Goodhart 1975 Checked against the primary source
"When a measure becomes a target, it ceases to be a good measure": the common wording, written about examination performance Marilyn Strathern, p. 308 1997 Checked against the primary source
A similar point about social indicators, including test scores Donald Campbell. Sources date it 1976 to 1979, so the talk says "in the 1970s" 1970s Checked against the primary source
China’s imperial civil-service examinations ran from the Sui dynasty (early 600s) until 1905, about 1,300 years. The talk says "more than a thousand years" General historical knowledge 1905 Well-known background; not re-checked

Understanding is the only thing left worth grading

No new numbers. "More than 500 students" is sourced under the book slide.

It can go wrong

Number Source Year How checked
A course AI tutor lowered grades, most for first-generation studentsHold it loosely: it has not been through peer review. Liu et al., EdWorkingPaper 26-1598 (University of Maryland trial; not peer-reviewed) 2026 Checked against the primary source
Free tools help the already-engaged most Reich, on MOOCs 2020 Secondary report; not checked against the primary

What I’d ask of any AI in a classroom

No numbers. The five commitments are my own.

Other numbers on the /teachers pages

Number Source Year How checked
AI-text detectors flagged 61% of TOEFL essays (written by non-native English speakers) as AI-written (do-it-yourself guide, tip 4) Liang et al., Patterns 2023 Secondary report; not checked against the primary
A $10,000 grant request (July 2026) said $5,000 would buy three to five complete lessons. It was not selected, and the site was built anyway AI Catalyst grant request, July 2026 2026 My own data or records; no outside check
Model scores, token counts and narration costs in “How it’s built” The project files named next to each figure on that page 2026 My own data or records; no outside check
Fact-check numbers in the do-it-yourself guide and in “How it’s built” (3,468 claims, 49 wrong, 453 imprecise, 287 unverifiable, 841 edits; about 4 wrong per lesson at first, under 1 later) My fact-check findings for the Cell Signaling course (44 lessons), 2026 2026 My own data or records; no outside check