Zahnow Library guides/Applied AI literacy
LEARN · EXPERIMENT · REFLECT

Applied AI lessons

10 lessons and six hands-on experiments. Work through them in order, at your own pace.

Lesson 3 of 10

Measuring AI and the AGI debate

In plain termsA benchmark is a standardized test for AI, like an exam that every model takes so we can compare them. Scores tell you what a model can do on that test, not whether it will be right about your assignment. “Time horizon” measures how long an agent can work on a task by itself before it needs help — a useful way to see how fast agents are improving.

Measuring AI

How do we grade AI? Benchmarks and time horizons

A benchmark is a standardized test for AI. Here is how GPT-6 Astra did — and how long AI agents can now keep working on their own.

98%FrontierMath Tier 4Very hard, research-level math problems
99.9%ARC-AGI-3Puzzle games that test learning new skills. 62.7% with the standard setup
166Epoch Capabilities IndexOne score combining many tests. #1 at launch, #2 of 257 models now

Time horizon: how long a task (in human-expert time) an AI agent finishes successfully half the time. It has doubled about every 7 months since 2019 — and about every 4 months since 2023 on the updated suite.

Task length AI completes at 50% success

  1. GPT-4 (2023)4 min
  2. GPT-4o (2024)7 min
  3. o1 (2024)39 min
  4. o3 (2025)~2 h
  5. Opus 4.5 (2025)~5 h
  6. Opus 4.6 (2026)~12 h
  7. Mythos (2026)~17 h

Takeaway: High scores are not the same as good judgment. Test it on your own work. (Mostly coding and research tasks; 50% success is a coin flip; numbers shift as METR updates its method.)

Sources: OpenAI (Sept 2026); ARC Prize (Sept 3, 2026); Epoch AI; METR (metr.org/time-horizons, 2026); Kwa et al. (2025). Some model values are secondary; verify before quoting.

A magnifying glass over a checked exam, with a scale and stopwatch

In plain termsAGI, or artificial general intelligence, usually means AI that can do most mental tasks as well as people can. Experts disagree on whether we are close, already there, or still far away — partly because they define it differently. When you read a bold AGI headline, ask: who is saying it, how are they defining it, and what would they gain if you believed them?

The AGI debate

Is AGI already here? Four voices say yes, with fine print

AGI has long been the stated mission of frontier labs. Alan Turing proposed the imitation game in 1950. Within a year, the idea moved from science fiction to analyst, journal, and CEO statements.

Nature · Feb 2026

Peer-reviewed essay

Chen, Belkin, Bergen and Danks argue AGI may already have arrived: “we have artificial systems that are generally intelligent.” An argument, not a forecast.

Jensen Huang · Mar 23, 2026

NVIDIA CEO

“I think it’s now. I think we’ve achieved AGI.” Based on Lex Fridman’s narrow test: an AI that can start and grow a $1B tech company.

Greg Brockman · Sept 2026

OpenAI president

On GPT-6 Astra: “I think it might be about this model.” Hedged, not a formal declaration. ARC-AGI-3: 99.9% (OpenAI), 62.7% (ARC).

Forrester · Sept 10, 2026

Analysts

“AGI is here, but it’s not superintelligence; in fact, it is just barely competent.” Forrester had predicted “competent AGI” for 2026–2030.

Fine print

  • There is no universal definition of AGI (Fortune, Mar 30, 2026). Monash’s Prof. James Bailey: the launch “doesn’t show that AGI has arrived.”
  • Huang’s claim rested on Fridman’s narrow $1B-company test; Gary Marcus said “no evidence and no definitions.”
  • Independent evaluators say headline scores rely on heavy optimization (Live Science, Oct 8, 2026).
  • Whether or not you call it AGI, the tools are real, and you can use them today.

The new finish line: superintelligence

OpenAI

Says it is turning its aim toward superintelligence (Altman, Jan 2025).

Meta

Created Meta Superintelligence Labs in 2025.

Safe Superintelligence

Founded by Ilya Sutskever in 2024 with one goal.

Microsoft AI

Building a “humanist superintelligence” team (Suleyman, 2025).

Kurzweil predicted it. The Atlantic says we’re in it.

Ray Kurzweil’s predictions

2029: AI reaches human-level intelligence. 2045: the Singularity, when human and machine intelligence merge. From The Singularity Is Near (2005), reaffirmed in The Singularity Is Nearer (2024).

The Atlantic’s take

“Whatever the AI future is, we’re in it right now.” — Matteo Wong and Charlie Warzel, Sept 2026. Even Sam Altman said: “We’re now in the singularity” (as quoted, July 2026).

Takeaway: Presenter interpretation: for students, the debate is the signal — cross-domain tools are now a baseline. Same race, higher stakes: nobody agrees what happens if someone gets there.

Sources: Nature 650, 36–40 (Feb 2026); Fortune (Mar 30, 2026); Forrester (Sep 10, 2026); Live Science (Oct 8, 2026); Global Future Nexus (Sep 5, 2026); company statements 2024–2025; Kurzweil (2005, 2024); The Atlantic (Sept 1 and July 30, 2026).

In plain termsThere are no right answers to these questions, and that’s the point. Talking them through with a classmate, or writing a short reflection, helps you form your own view instead of borrowing one from a headline.

Discuss

What do you make of the AI race?

Use these questions in class, a study group, or a reflection journal.

  1. Did you know AI was a race? Does that change things?
  2. Should global superpowers chase superintelligent AI?
  3. What does this race mean for you as a student, worker, or creator?
  4. What do you hope AI does for your field? What worries you?

Check your understanding

1. A new model “beats the benchmark.” What does that actually tell you?
2. Two experts disagree about when AGI will arrive. What should you take from that?