08-28-2026, 03:24 PM
Are We Thinking Correctly About AI Intelligence?
Authors: Steven Strogatz and Janna Levin
Source:Quanta Magazine — The Joy of Why
Published: August 20, 2026
Melanie Mitchell argues that we may be asking the wrong question when we try to decide whether modern AI systems “think” or “reason” like humans. Large language models constitute something closer to an alien form of intelligence: they have been trained on enormous quantities of human-produced language and knowledge, yet the mechanisms by which they learn and generate solutions are fundamentally different from human cognition. Because fluent language strongly encourages us to anthropomorphize AI, impressive benchmark performance should not automatically be interpreted as evidence of human-like understanding. Mitchell proposes that AI research borrow experimental methods from developmental and comparative psychology, fields that already study intelligences that cannot simply explain their own internal processes — babies and animals.
A central distinction is between performance and competence. An AI may correctly solve a problem without possessing the generalized understanding we would attribute to a human who solved it. Mitchell compares this with a student who memorizes a solution but fails when the problem is slightly altered. She invokes the famous case of Clever Hans, the horse apparently capable of arithmetic who was actually responding to unconscious signals from humans. AI benchmarks can suffer from the same problem: a model may exploit correlations, clues or artifacts in the test rather than the cognitive ability the experiment supposedly measures. For this reason, Mitchell emphasizes controlled experiments, novel variations of benchmark questions, independent replication and careful investigation of failures.
The issue becomes particularly important in mathematics. Modern AI systems have shown striking abilities to solve Olympiad problems and even contribute to research-level mathematics by combining ideas from different areas. Mitchell accepts that such behavior can reasonably be called creative, but warns that we still know very little about how the systems arrive at their successful ideas or how many unsuccessful reasoning paths precede them. Moreover, success at a collection of tasks does not mean an AI can replace the human profession associated with those tasks. A system that performs well on radiology tests is not automatically capable of doing everything a radiologist does; similarly, proving difficult theorems is not equivalent to choosing fruitful mathematical questions, creating concepts, developing theories or connecting mathematics with physical experience.
Mitchell therefore proposes six principles for scientifically assessing AI cognition:
Key takeaways
A particularly important idea from the article can be summarized as:
ARTICLE [PDF]
Authors: Steven Strogatz and Janna Levin
Source:Quanta Magazine — The Joy of Why
Published: August 20, 2026
Melanie Mitchell argues that we may be asking the wrong question when we try to decide whether modern AI systems “think” or “reason” like humans. Large language models constitute something closer to an alien form of intelligence: they have been trained on enormous quantities of human-produced language and knowledge, yet the mechanisms by which they learn and generate solutions are fundamentally different from human cognition. Because fluent language strongly encourages us to anthropomorphize AI, impressive benchmark performance should not automatically be interpreted as evidence of human-like understanding. Mitchell proposes that AI research borrow experimental methods from developmental and comparative psychology, fields that already study intelligences that cannot simply explain their own internal processes — babies and animals.
A central distinction is between performance and competence. An AI may correctly solve a problem without possessing the generalized understanding we would attribute to a human who solved it. Mitchell compares this with a student who memorizes a solution but fails when the problem is slightly altered. She invokes the famous case of Clever Hans, the horse apparently capable of arithmetic who was actually responding to unconscious signals from humans. AI benchmarks can suffer from the same problem: a model may exploit correlations, clues or artifacts in the test rather than the cognitive ability the experiment supposedly measures. For this reason, Mitchell emphasizes controlled experiments, novel variations of benchmark questions, independent replication and careful investigation of failures.
The issue becomes particularly important in mathematics. Modern AI systems have shown striking abilities to solve Olympiad problems and even contribute to research-level mathematics by combining ideas from different areas. Mitchell accepts that such behavior can reasonably be called creative, but warns that we still know very little about how the systems arrive at their successful ideas or how many unsuccessful reasoning paths precede them. Moreover, success at a collection of tasks does not mean an AI can replace the human profession associated with those tasks. A system that performs well on radiology tests is not automatically capable of doing everything a radiologist does; similarly, proving difficult theorems is not equivalent to choosing fruitful mathematical questions, creating concepts, developing theories or connecting mathematics with physical experience.
Mitchell therefore proposes six principles for scientifically assessing AI cognition:
- Beware of anthropomorphic bias.
- Be skeptical and use proper control experiments.
- Test new variations to measure robustness and generalization.
- Probe the internal and external behavior of models rather than simply treating them as inscrutable black boxes.
- Distinguish performance from genuine competence.
- Study failures and negative results as carefully as successes.
Key takeaways
- AI intelligence may be genuinely powerful without being human-like.
- Benchmark success is evidence of performance, not automatically evidence of understanding.
- The most important question is increasingly not “Can the AI solve this?” but “Why can it solve it, and under what variations does that ability survive?”
- Mathematics could be transformed substantially by AI, but solving mathematical problems is only one component of what mathematicians actually do.
- Mitchell believes mechanistic interpretability — understanding the internal computations of neural networks — could eventually give us something analogous to neuroscience for artificial minds.
A particularly important idea from the article can be summarized as:
Quote:AI can give the right answer for reasons completely different from the reasons we assume.
ARTICLE [PDF]
┌────────────────────────────────┐
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘