TL;DR
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
At an MIT symposium in 2011, Noam Chomsky questioned whether statistical language models that reproduce patterns in data count as scientific explanations. An essay responding to his exchange with Steven Pinker argued that predictive and engineering successes can inform science, while acknowledging that statistical models have limits and that language learning remains unresolved.
At an MIT symposium in 2011, linguist Noam Chomsky challenged the scientific value of statistical language models, arguing that success at matching data is not the same as explaining language. A response published by Peter Norvig argued that such models can contribute to science as well as engineering, while conceding that prediction alone does not reveal the principles behind human language.
The discussion took place during the Brains, Minds, and Machines symposium, held as part of MIT’s 150th-birthday events. According to Norvig’s account, Chomsky was responding to a question from psychologist Steven Pinker about the success of probabilistic models trained with statistical methods. Technology Review reported on the remarks, and Norvig later cited a transcript in his essay.
Chomsky acknowledged that researchers had applied statistical models to linguistic problems and that there had been “some successes,” but also “a lot of failures.” He objected to a conception of success based on “approximating unanalyzed data,” which he called novel in the history of science. Norvig interpreted the criticism as a broader concern that statistical systems may simulate observable behavior without explaining the mechanisms or principles that produce it.
Norvig’s reply distinguished practical performance from scientific explanation without treating them as unrelated. Engineering results, he argued, can be evidence that a model captures something useful about a phenomenon, though not proof that it explains the phenomenon fully. He also contended that scientific progress requires both gathering facts and building theories, and that examining where models succeed, fail, and improve with more data can produce insight even when their internal parameters are too numerous to interpret individually.
Prediction and Explanation in Language
The exchange addresses a lasting question in research: when does a system that predicts or reproduces patterns count as an explanation? For language technologies, statistical methods can help with tasks such as speech recognition, where listeners must infer what a speaker meant from noisy input. Norvig argued that this kind of interpretation is inherently probabilistic, making probability-based models useful tools for both processing language and studying it.
That case does not settle whether a model captures how people understand or produce language. A system can perform well on a task without showing that its internal workings resemble human cognition, or that it has identified the underlying linguistic principles. The distinction matters to readers because claims about an application’s performance and claims about what the application teaches researchers are different claims, requiring different evidence.
Norvig also cautioned against treating one type of model as a complete account. A word-probability model based on nearby words cannot represent every feature of language, he wrote; a compact model built around syntactic trees without probabilities also has limits. His proposed direction was a probabilistic account covering words, syntax, semantics, context and discourse, rather than dismissing probabilistic methods based on the shortcomings of a particular older model.
Popular right now
Hisense 40-Inch Class QD4SR Series Hi-QLED FHD Smart Roku TV (2026 Model)
De'Longhi Stilosa Compact Manual Espresso Machine with Milk Frother
Samsung 75-Inch Class The Frame QLED 4K Art TV (LS03HE, 2026)
As an affiliate, we earn on qualifying purchases.
The Dispute Behind the Exchange
The 2011 conversation sits within a long-running debate about language learning and the role of innate human capacities. Chomsky’s criticism, as Norvig summarized it, was not simply that statistical techniques sometimes fail. It raised a methodological objection: accurate descriptions of linguistic data may amount to cataloguing observations unless researchers also identify the principles that generate them.
Norvig agreed that scientific work cannot be reduced to engineering performance or data matching. He disagreed that these activities are irrelevant to science, arguing instead that observation, modeling and practical tests can develop together. He also noted that a model need not be transparent at the level of every parameter to be studied: researchers can analyze its behavior, its errors and how its performance changes with training data.
The disagreement also concerned what learning theory says about language acquisition. Norvig referred to Gold’s 1967 work on theoretical limits to learning formal languages, while arguing that those results did not establish that natural language cannot be learned statistically. He cited Horning’s 1969 work on learning probabilistic context-free grammars as a counterpoint. These historical results are part of Norvig’s argument, not a resolution of how children acquire natural language.
““I think there have been some successes, but a lot of failures.””
— Noam Chomsky, as quoted in Peter Norvig’s essay
What the Debate Does Not Resolve
The supplied account does not establish whether Chomsky or Norvig’s position has since prevailed, or provide a later assessment of the models discussed. The transcript and essay document a disagreement; they do not settle how much statistical learning can explain about human language, or whether particular models reproduce human learning processes.
Norvig acknowledged that researchers did not know enough about humans’ innate capacity for language to specify how it works. He also said it remained unclear whether that capacity is specific to language or part of a broader learning ability. The source offers arguments and historical references, but no comparative evidence that resolves those questions.
Further Evidence on Language Learning
The source describes a completed symposium discussion and a published response, not a pending decision, scheduled study or announced follow-up. Any assessment beyond the 2011 exchange would require later evidence about how statistical models perform, what their behavior reveals, and how those findings relate to human language learning. No such later development is identified in the material provided.
For readers revisiting the argument, the key distinction remains between demonstrating that a model can perform a language task and demonstrating that it explains the human capacity behind language. The supplied essay makes a case for studying both questions, but leaves the underlying account of language learning open.
Key Questions
What did Chomsky criticize?
He questioned whether matching linguistic data counts as scientific explanation, arguing that models can reproduce behavior without identifying the principles behind it.
What was Norvig’s response?
Norvig argued that engineering performance can provide evidence relevant to science, though it is not proof. He also said researchers can learn from a model’s successes, failures and response to more data.
Did Chomsky say statistical models had no successes?
No. In the wording quoted by Norvig, Chomsky said there had been “some successes” as well as “a lot of failures.”
Does the exchange settle whether language is learned statistically?
No. Norvig argued that statistical learning could be part of the process, but acknowledged that the workings and scope of humans’ innate language-learning ability were not known.
Source: norvig.com
