Nirmal Utwani
Week 04 of my learning series on Turing Award winners: Marvin Minsky.
Marvin Minsky, along with John McCarthy, was hugely pivotal in starting AI as a field. They organized the Dartmouth conference in 1956, got the best minds in the room, named the discipline of AI, and then went off and built the two institutions that shaped everything after: Minsky at MIT, McCarthy at Stanford.
The MIT AI Lab is still running today as CSAIL.
---
The first highlight is moving from flat lists of rules to structured knowledge. Before his frames paper, AI systems stored knowledge as giant if-then rule lists, which doesn’t scale well. Minsky’s intuition was that this is very different than how humans think. When you see a chair, your brain doesn’t rederive what a chair is from scratch every time. It has a mental template made up of four legs, a back, a seat, some material, and you fill in the details. He called that idea of structure “frames”: structured objects with named slots and defaults. This is how most programming languages work today in terms of classes and objects.
The second, and the more important one, is his book on perceptrons and their limitations. Rosenblatt’s single-layer network was the hot AI thing in 1958, and there was a lot of optimism that you could scale this infinitely and solve intelligence. Minsky was the party pooper and proved mathematically that regardless of data size or compute, a single layer isn’t enough for a computer to understand even a basic operation like XOR. This did play a big role in the first AI winter: funding dried up, research stalled, the whole neural network agenda lost momentum for nearly a decade. Minsky, to his credit, also explicitly said that multi-layer networks wouldn’t have this problem and pushed researchers towards training them. Backpropagation for multi-layer networks arrived in 1986, which was a big unlock, followed by deep learning. The AI era we know today: H100 clusters, hyperscalers, Mag-7, all training the next frontier models. All of it traces back to Minsky pushing the world towards multi-layer networks.
---
Marvin Minsky won the Turing Award in 1969 “for his central role in creating the science of artificial intelligence.”
Two contributions stand out:
Before Minsky’s frames paper, AI systems stored knowledge as giant if-then rule lists. If you wanted a computer to understand what a chair is, you’d write hundreds of rules: “if object has four legs AND has a back AND has a seat, then object is chair.”
It doesn’t scale. The rules explode combinatorially.
Minsky’s insight: humans don’t work that way. When you see a chair, your brain doesn’t rederive what a chair is from scratch. It has a mental template. Four legs. A back. A seat. Some material. Default values. You fill in the details.
He called this structure “frames”: objects with named slots and defaults.
Frame: CHAIR
slots:
legs: 4 (default)
back: yes (default)
seat: yes (required)
material: wood (default)This became the foundation of object-oriented programming. Classes with properties. Inheritance. Defaults. It’s how Python, Java, JavaScript work today.
In 1958, Frank Rosenblatt built a learning machine called the perceptron. You showed it examples — this input means yes, that input means no — and it adjusted its internal numbers (weights) until it got them right.
It had a mathematical guarantee: if a pattern can be separated by a straight line, the perceptron will find the weights.
The problem was the fine print: “if a pattern can be separated by a straight line.”
The XOR Problem
XOR is a simple logic function:
0 XOR 0 = 0
0 XOR 1 = 1
1 XOR 0 = 1
1 XOR 1 = 0Plot these four points on a grid. The 1s sit at diagonal corners `(0,1)` and `(1,0)`. The 0s sit at the other diagonal corners `(0,0)` and `(1,1)`.
No straight line can separate them. Try it. Every line you draw leaves at least one point on the wrong side.
A perceptron computes the sign of a linear function: `w1·x1 + w2·x2 + b`. The decision boundary is a straight line. Since XOR isn’t linearly separable, no perceptron can learn it — ever, no matter how many training rounds you run.
Minsky’s Proof
Minsky and Papert spent years in the 1960s mapping out exactly which patterns fail. Their 1969 book proved it rigorously.
For XOR, the constraints are:
(0,1) → 1: w2 + b ≥ 0 → w2 ≥ -b
(1,0) → 1: w1 + b ≥ 0 → w1 ≥ -b
(0,0) → 0: b < 0 → b < 0
(1,1) → 0: w1 + w2 + b < 0From `b < 0`, say `b = -1`. Then `w1 ≥ 1` and `w2 ≥ 1`. But then `w1 + w2 + b ≥ 1 + 1 - 1 = 1 > 0`, so `(1,1)` fires — contradicting the fourth constraint.
No values of `w1`, `w2`, `b` satisfy all four constraints simultaneously. The system is inconsistent.
This is not a computational problem. It is a proof. Single-layer networks cannot solve XOR, period.
The First AI Winter
The book effectively ended the first era of neural network optimism. Funding dried up. Research stalled. The whole neural network agenda lost momentum for nearly a decade. People remember this as “Minsky killed neural networks.” But that’s not the full story.
What Minsky Actually Said
The book acknowledged explicitly that adding hidden layers broke the representational limits. Multi-layer networks could solve XOR. The problem was that no training algorithm existed for multi-layer networks in 1969.
The perceptron learning rule applies only to the output layer. Updating hidden layer weights requires knowing how each hidden neuron contributes to the output error, which requires propagating the error signal backward through the network.
That algorithm — backpropagation — arrived in 1986.
The Two-Layer Solution
With two layers, XOR is solvable. The hidden layer learns two intermediate features (OR and NAND) and reshapes the problem into one the output layer can handle with a single line.
Here’s the geometry:
Hidden neuron h1 (computes OR):
(0,0) → h1=0
(0,1) → h1=1
(1,0) → h1=1
(1,1) → h1=1
Hidden neuron h2 (computes NAND):
(0,0) → h2=1
(0,1) → h2=1
(1,0) → h2=1
(1,1) → h2=0
New feature space (h1, h2):
(0,0) → (0, 1)
(0,1) → (1, 1)
(1,0) → (1, 1)
(1,1) → (1, 0)In this new space, XOR is linearly separable. The output neuron computes AND of h1 and h2. Done.
The hidden layer didn’t learn XOR directly. It learned two simpler features and created a transformed space where the XOR points ARE linearly separable.
Minsky’s proof said: you cannot do this with one layer. He was right. Two layers is the minimum.
---
Backpropagation (1986): Rumelhart, Hinton, and Williams popularized the algorithm for training multi-layer networks. Gradient descent through a network of sigmoid units. Automatic differentiation. The training method Minsky said was missing.
Deep Learning (2006+): Stacking many hidden layers. GPUs, large datasets, architectural improvements (ReLU, dropout, batch normalization). GPT, DALL-E, AlphaFold, every major neural network today.
Universal Approximation Theorem (1989): Proved that any continuous function can be approximated arbitrarily closely by a single hidden layer of sufficient width. The Minsky-Papert limitation applies only to single-layer networks.
All of it traces back to Minsky proving what single-layer networks can’t do. The proof defined the problem space. Backpropagation solved it.
---
I built a perceptron simulator that shows the XOR failure:
Perceptron on AND (converged in 6 epochs):
0 0 → 0 (expected 0) ok
1 1 → 1 (expected 1) ok
LEARNEDPerceptron on XOR (1000 epochs, never converges):
0 0 → 1 (expected 0) WRONG
0 1 → 1 (expected 1) ok
1 0 → 0 (expected 1) WRONG
1 1 → 0 (expected 0) ok
FAILED (accuracy 50% — best any single-layer net can do)
Two-layer net on XOR (10000 epochs, backprop):
0 0 → 0 (expected 0) ok
0 1 → 1 (expected 1) ok
1 0 → 1 (expected 1) ok
1 1 → 0 (expected 0) ok
LEARNED
With `--verbose`, you see the weights update, the hidden layer learn OR and NAND, and the output layer combine them.
Full code and worked example on GitHub: github.com/nirmal91/turing-award-series
---
Minsky didn’t kill neural networks. He defined their limits precisely, mathematically, with a proof.
That precision is what made progress possible.
Before the Perceptrons book, researchers made expansive claims about what single-layer networks could eventually learn. There was no formal framework for asking: what can’t a perceptron learn, and why?
After the book, the question was answered. Single-layer networks have a geometric limitation. Add a hidden layer. Learn intermediate features. Transform the space.
When backpropagation arrived in 1986, it gave researchers exactly what the Minsky-Papert proof said was missing. Every modern neural network is, in a sense, the answer to their 1969 theorem.
The H100 clusters training frontier models today. The hyperscalers. The Mag-7. All running multi-layer networks that solve problems Minsky proved single-layer networks cannot.
He was the party pooper who proved the old approach wouldn’t scale. And in doing so, he pointed to exactly what would.
---
This is Week 04 of the Turing Award Series. New entry every week. All code is on GitHub.
← Previous: Week 03 — When Bits Flip and Everything Breaks (Richard Hamming)