The History of AI (Part 2): From Neural Networks to Deep Blue and AlphaGo
The evolution of Artificial Intelligence can be observed through two parallel pillars: the mathematical breakthroughs in artificial neural networks and the practical benchmarks achieved through strategic board games. This article covers the dramatic rise, fall, and resurrection of neural networks, and how gaming environments pushed computational limits to the absolute edge.
1. The Neural Network Evolution: Perceptron to Backpropagation
1) The Origin of Artificial Neural Networks (ANN)
An Artificial Neural Network is a computational framework mathematically modeled after the biological neural circuits of the human brain. The foundational concept was first introduced by neurophysiologist Warren McCulloch and logician Walter Pitts in their seminal 1943 paper. They engineered a simplified mathematical model of an artificial neuron, establishing the architectural prototype for all modern connectionist AI systems.
2) The Perceptron and Its Limitations (1957)
In 1957, Frank Rosenblatt of the Cornell Aeronautical Laboratory introduced the Perceptron, the earliest supervised learning network. Designed as a layer-based computing system, the single-layer perceptron took inputs, applied numerical weights, and ran them through an activation function to output a binary result (0 or 1).
Initially, the Perceptron generated immense excitement due to its proven mathematical ability to converge on correct weight values during repetitive training. However, its fundamental weakness was soon exposed: it could only solve linearly separable problems (such as simple AND/OR operations). It was mathematically incapable of resolving non-linear problems like the XOR (Exclusive OR) gate. This limitation triggered the first "AI Winter," during which funding and research into neural networks significantly stagnated.
3) The Breakthrough of Backpropagation (1986)
The resurrection of neural networks arrived in 1986, catalyzed by a landmark paper co-authored by David Rumelhart, Geoffrey Hinton, and Ronald Williams. They popularized the Backpropagation algorithm for Multi-Layer Perceptrons (MLPs).
Backpropagation is a mathematical method that calculates the error gradient at the output layer and distributes it backward through the hidden layers of the network. By systematically recalculating and readjusting weights to minimize error, it allowed computers to effectively learn non-linear patterns, overcoming the historic flaws of the early Perceptron.
Despite its brilliance, implementing backpropagation on a massive scale was bottlenecked by the hardware limitations of the 1980s and 1990s. Neural networks faced vanishing gradients and immense computational costs. The world would have to wait for the exponential hardware scaling and massive data repositories of the 21st century to fully unleash this algorithm via Deep Learning. Today, backpropagation remains the core engine that trains all frontier LLMs.
2. The Board Game Milestones: Testing Computational Intelligence
Since the 1950s, board games have served as the ultimate sandbox for testing AI capabilities. Because games operate within closed environments with finite, immutable rules, they offer an ideal benchmark for cognitive software engineering.
1) The Checker Program (1950s–1970s)
Arthur Samuel pioneered early gaming AI by building a checkers-playing program at IBM. To map strategies, Samuel introduced foundational AI concepts: Knowledge Representation (formatting facts and rules into structures a system can process) and Heuristic Search (algorithms that navigate data trees to find optimal pathways).
Crucially, Samuel implemented a system where the program played against copies of itself to discover superior strategies over time. This breakthrough proved that machines could learn without explicit manual coding, effectively birthing the modern discipline of Machine Learning.
2) Deep Blue vs. Garry Kasparov (1997)
While checkers is relatively straightforward, chess represents an exponentially higher degree of complexity due to its massive game-tree complexity. For decades, hardware limitations prevented machines from competing with grandmasters.
That boundary was shattered in May 1997 when IBM’s Deep Blue defeated the reigning world chess champion, Garry Kasparov, in a formal six-game match. Deep Blue was not a modern neural network; it was a massively parallel supercomputer equipped with custom, chess-specific search hardware. By combining brute-force computation capable of evaluating 200 million positions per second with expert chess heuristics, Deep Blue compensated for its lack of genuine human intuition, marking a historic milestone in computational power.
3) AlphaGo and the Deep Learning Revolution (2016)
The ultimate computational challenge was Go—a game with a search space larger than the number of atoms in the observable universe, rendering brute-force approaches completely useless. A breakthrough required deep pattern recognition resembling human intuition.
That milestone was achieved by Google DeepMind’s AlphaGo. In 2016, AlphaGo shocked the world by defeating the legendary 9th-dan grandmaster Lee Sedol (4-1) in Seoul. AlphaGo achieved this by fusing Deep Learning (using policy and value networks trained on millions of expert human moves) with advanced Monte Carlo Tree Search (MCTS) technologies.
AlphaGo rapidly evolved into AlphaGo Zero, which learned the game purely from scratch via self-play, and eventually Alpha Zero, which mastered chess, shogi, and Go using a single generalized algorithm. In the years following, DeepMind scaled this architecture beyond gaming to solve critical scientific challenges, culminating in AlphaFold—a system that predicts protein structures with pinpoint accuracy, fundamentally revolutionizing biology and medicine.
(Continued in Part 3: The Era of Big Data and Generative AI)