Many of today’s highest-performing artificial intelligence (AI) systems, like speech recognizers and automatic translation, are produced using deep learning. Inspired by the human brain, this technique is based on artificial neural networks, which have been around for more than 70 years.
Neural networks are a technique for machine learning where a computer learns to perform a task by analyzing training examples. Modeled loosely on the human brain, a neural network is made up of thousands, or even millions, of densely interconnected processing nodes. When we talk about deep learning, we’re referring to these neural networks.
“The neural network is this kind of technology that is not an algorithm, it is a network that has weights on it, and you can adjust the weights so that it learns. You teach it through trials.”
Howard Rheingold
Neural networks were first proposed in 1944 by Warren McCullough and Walter Pitts, two researchers at the University of Chicago. They quickly became a major area of research in both neuroscience and computer science. The first trainable neural network, the perceptron, was demonstrated by Cornell University psychologist Frank Rosenblatt in 1957. However, MIT mathematicians Marvin Minsky and Seymour Papert poured cold water on the innovation by highlighting how time consuming it was.
Revival and Innovation
By the 1980s, however, researchers had developed algorithms efficient enough for networks with more than one layer. The recent resurgence, or deep learning revolution, was ignited by the computer game industry. The complex imagery and rapid pace of modern video games gave rise to graphics processing units (GPUs), packing thousands of processing cores on a single chip. Researchers quickly realized that the architecture of a GPU was remarkably like a neural network. The one-layer networks of the 1960s quickly deepened with 10, 15, or even 50 layers.
Deep learning is currently responsible for high-performing systems in almost all kinds of AI applications, from financial forecasting to fraud detection.
Key Dates
-
1944
The First Artificial Neural Network
Warren McCullough and Walter Pitts create a computational model for neural networks, based on algorithms called threshold logic. Many consider this the first artificial neural network.
-
1957
The Perceptron
Frank Rosenblatt invents The Perceptron, an algorithm for the supervised learning of binary classifiers.
-
1992
Tuning Neuron Weights
Jürgen Schmidhuber overcome difficulties in tuning neuron weights. He adopts a multi-level hierarchy of networks: He pre-trains one level at a time by unsupervised learning and fine-tunes by backpropagation.
