Word2Vec

Word2Vec is a shallow, two-layer neural network that was designed to reconstruct the linguistic contexts of words. Introduced by Tomas Mikolov and his team at Google in 2013, Word2Vec was groundbreaking as it was the first word embedding model capable of preserving...

ImageNet

Alex Krizhevsky et al. significantly improved previous results on the ImageNet challenge by parallelizing the training of their neural network on multiple GPUs. Their approach utilized a deep convolutional neural network (CNN) architecture, later known as AlexNet,...

Backpropagation applied to Deep Learning

This paper delves into the effectiveness of using large, deep neural networks for the task of handwritten digit recognition. It builds on the MNIST dataset, a benchmark in the field of image recognition, showcasing significant improvements in accuracy through the use...

Fast Learning Algorithm for Deep Belief Nets

This paper introduces a fast, greedy learning algorithm for training Deep Belief Networks (DBNs). The algorithm involves layer-by-layer training of a stack of Restricted Boltzmann Machines (RBMs), where each layer learns to represent features of the data based on the...

First Deep Learning (8 layer networks)

A.G. Ivakhnenko's 1971 paper, “Polynomial Theory of Complex Systems,” published in IEEE Transactions on Systems, presents a novel approach to modeling complex systems using polynomial functions. Ivakhnenko details the development and application of...

The Turing Machine

The Turing machine, introduced by Alan Turing in 1936, is a fundamental theoretical model in computer science and mathematics. Designed to formalize the notion of computation, the Turing machine explores the limits of what can be computed. Despite its simplicity, it...