Sequence-to-Sequence

The sequence to sequence (Seq2Seq) paper by Ilya Sutskever, Oriol Vinyals, and Quoc V. Le set a new standard for sequence to sequence translation by introducing a novel approach that successfully maps a sentence to a fixed-length vector representation and back to a...

Word2Vec

Word2Vec is a shallow, two-layer neural network that was designed to reconstruct the linguistic contexts of words. Introduced by Tomas Mikolov and his team at Google in 2013, Word2Vec was groundbreaking as it was the first word embedding model capable of preserving...

ImageNet

Alex Krizhevsky et al. significantly improved previous results on the ImageNet challenge by parallelizing the training of their neural network on multiple GPUs. Their approach utilized a deep convolutional neural network (CNN) architecture, later known as AlexNet,...

Backpropagation applied to Deep Learning

This paper delves into the effectiveness of using large, deep neural networks for the task of handwritten digit recognition. It builds on the MNIST dataset, a benchmark in the field of image recognition, showcasing significant improvements in accuracy through the use...

Fast Learning Algorithm for Deep Belief Nets

This paper introduces a fast, greedy learning algorithm for training Deep Belief Networks (DBNs). The algorithm involves layer-by-layer training of a stack of Restricted Boltzmann Machines (RBMs), where each layer learns to represent features of the data based on the...