First version of dropout

The 1990 paper “A Stochastic Version of the Delta Rule” by Stephen J. Hanson introduces a modified version of the delta rule incorporating synaptic noise into the weight update process of neural networks. This stochastic approach, termed the Stochastic...

Conv Nets by Yann LeCun

The researchers demonstrated the application of backpropagation in neural networks for recognizing handwritten digits, specifically focusing on zip code digits provided by the U.S. Postal Service. The network architecture was specifically designed and constrained for...

Learning from delayed rewards

C. Watkins' 1989 Ph.D. thesis, “Learning from Delayed Rewards,” completed at the University of Cambridge, introduces a fundamental advancement in the field of reinforcement learning through the development of the Q-learning algorithm. This work...

New training regime of Adaline algorithms

The 1988 paper “MADALINE RULE II: A Training Algorithm for Neural Networks,” authored by B. Widrow et al. and presented at the International Conference on Neural Networks (ICNN), introduces an advanced training algorithm known as MADALINE RULE II. This...