First Modern Backpropagation

In his 1976 paper “Taylor Expansion of the Accumulated Rounding Error,” S. Linnainmaa presented a novel approach for addressing rounding errors in numerical calculations through the use of Taylor series expansion. By systematically expanding the rounding errors, Linnainmaa provided a method to analyze their propagation and accumulation in computational processes. This work is particularly significant as it forms the basis for the development of automatic differentiation (AD) methods, which are crucial in various fields such as optimization, machine learning, and scientific computing. The ability to accurately compute derivatives automatically has had a profound impact on the efficiency and precision of numerical algorithms, making Linnainmaa's contributions a cornerstone in the domain of numerical analysis.