History

From RNNs and LSTMs to the Transformer: The Evolution of Sequence Architectures

An intuitive guide to the evolution of sequence architectures: how RNNs work, their limits like vanishing gradients and the inability to parallelize, the gates of the LSTM, and why attention won out with the Transformer.