Get through the pile.
Paste several links at once. Yank finds the five things worth knowing in each, with their sources, then read them in turn, or play them through: videos and podcasts in their own voices, everything else read aloud by an AI voice.
1 source · 5 lines · about 1 min to play
1 sources · 1 min in all
Videos and podcasts play in their own players, from two seconds before each line. Articles and documents are read aloud by an AI voice from ElevenLabs, reading only the words already on each yank's page, or by your device's own voice if you choose it. It keeps playing if you go to another page.
Attention Is All You Need: The Transformer Paper's Five Consequential Claims
The 2017 Google Brain paper that scrapped recurrence and convolutions entirely, proposing the Transformer architecture that now underlies nearly every large language model.
- We propose the Transformer, the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.
- The Transformer (big) achieves 28.4 BLEU on WMT 2014 English-to-German, outperforming all previously reported models including ensembles by more than 2 BLEU, trained in 3.5 days on 8 P100 GPUs.
- A self-attention layer connects all positions with a constant number of sequentially executed operations, whereas a recurrent layer requires O(n) sequential operations.
- “Multi-head attention allows the model to jointly attend to information from different representation subspaces at different positions. With a single attention head, averaging inhibits this.”
- Label smoothing hurts perplexity, as the model learns to be more unsure, but improves accuracy and BLEU score.