Forgetting Transformer: Softmax Attention with a Forget Gate
Zhixuan Lin et al. (4 authors)
2025ICLR
A data-dependent forget gate down-weights unnormalized attention scores, bringing recurrent-style forgetting to softmax attention.
One paper is all you need.
Ask anything about your field. Minlit reads the literature and comes back with the papers that answer it.
Papers live in a thousand places.
Start from one paper.See what’s new around it.
Every paper has a past and a future.
See where an idea came from, and everything it led to, on one timeline.
What it built onSeed · Attention Is All You Need · 2017What it led to
Stop checking a thousand sites.Minlit already did.
Minlit learns from what you save, read and highlight, and builds you a feed of new papers. Fresh every morning, all in one place.
Any paper, in a minute.
Ask for a TL;DR to get the core of a paper fast. Then chat with it about the rest.
Your assistant finds it.Minlit keeps it.
Papers your assistant finds get buried in old chats. Connect Minlit to ChatGPT, Claude or Cursor, and the ones you save land in your library, one search away.
Oh, and one more thing.