Attention Should Be Allowed to Say No

Qwen's Gated Attention separates where an attention head reads from whether its output should influence the model.

Read essay