Transformer Architecture · Chapter 08

Feed-Forward Network

max(0, xW₁+b₁)W₂+b₂ — the per-position nonlinearity

This chapter is part of the Pro track. Pro beta members receive the complete self-paced architecture sequence.