Transformer Architecture · Chapter 11

Training

Cross-entropy, label smoothing, warmup schedule, dropout

This chapter is part of the Pro track. Pro beta members receive the complete self-paced architecture sequence.

Build the context for this chapter

Pre-training

Fine-Tuning

Scaling Laws