Module 1: Generative AI foundations
Transformers and attention
Transformers process relationships among tokens using attention. Attention helps the model weigh relevant words and concepts, while multiple layers build increasingly useful representations.
Practice exercise
Explain attention using a sentence containing an ambiguous pronoun.