Softmax

A function that turns a set of raw scores into a probability distribution that sums to one. Transformers use it inside attention to normalise the filter, and at the final layer to turn vocabulary scores into the probability of each possible next word.

Terms like this come up in real clinical scenarios across the HelloAI courses: bite-sized modules with verifiable certificates. An account takes one minute, no password needed.