Tracing Contextual Token Representations Through Transformer Layers

The same token can begin with the same embedding and develop different representations. Calling a word “a vector” hides this transformation: inside a Transformer, the initial embedding is only the starting point of a computation shaped by…