magazine.sebastianraschka.com/p/understanding-positional-embeddings Understanding Positional Embeddings in Modern LLMs
A technical walkthrough explains how large language models encode token position, comparing learned absolute embeddings such as GPT-2's table with fixed sinusoidal schemes and the rotary method RoPE. It supplies visual examples, light math, a small PyTorch implementation, and a survey of positional choices in recent open-weight architectures.