magazine.sebastianraschka.com/p/understanding-positional-embeddings Understanding Positional Embeddings in Modern LLMs
A technical walkthrough explains how large language models inject token-position information, contrasting GPT-2's learned absolute position table with rotary schemes such as RoPE that act inside attention. It adds a small PyTorch implementation and previews which positional methods appear in 2026 open-weight architectures.