Transformer forward FLOPs

A forward pass costs roughly two floating-point operations per parameter per token — one multiply and one add. Training adds the backward pass for about three times more, giving the familiar six-N-D rule.