Pre-LN Transformer Block

Combine LayerNorm, causal multi-head attention, a GELU MLP, and two residual paths. A free ML coding interview task on NudgeCode: write it in Python and check it with tests in your browser. Hard · Transformer, Pre-LN, GELU.

Implement a decoder-only Pre-LN block without dropout: residual self-attention over LayerNorm(x), followed by a residual GELU MLP over a second LayerNorm. Use one fused QKV projection; weights follow nn.Linear's (out,in) layout.