research
SinGatedLM
Parameter-matched small language-model experiments with a sinusoidal attention gate.
experimentslanguage-modelsgating
Published work connected by this topic.
Parameter-matched small language-model experiments with a sinusoidal attention gate.
A careful reading of parameter-matched Tiny Shakespeare results and their limits.