research
SinGatedLM
Parameter-matched small language-model experiments with a sinusoidal attention gate.
experimentslanguage-modelsgating
Questions, controlled experiments, and open artifacts. An evolving archive of work on language models and their underlying structures.