research
Alethic
Language-model architecture research, from controlled small-model experiments to a planned ~1B Phase I.
alethiclanguage-modelsresearch
Questions, controlled experiments, and open artifacts. An evolving archive of work on language models and their underlying structures.
Language-model architecture research, from controlled small-model experiments to a planned ~1B Phase I.
Sin-Gated Control Attention: separating attention retrieval from learned residual control.
Parameter-matched small language-model experiments with a sinusoidal attention gate.
Experimental pretraining and SinGatedAttention exploration on constrained compute.