research
Alethic-151M
Experimental pretraining and SinGatedAttention exploration on constrained compute.
alethictrainingmodels
Published work connected by this topic.
Experimental pretraining and SinGatedAttention exploration on constrained compute.
A supplied Python-data preparation record, separate from training progress.
The current context decision, separate from earlier long-context curricula.
Architecture targets, constrained compute, and infrastructure for an inspectable run.
What source-ID deduplication, supervised tokens, and packed windows say about preparation.