Alethic
Language-model architecture research, from controlled small-model experiments to a planned ~1B Phase I.
Exploring how intelligence works.
Building the systems to understand it.
My research explores language modelsNeural networks trained to predict the next token of text. and the structures behind them. From controlled experiments to open model artifactsReleased weights and code that anyone can inspect and rerun., this archive brings together the questions, methods, and limitations.
SGCA passes the attention output A through sin(A), then scales it by a learned amplitude α. For small A, sin(A) ≈ A, so it behaves like a plain scaled copy. As A grows, the sine folds back.
Worked numbers, with α = 1: A = 0.5 gives sin(0.5) ≈ 0.48, A = 1.5 gives 1.00, and A = 3.0 gives 0.14.
Language-model architecture research, from controlled small-model experiments to a planned ~1B Phase I.
Sin-Gated Control Attention: separating attention retrieval from learned residual control.
Parameter-matched small language-model experiments with a sinusoidal attention gate.
Ideas, with their evidence attached.Historical experiments and model artifacts are presented with source links. Planned work, hypotheses, and measured results stay distinct.
This is the shape of a language project such as Flang-Lite: text becomes tokens, tokens become a tree, the tree becomes instructions, and a machine runs them.
The line x = 2 + 3 * 4 evaluates to 2 + 12 = 14, because the tree puts the multiplication below the addition.
An AI-first operating system on the Linux kernel: a Rust desktop shell and an agent daemon behind a permission broker and audit log, built with AI tools.
A language written in C11: typed compiler, optimizer, bytecode VM, tensors with autodiff, and a package manager. 10 versions and a full test suite.
Commerce search and aggregation for Uzbek and regional product discovery.
The SGCA research question, residual update, and requirements for a fair comparison.
A careful reading of parameter-matched Tiny Shakespeare results and their limits.
Architecture targets, constrained compute, and infrastructure for an inspectable run.
A supplied Python-data preparation record, separate from training progress.
The current context decision, separate from earlier long-context curricula.
I’m Bekhruz Suleyman, an independent researcher and engineer from Uzbekistan.
Have a question, an interesting problem, or a project to discuss?