b.
Color theme
Let’s talk
Explore
Appearance
Color theme
Independent researcher & engineer

BekhruzSuleyman.

Exploring how intelligence works.
Building the systems to understand it.

Follow the story
UZBEKISTAN / INDEPENDENT
01 / Research

My research explores language modelsNeural networks trained to predict the next token of text. and the structures behind them. From controlled experiments to open model artifactsReleased weights and code that anyone can inspect and rerun., this archive brings together the questions, methods, and limitations.

Step 3: the sine term

SGCA passes the attention output A through sin(A), then scales it by a learned amplitude α. For small A, sin(A) ≈ A, so it behaves like a plain scaled copy. As A grows, the sine folds back.

Worked numbers, with α = 1: A = 0.5 gives sin(0.5) ≈ 0.48, A = 1.5 gives 1.00, and A = 3.0 gives 0.14.

Move over the graph to probe a token position. This is an illustration of the mechanism, not a measured result.
Alethic news / 10 Nov 2026The gate, not the narrower FFN, explains the gapGQA-narrow (the baseline with SGCA’s FFN width and no gate, 277.7M parameters) finished at 0.8110 BPB, only 0.0042 behind full GQA. About 90% of SGCA’s 0.0404 gap belongs to the gate.
Read the results
Selected investigationsAll research

Ideas, with their evidence attached.Historical experiments and model artifacts are presented with source links. Planned work, hypotheses, and measured results stay distinct.

02 / Engineering

All projects

Follow one line through a compiler

This is the shape of a language project such as Flang-Lite: text becomes tokens, tokens become a tree, the tree becomes instructions, and a machine runs them.

The line x = 2 + 3 * 4 evaluates to 2 + 12 = 14, because the tree puts the multiplication below the addition.

The program is plain text: x = 2 + 3 * 4.
03 / Notebook

Writing & notes
04 / Beyond the work

A little about me

I’m Bekhruz Suleyman, an independent researcher and engineer from Uzbekistan.

Have a question, an interesting problem, or a project to discuss?