Planning a 1B language-model experiment independently
Architecture targets, constrained compute, and infrastructure for an inspectable run.
A supplied Python-data preparation record, separate from training progress.
Supplied on 1 October 2026; a point-in-time record, not a live dashboard.
Counter | Recorded value |
|---|---|
Segment | stack_v3_python |
JSONL records inspected | 125,215 |
Train windows packed | 20,638 / 32,767 |
Supervised tokens | 170,057,120 |
Validation windows | 512 |
Identical source-ID duplicates skipped | 62,375 |
Window packing is about 63% of this stated target. The token count does not mean these tokens were consumed by an optimizer. Source-ID deduplication is not semantic deduplication.
The snapshot does not establish final corpus size or exact binary bytes. Full configuration is needed to interpret separators, masks, padding, and token storage.