SupraLabs
Research

We're democratizing AI by training high-performance models on consumer hardware. 100% Open Source. 0% Slop.

// Research papers

Dynamic Curriculum Shifting

Information Routing — Curriculum Shifting

More Epochs vs. More Data for SLMs

Satiating the Latent Space: Unique Tokens vs. Cycles

The optimal vocab size

The Embedding Bottleneck: Optimal Vocab Scales

Depth vs. Width

Hidden Topology: Depth vs. Width Scaling for SLMs

Is One Epoch Really All You Need for SLMs?

Researching the best epoch count for SLMs

5M SLM Data-Mix Benchmarks

Researching the best dataset(s) for SLMs