Instructions to use Compactbot/tinystories-40m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Compactbot/tinystories-40m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Compactbot/tinystories-40m", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Compactbot/tinystories-40m", trust_remote_code=True)
model = AutoModel.from_pretrained("Compactbot/tinystories-40m", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Compactbot/tinystories-40m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Compactbot/tinystories-40m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Compactbot/tinystories-40m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Compactbot/tinystories-40m

SGLang

How to use Compactbot/tinystories-40m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Compactbot/tinystories-40m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Compactbot/tinystories-40m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Compactbot/tinystories-40m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Compactbot/tinystories-40m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Compactbot/tinystories-40m with Docker Model Runner:
```
docker model run hf.co/Compactbot/tinystories-40m
```

TinyStories-40M

A 39.6M-parameter LLaMA-style transformer trained from scratch on the full TinyStories corpus. Published as a from-scratch training demonstration at the ~40M scale — it is not a coherent story generator (see Quality below).

Architecture

Parameter	Value
Layers	12
Hidden size	512
Attention heads (Q)	8
Attention heads (KV)	4 (GQA 2:1)
Head dim	64
FFN (SwiGLU)	1408
Vocab size	8192 (BPE)
Context length	512
RoPE θ	10000
Norm	RMSNorm (pre-norm)
Tied embeddings	Yes
Precision	FP32

Total parameters: 39,596,544 (86 tensors, head weight tied to the token embedding). Verified against the published model.safetensors.

Training

Data: roneneldan/TinyStories (~1.9B chars, ~490M tokens at seq 512)
Steps: 15,000
Batch size: 64 sequences × 512 tokens (32,704 tok/step)
Optimizer: AdamW, lr 3e-4, cosine decay (min-lr-frac 0.1), 300-step warmup
Hardware: RTX 5090 (32 GB), ~4 hours
Final train loss: 3.23 (step 15000)

Quality

Honest picture, from running the published weights (temp 0.7, top-k 50):

On the canonical prompt "Once upon a time" the model produces a story-like first sentence, then degrades:

"Once upon a time, his mom and his clapped and cheered for him. They all enjoyed spending the rest of their special day in the park, his mom's light and a Stop being himself."
On other prompts it is weaker still — often a single short phrase or an immediate stop:

"In the forest" → "In the forest things." "She opened the door" → "She opened the door."
Longer generations drift into incoherent, non-grammatical text.

This is what a 40M from-scratch model on a single 490M-token corpus demonstrably does: it learns the surface distribution of story text (word order, names, punctuation, the "Once upon a time" register) but does not sustain coherent narrative. It is published as a training demonstration, not as a usable story generator.

Usage

This is a custom transformers model — load with trust_remote_code=True:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Compactbot/tinystories-40m", trust_remote_code=True, torch_dtype=torch.float32
)
tok = AutoTokenizer.from_pretrained("Compactbot/tinystories-40m")

prompt = "Once upon a time"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=100, temperature=0.7, top_k=50)
print(tok.decode(out[0], skip_special_tokens=True))

Note: the bundled generate() always samples (no do_sample flag); pass temperature and top_k to control it.

Notes

From-scratch training: initialized randomly, trained end-to-end on TinyStories. No pre-training from any other model.
GQA (Grouped Query Attention) 2:1, SwiGLU, RoPE — LLaMA-2 recipe at 40M scale.
~151 MB in FP32 — under 200 MB.

Files

file	what
`model.safetensors`	39,596,544 params, 86 tensors, FP32
`config.json`	architecture + training metadata
`modeling_tinystories.py`	the model class (loaded via `trust_remote_code`)
`tokenizer.json`	BPE-8k tokenizer (HF `tokenizers` format)

Downloads last month: -

Safetensors

Model size

39.6M params

Tensor type

F32

Compactbot
/

tinystories-40m