HuggingFaceFW/dclm_100BT
Viewer • Updated • 89.3M • 2.35k • 3
🤏 Smol-Data
Tried and tested mixes for strong pretraining. Inspired by https://huggingface.co/blog/codelion/optimal-dataset-mixing
-
-
HuggingFaceFW/dclm_100BT-shuffled
Viewer • Updated • 89.3M • 1.17k • 5 -
HuggingFaceFW/finepdfs_100BT
Viewer • Updated • 29.9M • 1.37k • 2 -
HuggingFaceFW/finepdfs_100BT-shuffled
Viewer • Updated • 14.6M • 667 -
HuggingFaceFW/finepdfs_edu_100BT
Viewer • Updated • 17.8M • 1.63k • 2 -
HuggingFaceFW/finepdfs_edu_100BT-shuffled
Viewer • Updated • 17.8M • 885 -
HuggingFaceFW/fineweb_100BT
Viewer • Updated • 161M • 715 • 5 -
HuggingFaceFW/fineweb_100BT-shuffled
Viewer • Updated • 161M • 840 • 2 -
HuggingFaceFW/fineweb_edu_100BT
Preview • Updated • 3.93k • 5 -
HuggingFaceFW/fineweb_edu_100BT-shuffled
Viewer • Updated • 102M • 1.54k • 7 -
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT
Viewer • Updated • 56.1M • 12.1k -
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer • Updated • 56.1M • 1.56k • 3 -
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT
Viewer • Updated • 62.1M • 1.95k • 4 -
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer • Updated • 62.1M • 675 • 7