JonathanMiddleton/fineweb-edu-dedup-shuffled-pretokenized download history

JonathanMiddleton/fineweb-edu-dedup-shuffled-pretokenized is a text generation dataset on the Hugging Face Hub. In the last 30 days it was downloaded 60 times (10 in the last 7 days), and 8,235 times in total. It ranks #162,992 among datasets by monthly downloads.

FineWeb-Edu-Dedup Shuffled Pretokenized Pretokenized training shards built from a globally shuffled version of FineWeb-Edu-Dedup. Ready for direct consumption by the Daisy pretraining loop. Summary Property Value Total tokens 181,465,257,766 (~181.5B) Train tokens

Open JonathanMiddleton/fineweb-edu-dedup-shuffled-pretokenized on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.