alexkstern/fineweb-gpt2bpe-20B download history

alexkstern/fineweb-gpt2bpe-20B is a dataset on the Hugging Face Hub. In the last 30 days it was downloaded 26 times (5 in the last 7 days), and 232 times in total. It ranks #306,850 among datasets by monthly downloads.

fineweb-gpt2bpe-20B FineWeb-Edu (from karpathy/fineweb-edu-100b-shuffle), pre-tokenized with the gpt2bpe tokenizer (vocab 50,257) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin train 20,000,000,000 val.bin val 54,798,984

Open alexkstern/fineweb-gpt2bpe-20B on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.