alexkstern/c4-gpt2bpe-10B download history

alexkstern/c4-gpt2bpe-10B is a dataset on the Hugging Face Hub. In the last 30 days it was downloaded 20 times (15 in the last 7 days), and 279 times in total. It ranks #374,093 among datasets by monthly downloads.

c4-gpt2bpe-10B C4 (en) (from allenai/c4), pre-tokenized with the gpt2bpe tokenizer (vocab 50,257) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin train 10,000,000,000 val.bin val 174,033,902 train and val are disjoint

Open alexkstern/c4-gpt2bpe-10B on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.