BlackwoodAI/pretrain-shards download history

BlackwoodAI/pretrain-shards is a dataset on the Hugging Face Hub. In the last 30 days it was downloaded 165 times (163 in the last 7 days), and 16,016 times in total. It ranks #78,349 among datasets by monthly downloads.

Pretrain shards Pre-tokenized shards for pretraining. License: CC BY-NC 4.0 — non-commercial only. Format *.bin: uint32 little-endian token IDs *.json: per-shard sidecar with doc_lengths[] and stats Layout code/ shard_NNNNNN.bin + .json text/ shard_NNNNNN.bin + .

Open BlackwoodAI/pretrain-shards on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.