nikolina-p/fineweb_10BT_tokenized download history
nikolina-p/fineweb_10BT_tokenized is a text generation dataset on the Hugging Face Hub. In the last 30 days it was downloaded 463 times (40 in the last 7 days), and 866 times in total. It ranks #35,191 among datasets by monthly downloads.
Dataset card for FineWeb-Edu 10B tokenized dataset This dataset contains tokenized texts from FineWeb-Edu sample-10B HuggingFaceFW/fineweb-edu. The data was tokenized using the OpenAI's tiktoken tokenizer, and structured for efficient streaming and distributed (DDP) training.