PY007/slimpajama_llama_tokenized_upsample_4096_chunk_1M download history

PY007/slimpajama_llama_tokenized_upsample_4096_chunk_1M is a dataset on the Hugging Face Hub. In the last 30 days it was downloaded 99 times (14 in the last 7 days), and 7,494 times in total. It ranks #114,681 among datasets by monthly downloads.

Generated using https://github.com/FranxYao/Long-Context-Data-Engineering with the below command: mkdir logs mkdir data mkdir data/slimpajama mkdir data/slimpajama/per_source_downsample cd data_engineering PATH_TO_SLIMPAJAMA=rokset3/slim_pajama_chunk_1 nohup python -u slimpajama_packing.py\ --d

Open PY007/slimpajama_llama_tokenized_upsample_4096_chunk_1M on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.