DAMO-NLP-SG/LongCorpus-2.5B download history

DAMO-NLP-SG/LongCorpus-2.5B is a text generation dataset on the Hugging Face Hub. In the last 30 days it was downloaded 117 times (19 in the last 7 days), and 2,747 times in total. It ranks #134,102 among datasets by monthly downloads.

We collect a 2.5B training dataset from various domains for long-context continual pre-training. The composition of this dataset is as follows (partially inspired by Long-Data-Collection): Domain Proportion Source Book 40% Redpajama-Book Arxiv 20% Redpajama-Arxiv General 20% Redpajama

Models trained on LongCorpus-2.5B

3 models list it as training data.

Open DAMO-NLP-SG/LongCorpus-2.5B on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.