MiniLLM/pile-diff_samp-qwen_1.8B-qwen_104M-r0.5 download history

MiniLLM/pile-diff_samp-qwen_1.8B-qwen_104M-r0.5 is a text generation dataset on the Hugging Face Hub. In the last 30 days it was downloaded 713 times (138 in the last 7 days), and 7,511 times in total. It ranks #35,765 among datasets by monthly downloads.

This repository contains the refined pre-training corpus from the paper MiniPLM: Knowledge Distillation for Pre-Training Language Models. Code: https://github.com/thu-coai/MiniPLM

Models trained on pile-diff_samp-qwen_1.8B-qwen_104M-r0.5

14 models list it as training data.

Open MiniLLM/pile-diff_samp-qwen_1.8B-qwen_104M-r0.5 on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.