sam-paech/mmlu-pro-irt-1-0 download history

sam-paech/mmlu-pro-irt-1-0 is a dataset on the Hugging Face Hub. In the last 30 days it was downloaded 109 times (31 in the last 7 days), and 2,558 times in total. It ranks #141,440 among datasets by monthly downloads.

MMLU-Pro-IRT This is a small subset of MMLU-Pro, selected with Item Response Theory for better separation of scores across the ability range. It contains 2059 items (compared to 12000 in the full MMLU-Pro), so it's faster to run. It takes ~6 mins to evaluate gemma-2-9b on a RTX-4090 u

Open sam-paech/mmlu-pro-irt-1-0 on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.