hf-internal-testing/tokenizers-test-data download history

hf-internal-testing/tokenizers-test-data is a dataset on the Hugging Face Hub. In the last 30 days it was downloaded 40,400 times (9,726 in the last 7 days), and 67,583 times in total. It ranks #647 among datasets by monthly downloads.

tokenizers-test-data Test and benchmark fixtures for huggingface/tokenizers, pulled on demand by the repo Makefiles (make test / make bench / make fixtures via hf download). Layout fixtures/ — multilingual + modality corpora for cross-language encode benchmarks. Organi

Spaces using tokenizers-test-data

Open hf-internal-testing/tokenizers-test-data on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.