Tokenizer Arena: likes over time
xu-song/tokenizer-arena is a gradio Space on the Hugging Face Hub. It has 61 likes, 0 in the last 7 days and 0 in the last 30, and ranks #2,494 among Spaces by likes.
Compare different tokenizers in char-level and byte-level.
What it uses
- cc100 (dataset)