procmarco/ita-en-code-tokens-48k download history
procmarco/ita-en-code-tokens-48k is a text generation dataset on the Hugging Face Hub. In the last 30 days it was downloaded 401 times (119 in the last 7 days), and 1,586 times in total. It ranks #39,320 among datasets by monthly downloads.
ita-en-code-tokens-48k Pretokenized 40% Italian / 40% English / 20% code pretraining tokens (~40B), tokenized with procmarco/ita-en-code-bpe-48k (vocab 49152). Documents packed <|bos|> … <|eos|>. Sources: FineWeb-2 ita_Latn (filtered), FineWeb-Edu sample/100BT (int_score≥3), codeparro