seonglae/wikipedia-256 download history
seonglae/wikipedia-256 is a question answering dataset on the Hugging Face Hub. In the last 30 days it was downloaded 310 times (17 in the last 7 days), and 3,368 times in total. It ranks #47,862 among datasets by monthly downloads.
This is Wikidedia passages dataset for ODQA retriever. Each passages have 256~ tokens splitteed by gpt-4 tokenizer using tiktoken. Token count {'~128': 1415068, '128~256': 1290011, '256~512': 18756476, '512~1024': 667, '1024~2048': 12, '2048~4096': 0, '4096~8192': 0, '8192~16384': 0, '16384~32768':