uv-scripts/deduplication download history

uv-scripts/deduplication is a dataset on the Hugging Face Hub. In the last 30 days it was downloaded 27 times (8 in the last 7 days), and 275 times in total. It ranks #298,367 among datasets by monthly downloads.

Semantic Deduplication UV Script Part of uv-scripts — self-contained UV scripts you run locally or on Hugging Face Jobs in one command. Remove duplicate / near-duplicate text samples from a Hugging Face dataset by semantic similarity — clean training data and prevent train/test leak

Open uv-scripts/deduplication on Hugging Face

Sister project: Paper Pulse, the upvote history of every Hugging Face Daily Paper.