From 6fe6e4cd55fe930820b7e9cd200bb5e628167242 Mon Sep 17 00:00:00 2001 From: Masahiko AMANO Date: Mon, 22 Jun 2026 22:09:04 +0300 Subject: [PATCH] chore(project): lower duplicate hash threshold to 4 in example env MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The distance histogram over a real ~65k-pair table is flat-to-rising from distance 1 and nearly doubles at 10 — the 8-10 band is dominated by coincidental matches, not duplicates. Default the example to 4 (the valley minimum) to keep the genuine-duplicate signal and cut the pairs to cluster. --- .env.example | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/.env.example b/.env.example index 389fbe1..11f7c11 100644 --- a/.env.example +++ b/.env.example @@ -131,9 +131,12 @@ IMPORT_PATH=/data/import # Maximum perceptual-hash distance (Hamming, out of 64 bits) for two files to be # treated as duplicate candidates. Lower = stricter (fewer, more confident # matches); higher = looser (catches more re-encodes/resizes but risks false -# positives). Used only by the dedup tool's pairs rebuild — see the dedup CLI / -# `docker compose run --rm dedup`. Default 10. -DUPLICATE_HASH_THRESHOLD=10 +# positives). On real libraries the distance histogram climbs steeply in the 8–10 +# band — coincidental "vaguely similar" pairs, not duplicates — so 4 keeps the +# genuine-duplicate signal without that noise (and far fewer pairs to cluster). +# Used only by the dedup tool's pairs rebuild — see the dedup CLI / +# `docker compose run --rm dedup`. Code default is 10. +DUPLICATE_HASH_THRESHOLD=4 # --------------------------------------------------------------------------- # Static SPA