Duplicates, delete & storage, users page with J-ID avatars

Backend:
- Perceptual hashes (aHash/dHash/pHash/wHash via imagehash, no imgdd)
  stored on items, computed on upload/download and by the new
  compute_visual_hashes command
- Duplicates API: exact duplicates (multi-location items), visual matches
  for one item, union-find similarity groups with pagination
- Delete API with ownership/staff checks, per-item and per-copy deletion,
  watched-folder path validation; storage overview and temp cleanup;
  file list accepts j_ids batches
- Staged uploads are flagged visual_match with their library matches
  (threshold via VISUAL_MATCH_THRESHOLD)
- Staff users API: list with upload counts, set role and avatar by J-ID;
  User.avatar FK with signed avatar URLs
- Download threads close their DB connection and stale tasks are reaped,
  keeping behaviour Gunicorn-friendly

Frontend:
- /duplicates: exact duplicate groups with per-copy delete, visual
  similarity controls, search similar to a J-ID, paginated groups with
  selection, bulk delete and dismiss
- /delete: storage cards, delete by J-ID with preview grid, temp cleanup
- /users: staff directory with role selects and avatar J-ID inputs
- Nav + command palette entries; top-bar avatar; upload cards and the
  metadata modal show library visual matches
This commit is contained in:
2026-09-17 12:49:10 -05:00
parent 75b7ed35eb
commit cd490b0a23
28 changed files with 1966 additions and 22 deletions
+43
View File
@@ -1,5 +1,6 @@
import hashlib
import json
import logging
import mimetypes
import os
import re
@@ -7,12 +8,19 @@ import shutil
import subprocess
from pathlib import Path
import imagehash
from django.conf import settings
from django.http import FileResponse, Http404, HttpResponse
from django.utils.text import get_valid_filename
from PIL import Image
from .models import MediaItem, MediaLocation
logger = logging.getLogger(__name__)
HASH_FIELDS = ("ahash", "dhash", "phash", "whash")
IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".gif", ".apng", ".webp"}
ALLOWED_EXTENSIONS = {
".jpg",
".jpeg",
@@ -82,6 +90,41 @@ def rename_location_to_j_id(item, location):
return location
def compute_visual_hashes(path):
"""Perceptual hashes for an image file (empty dict for other files)."""
path = Path(path)
if path.suffix.lower() not in IMAGE_EXTENSIONS:
return {}
try:
with Image.open(path) as image:
converted = image.convert("RGB")
return {
"ahash": str(imagehash.average_hash(converted, hash_size=8)),
"dhash": str(imagehash.dhash(converted, hash_size=8)),
"phash": str(imagehash.phash(converted, hash_size=8)),
"whash": str(imagehash.whash(converted, hash_size=8)),
}
except Exception: # noqa: BLE001 - hashing must never break indexing
logger.exception("Could not compute visual hashes for %s", path)
return {}
def ensure_visual_hashes(item):
"""Fill in missing perceptual hashes for a media item."""
if all(getattr(item, field) for field in HASH_FIELDS):
return item
location = item.locations.first()
if location is None:
return item
hashes = compute_visual_hashes(location.path)
if not hashes:
return item
for field, value in hashes.items():
setattr(item, field, value)
item.save(update_fields=[*hashes.keys(), "updated_at"])
return item
def parse_tags(raw):
"""Normalize a comma-separated string or JSON list into a list of tags."""
if raw is None: