Match local files to e621: MD5 lookups, manual links, batch scans

- MediaItem gains e621_match_status (unknown/matched/not_found/deleted)
  and e621_checked_at, backfilled for existing matched items.
- Server-side e621 client (apps/library/e621.py) using the user's stored
  credentials, throttled to 2 req/s, with typed errors.
- Matching service: MD5 lookup, manual post linking (flags MD5
  mismatches), unlink, metadata refresh, deleted-post detection.
- Detail actions POST /api/files/J-x/match/ and /unlink/ (uploader or
  staff only).
- Background library scans: MatchTask + /api/matches/ with missing/all
  scopes, progress polling, cancel and stale-task reaping; the scan
  counts toward the footer's Active Workers. Same pass available as
  manage.py match_e621 for cron.
- Library gains not_found/deleted status filters; the detail page adds
  an e621 match card (check / link by post ID / unlink) and the metadata
  card warns when a post was deleted on e621.
This commit is contained in:
2026-09-17 13:41:08 -05:00
parent db74f7ab18
commit 09405d1a0f
16 changed files with 1084 additions and 27 deletions
+5 -5
View File
@@ -9,15 +9,15 @@ from .system_info import get_os_info
def _worker_counts():
"""Active/queued background jobs (download tasks for now)."""
from apps.library.models import DownloadTask
"""Active/queued background jobs (downloads and e621 match scans)."""
from apps.library.models import DownloadTask, MatchTask
active = DownloadTask.objects.filter(
status=DownloadTask.STATUS_DOWNLOADING
).count()
).count() + MatchTask.objects.filter(status=MatchTask.STATUS_RUNNING).count()
queued = DownloadTask.objects.filter(
status=DownloadTask.STATUS_PENDING
).count()
).count() + MatchTask.objects.filter(status=MatchTask.STATUS_PENDING).count()
return {"active": active, "queued": queued}
@@ -35,7 +35,7 @@ class StatusView(APIView):
"version": settings.APP_VERSION,
"os": get_os_info(),
"watched_folder": settings.WATCHED_FOLDER,
"e621_time_ms": None, # no e621 calls in the backend yet
"e621_time_ms": None, # e621 latency is measured client-side
"workers": _worker_counts(),
}
payload["server_time_ms"] = round((time.perf_counter() - started) * 1000, 1)