Match local files to e621: MD5 lookups, manual links, batch scans

- MediaItem gains e621_match_status (unknown/matched/not_found/deleted)
  and e621_checked_at, backfilled for existing matched items.
- Server-side e621 client (apps/library/e621.py) using the user's stored
  credentials, throttled to 2 req/s, with typed errors.
- Matching service: MD5 lookup, manual post linking (flags MD5
  mismatches), unlink, metadata refresh, deleted-post detection.
- Detail actions POST /api/files/J-x/match/ and /unlink/ (uploader or
  staff only).
- Background library scans: MatchTask + /api/matches/ with missing/all
  scopes, progress polling, cancel and stale-task reaping; the scan
  counts toward the footer's Active Workers. Same pass available as
  manage.py match_e621 for cron.
- Library gains not_found/deleted status filters; the detail page adds
  an e621 match card (check / link by post ID / unlink) and the metadata
  card warns when a post was deleted on e621.
This commit is contained in:
2026-09-17 13:41:08 -05:00
parent db74f7ab18
commit 09405d1a0f
16 changed files with 1084 additions and 27 deletions
+27 -1
View File
@@ -4,7 +4,7 @@ from pathlib import Path
from django.core import signing
from rest_framework import serializers
from .models import MediaItem, MediaLocation, TempUpload, DownloadTask
from .models import MediaItem, MediaLocation, TempUpload, DownloadTask, MatchTask
from .services import (
MEDIA_FILE_SALT,
UPLOAD_FILE_SALT,
@@ -55,6 +55,8 @@ class MediaItemSerializer(serializers.ModelSerializer):
"uploaded_by_id",
"e621_post_id",
"e621_data",
"e621_match_status",
"e621_checked_at",
"locations",
"created_at",
"updated_at",
@@ -74,6 +76,8 @@ class MediaItemSerializer(serializers.ModelSerializer):
"uploaded_by_id",
"e621_post_id",
"e621_data",
"e621_match_status",
"e621_checked_at",
"locations",
"created_at",
"updated_at",
@@ -220,3 +224,25 @@ class DownloadTaskSerializer(serializers.ModelSerializer):
def get_library_j_id(self, obj):
return f"J-{obj.library_item_id}" if obj.library_item_id else None
class MatchTaskSerializer(serializers.ModelSerializer):
task_id = serializers.UUIDField(source="id", read_only=True)
class Meta:
model = MatchTask
fields = [
"task_id",
"scope",
"status",
"total",
"processed",
"matched",
"not_found",
"deleted",
"failed",
"error",
"created_at",
"updated_at",
]
read_only_fields = fields