Upload pipeline: staging, MD5 auto-match, IQDB, three-column board

Backend:
- TempUpload model: staged files (pending / visual_match / completed /
  error) with resolution, e621 payload, custom metadata and IQDB data
- Files land in a temp folder and only move into the watched library
  folder once resolved; duplicates resolve immediately without a copy
- Endpoints: stage (multipart), list, retrieve, temp file, IQDB save,
  resolve (link to post or custom metadata), discard/dismiss
- cleanup_temp_uploads command for old staged files
- Replaces the old direct-to-library upload endpoint

Frontend:
- Upload page is now a three-column board (Pending & Unmatched /
  Visual Similarity Detected / Auto-uploaded & Indexed)
- After upload: MD5s are batch-checked against e621 and matches
  auto-complete with full post metadata; remaining files run through
  IQDB and move to the similarity column when candidates exist
- Metadata modal with IQDB candidates, post-ID linking and custom
  tags/rating/notes; discard and dismiss actions
- e621 client gains fetchPostsByMd5 and iqdbSearch helpers

Roadmap updated with the completed upload items.
This commit is contained in:
2026-09-17 11:12:03 -05:00
parent 60e1113231
commit d0e2901c92
13 changed files with 1189 additions and 118 deletions
-50
View File
@@ -8,7 +8,6 @@ from django.http import Http404
from django.shortcuts import get_object_or_404
from rest_framework import mixins, status, viewsets
from rest_framework.decorators import action
from rest_framework.parsers import FormParser, MultiPartParser
from rest_framework.permissions import AllowAny, IsAuthenticatedOrReadOnly
from rest_framework.response import Response
from rest_framework.views import APIView
@@ -165,55 +164,6 @@ class MediaItemViewSet(
)
class UploadView(APIView):
permission_classes = [CanUpload]
parser_classes = [MultiPartParser, FormParser]
def post(self, request):
upload = request.FILES.get("file")
if upload is None:
return Response(
{"detail": "A file is required."}, status=status.HTTP_400_BAD_REQUEST
)
folder = Path(settings.WATCHED_FOLDER)
destination = services.unique_destination(folder, upload.name)
with open(destination, "wb") as output:
for chunk in upload.chunks():
output.write(chunk)
try:
item, created_item, _ = services.index_file(destination, folder)
except Exception as exc: # noqa: BLE001 - report indexing failures
destination.unlink(missing_ok=True)
return Response(
{"detail": f"Could not index upload: {exc}"},
status=status.HTTP_400_BAD_REQUEST,
)
update_fields = []
if created_item and item.uploaded_by_id is None:
item.uploaded_by = request.user
update_fields.append("uploaded_by")
rating = request.data.get("rating") or ""
if rating in {"s", "q", "e"} and item.rating != rating:
item.rating = rating
update_fields.append("rating")
tags = services.parse_tags(request.data.get("tags"))
if tags is not None:
item.tags = tags
update_fields.append("tags")
notes = request.data.get("notes")
if notes is not None:
item.notes = str(notes)
update_fields.append("notes")
if update_fields:
item.save(update_fields=update_fields + ["updated_at"])
return Response(
MediaItemSerializer(item, context={"request": request}).data,
status=status.HTTP_201_CREATED,
)
class DownloadToLibraryView(APIView):
"""Fetch an e621 file into the watched folder and index it."""