- /similar (nav: Similar): drop a file to get the exact MD5 match, the
perceptual matches against the library, and e621 IQDB candidates
(auto-run for images when credentials are configured). Read-only —
nothing enters the library.
- SimilarityCheck model + /api/similarity/ (create/list/retrieve/delete)
with signed preview URLs and an expires_at timestamp.
- Temp files are wiped on startup (AppConfig.ready, file-only so no
database access during initialization), lazily past
SIMILARITY_TTL_MINUTES (default 30, env-overridable), on delete, and
by manage.py cleanup_similarity.
- uploadFile() takes a target path; .env.example documents the TTL.
- MediaItem gains e621_match_status (unknown/matched/not_found/deleted)
and e621_checked_at, backfilled for existing matched items.
- Server-side e621 client (apps/library/e621.py) using the user's stored
credentials, throttled to 2 req/s, with typed errors.
- Matching service: MD5 lookup, manual post linking (flags MD5
mismatches), unlink, metadata refresh, deleted-post detection.
- Detail actions POST /api/files/J-x/match/ and /unlink/ (uploader or
staff only).
- Background library scans: MatchTask + /api/matches/ with missing/all
scopes, progress polling, cancel and stale-task reaping; the scan
counts toward the footer's Active Workers. Same pass available as
manage.py match_e621 for cron.
- Library gains not_found/deleted status filters; the detail page adds
an e621 match card (check / link by post ID / unlink) and the metadata
card warns when a post was deleted on e621.
Backend:
- MediaItem gains search_tags (custom + e621 tags, lowercase) and
has_custom_data, maintained on save with a data migration backfill
- File list search accepts search_type=filename|tags|both (tag search is
word-AND across the flattened tag text) and status=matched|custom|
unknown filters
- New /api/tags/cloud/ endpoint (cached 2 min per guest/auth, invalidated
on item changes and deletions) returning the most-used tags, honouring
guest visibility
Frontend:
- Library sidebar: Filename/Tags/Both selector, status pill toggles
(persisted), and a clickable tag cloud that runs a tag search
- Roadmap updated
Backend:
- Perceptual hashes (aHash/dHash/pHash/wHash via imagehash, no imgdd)
stored on items, computed on upload/download and by the new
compute_visual_hashes command
- Duplicates API: exact duplicates (multi-location items), visual matches
for one item, union-find similarity groups with pagination
- Delete API with ownership/staff checks, per-item and per-copy deletion,
watched-folder path validation; storage overview and temp cleanup;
file list accepts j_ids batches
- Staged uploads are flagged visual_match with their library matches
(threshold via VISUAL_MATCH_THRESHOLD)
- Staff users API: list with upload counts, set role and avatar by J-ID;
User.avatar FK with signed avatar URLs
- Download threads close their DB connection and stale tasks are reaped,
keeping behaviour Gunicorn-friendly
Frontend:
- /duplicates: exact duplicate groups with per-copy delete, visual
similarity controls, search similar to a J-ID, paginated groups with
selection, bulk delete and dismiss
- /delete: storage cards, delete by J-ID with preview grid, temp cleanup
- /users: staff directory with role selects and avatar J-ID inputs
- Nav + command palette entries; top-bar avatar; upload cards and the
metadata modal show library visual matches
Backend:
- DownloadTask model + background thread runner: streams the file with
progress (%, bytes, speed) and a cancel flag, then indexes it, names it
J-<id>.<ext> and applies the e621 metadata
- DownloadTaskViewSet (create/retrieve/cancel) replaces the synchronous
endpoint; the status footer's worker counts now reflect download jobs
- Client download proxy (/api/online/file/) streams an e621 original to
the browser with Content-Disposition: attachment, restricted to the
configured e621 CDN hosts so it cannot be used as an open proxy
Frontend:
- Online detail: progress bar with percentage, transferred size, speed
and cancel while downloading; success links to the new J-ID
- New 'Download to client' button available to everyone (guests too)
Backend:
- TempUpload model: staged files (pending / visual_match / completed /
error) with resolution, e621 payload, custom metadata and IQDB data
- Files land in a temp folder and only move into the watched library
folder once resolved; duplicates resolve immediately without a copy
- Endpoints: stage (multipart), list, retrieve, temp file, IQDB save,
resolve (link to post or custom metadata), discard/dismiss
- cleanup_temp_uploads command for old staged files
- Replaces the old direct-to-library upload endpoint
Frontend:
- Upload page is now a three-column board (Pending & Unmatched /
Visual Similarity Detected / Auto-uploaded & Indexed)
- After upload: MD5s are batch-checked against e621 and matches
auto-complete with full post metadata; remaining files run through
IQDB and move to the similarity column when candidates exist
- Metadata modal with IQDB candidates, post-ID linking and custom
tags/rating/notes; discard and dismiss actions
- e621 client gains fetchPostsByMd5 and iqdbSearch helpers
Roadmap updated with the completed upload items.
Roles:
- JakeBreathild is now staff + superuser (the real account); the 'jake'
smoke-test account was demoted to a regular user
- /me exposes is_superuser and the account page shows an admin badge
e621 metadata:
- MediaItem gains e621_post_id and e621_data (trimmed post payload:
tags by category, rating, score, favourites, comments, sources,
description, pools, relationships, file info, uploader)
- Download to Library accepts the post payload from the SPA and stores
it; the item's custom rating is seeded from the e621 rating when empty
- Library detail shows an e621 metadata card: link to the in-app post,
score/favourites/comments, taxonomy-coloured tags, DText description,
sources and pools; grid cards get an e621 badge and fall back to the
e621 rating for their colour (display_rating)
- Guest visibility now also considers e621 tags, so downloaded explicit
content is hidden from anonymous visitors
Backend:
- User.role (user/uploader/staff) with can_upload; uploads and downloads
gated to uploader+; owners and staff can edit their items
- MediaItem.uploaded_by plus J-<id> identity (serializer, admin,
scan_files --user, first superuser as default owner)
- API resolves J-<id>, bare numeric ids and MD5s; neighbors and lookup
return j_ids
- Guest safety: mirror e621's anonymous default blacklist into Redis
(parses comments, negations and wildcards), flag hidden_from_guests
and filter lists, details and lookups for anonymous users
- POST /api/online/downloads/ writes an e621 file into the watched
folder and indexes it for the uploader
- MariaDB + Redis via docker compose (host ports 3307/6380), PyMySQL
driver shim, Redis cache replacing the file cache; SQLite data
dumped and loaded into MariaDB
Frontend:
- Single /detail/:itemId route with an adaptive shell: J-<id> renders
the library item, bare numbers render the e621 post
- Legacy /view/<md5> and /online/view/<id> redirect to canonical URLs
- Cards expose J-IDs; library custom-data editor is read-only for
non-owners
- Role gating: Upload hidden/blocked for regular users, account shows
the role, guest hint on the library
Backend (Django 6.1 + DRF):
- Token auth with a custom User model (register/login/logout/me)
- Library models (MediaItem, MediaLocation) and REST endpoints
- File list/detail with search, rating filter, sorting, pagination
- Multipart upload with optional rating/tags/notes
- Range-aware media serving (video seeking) and ffmpeg thumbnails
- scan_files management command for the watched folder
Frontend (React 19 + Vite + TypeScript):
- Catppuccin Mocha design tokens from the design docs
- App shell, token persistence, protected routes
- Library grid with filters, file detail with custom data editor
- Upload page with per-file progress via XHR
- Dev proxy to the Django API
JakeBreath