JakeBreath
d0e2901c92
Backend: - TempUpload model: staged files (pending / visual_match / completed / error) with resolution, e621 payload, custom metadata and IQDB data - Files land in a temp folder and only move into the watched library folder once resolved; duplicates resolve immediately without a copy - Endpoints: stage (multipart), list, retrieve, temp file, IQDB save, resolve (link to post or custom metadata), discard/dismiss - cleanup_temp_uploads command for old staged files - Replaces the old direct-to-library upload endpoint Frontend: - Upload page is now a three-column board (Pending & Unmatched / Visual Similarity Detected / Auto-uploaded & Indexed) - After upload: MD5s are batch-checked against e621 and matches auto-complete with full post metadata; remaining files run through IQDB and move to the similarity column when candidates exist - Metadata modal with IQDB candidates, post-ID linking and custom tags/rating/notes; discard and dismiss actions - e621 client gains fetchPostsByMd5 and iqdbSearch helpers Roadmap updated with the completed upload items.
4.4 KiB
4.4 KiB
J621 Roadmap / To-Dos
Working list of what's still missing, roughly in priority order. Check items off as they land.
1. Library management
- Duplicates engine
- Exact MD5 duplicate detection with grouped results
- Perceptual similarity (aHash, dHash, pHash, wHash) with threshold slider
and algorithm toggles (needs
imagehash/imgddserver-side) - Visual similarity groups: pagination, "keep one / delete rest", dismiss group
- Delete & storage page
- Storage overview (watched folder, media folder)
- Delete by J-ID with preview grid and bulk selection
- Temp folder cleanup
- Library search upgrades
- Search by tags (custom + e621 tags), not just filename
- Tag cloud from the library
- Status filter (matched / not found / deleted / unknown / custom)
2. e621 integration
- Download progress bar on the detail view
- Backend download task model (status, progress %, downloaded/total bytes, speed)
- Progress endpoint the SPA polls; cancel support
- Frontend progress bar with speed and cancel on
/detail/<post>
- Match local files to e621
- Match by MD5 from the library detail (manual post ID entry)
- Batch cache status for the whole library (
matched/not_found/deleted) - Show match status + e621 metadata in the library (model already supports it)
- IQDB reverse search
- Search from a local file (
POST /iqdb_queries.json) - Show candidate posts to link
- Search from a local file (
- Follows
- Follow tags/pools, Followed screens with unseen badges
- "Followed" nav entry
Fshortcut to favorite a post, tag finder in the Ctrl+K palette
3. Uploads & ingestion pipeline
Files now stage first and are resolved before entering the library.
- Backend staging storage
TempUploadmodel: user, md5, filename, temp path, status (pending/visual_match/completed/error), resolution- Files land in a temp folder first; only completed uploads move into the watched library folder
- Pending/upload list endpoint, file serving, discard endpoint
cleanup_temp_uploadscommand for old staged files
- Auto-upload / auto-match
- MD5 computed on staging; exact duplicates resolve immediately
- MD5 batch-checked against e621; matches auto-complete with post metadata stored and rating seeded
- IQDB similarity on upload (SPA-driven)
- Automatic + manual IQDB checks with candidate posts
- "Visual Similarity Detected" state with candidate picker
- Perceptual-hash comparison against the library (shared with the duplicates engine, still pending)
- Upload UI
- Three-column board: Pending & Unmatched / Visual Similarity Detected / Auto-uploaded & Indexed
- Metadata modal (link to e621 post, IQDB candidates, custom metadata)
- Per-file progress plus batch processing indicator
4. Staff tools
- Optimization modal (image quality, GIF/APNG resolution+FPS+compression, video bitrate/presets/two-pass/hw-accel) with original vs processed preview and override
- Stats dashboard
- CPU / RAM / GPU / disk usage
- Active jobs list (this is what makes the footer "Active Workers" real)
- Live log tail
5. Shell & polish
- Toasts instead of inline messages / confirm dialogs
- Mobile drawer polish for metadata panels (design spec §layout)
- Avatar picker (choose from library)
- Profile extras (per-user browse preferences)
- Command palette: tag finder, recent searches
6. Infrastructure
- Guest blacklist refresh on a timer (
refresh_guest_blacklistvia cron/systemd) - Production setup: build the SPA, serve via Nginx (static +
/media+/library), systemd unit for Waitress - Automated tests (backend API + frontend components)
- Backfill e621 metadata for items downloaded before metadata was stored (re-download or a "match" action)
Dependencies / notes
- Duplicates and upload visual-similarity share the perceptual hashing layer
(
imagehash/imgddserver-side + a hash cache table). - Download progress, optimization jobs and the stats "Active Workers" count all want the same background-task/progress primitive — design it once.
- IQDB and e621 matching depend on e621 credentials being configured; the SPA already talks to e621 directly where possible.