JakeBreath
60e1113231
Includes the newly identified gaps: a download progress bar for Download to Library, and the upload pipeline rework (staging storage, MD5 auto-match, IQDB/visual similarity, three-column board).
4.5 KiB
4.5 KiB
J621 Roadmap / To-Dos
Working list of what's still missing, roughly in priority order. Check items off as they land.
1. Library management
- Duplicates engine
- Exact MD5 duplicate detection with grouped results
- Perceptual similarity (aHash, dHash, pHash, wHash) with threshold slider
and algorithm toggles (needs
imagehash/imgddserver-side) - Visual similarity groups: pagination, "keep one / delete rest", dismiss group
- Delete & storage page
- Storage overview (watched folder, media folder)
- Delete by J-ID with preview grid and bulk selection
- Temp folder cleanup
- Library search upgrades
- Search by tags (custom + e621 tags), not just filename
- Tag cloud from the library
- Status filter (matched / not found / deleted / unknown / custom)
2. e621 integration
- Download progress bar on the detail view
- Backend download task model (status, progress %, downloaded/total bytes, speed)
- Progress endpoint the SPA polls; cancel support
- Frontend progress bar with speed and cancel on
/detail/<post>
- Match local files to e621
- Match by MD5 from the library detail (manual post ID entry)
- Batch cache status for the whole library (
matched/not_found/deleted) - Show match status + e621 metadata in the library (model already supports it)
- IQDB reverse search
- Search from a local file (
POST /iqdb_queries.json) - Show candidate posts to link
- Search from a local file (
- Follows
- Follow tags/pools, Followed screens with unseen badges
- "Followed" nav entry
Fshortcut to favorite a post, tag finder in the Ctrl+K palette
3. Uploads & ingestion pipeline (major rework)
Current upload writes straight into the watched folder. The target flow matches the original app:
- Backend staging storage
TempUploadmodel: temp_id, user, md5, filename, temp path, status (pending/visual_match/completed/error), timestamps- Files land in a temp folder first; only "completed" moves them into the watched library folder
- Pending uploads list endpoint, delete/discard endpoint
- Auto-upload / auto-match
- Compute MD5 on upload; if it already exists in the library, mark as duplicate and resolve automatically (no new file)
- If the MD5 matches an e621 post, auto-complete: fetch and store the post metadata, seed rating/tags, mark as indexed
- Batch MD5 checks against e621 for staged uploads
- IQDB / visual similarity on upload
- Run IQDB on staged uploads and surface candidate posts
- Perceptual-hash check against the library → "Visual Similarity Detected" state with side-by-side comparison
- Upload UI
- Three-column board: Pending & Unmatched / Visual Similarity Detected / Auto-uploaded & Indexed
- Metadata modal (link to e621 post via search + custom tags/rating/notes)
- Per-file progress plus batch summary
4. Staff tools
- Optimization modal (image quality, GIF/APNG resolution+FPS+compression, video bitrate/presets/two-pass/hw-accel) with original vs processed preview and override
- Stats dashboard
- CPU / RAM / GPU / disk usage
- Active jobs list (this is what makes the footer "Active Workers" real)
- Live log tail
5. Shell & polish
- Toasts instead of inline messages / confirm dialogs
- Mobile drawer polish for metadata panels (design spec §layout)
- Avatar picker (choose from library)
- Profile extras (per-user browse preferences)
- Command palette: tag finder, recent searches
6. Infrastructure
- Guest blacklist refresh on a timer (
refresh_guest_blacklistvia cron/systemd) - Production setup: build the SPA, serve via Nginx (static +
/media+/library), systemd unit for Waitress - Automated tests (backend API + frontend components)
- Backfill e621 metadata for items downloaded before metadata was stored (re-download or a "match" action)
Dependencies / notes
- Duplicates and upload visual-similarity share the perceptual hashing layer
(
imagehash/imgddserver-side + a hash cache table). - Download progress, optimization jobs and the stats "Active Workers" count all want the same background-task/progress primitive — design it once.
- IQDB and e621 matching depend on e621 credentials being configured; the SPA already talks to e621 directly where possible.