From 60e111323106b279c8bca870d8f2d56cc871202e Mon Sep 17 00:00:00 2001 From: JakeBreath Date: Thu, 17 Sep 2026 10:59:18 -0500 Subject: [PATCH] Add roadmap with the remaining work Includes the newly identified gaps: a download progress bar for Download to Library, and the upload pipeline rework (staging storage, MD5 auto-match, IQDB/visual similarity, three-column board). --- ROADMAP.md | 101 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 ROADMAP.md diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..71b2240 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,101 @@ +# J621 Roadmap / To-Dos + +Working list of what's still missing, roughly in priority order. Check items off as +they land. + +## 1. Library management + +- [ ] **Duplicates engine** + - [ ] Exact MD5 duplicate detection with grouped results + - [ ] Perceptual similarity (aHash, dHash, pHash, wHash) with threshold slider + and algorithm toggles (needs `imagehash`/`imgdd` server-side) + - [ ] Visual similarity groups: pagination, "keep one / delete rest", dismiss group +- [ ] **Delete & storage page** + - [ ] Storage overview (watched folder, media folder) + - [ ] Delete by J-ID with preview grid and bulk selection + - [ ] Temp folder cleanup +- [ ] **Library search upgrades** + - [ ] Search by tags (custom + e621 tags), not just filename + - [ ] Tag cloud from the library + - [ ] Status filter (matched / not found / deleted / unknown / custom) + +## 2. e621 integration + +- [ ] **Download progress bar on the detail view** + - [ ] Backend download task model (status, progress %, downloaded/total bytes, speed) + - [ ] Progress endpoint the SPA polls; cancel support + - [ ] Frontend progress bar with speed and cancel on `/detail/` +- [ ] **Match local files to e621** + - [ ] Match by MD5 from the library detail (manual post ID entry) + - [ ] Batch cache status for the whole library (`matched` / `not_found` / `deleted`) + - [ ] Show match status + e621 metadata in the library (model already supports it) +- [ ] **IQDB reverse search** + - [ ] Search from a local file (`POST /iqdb_queries.json`) + - [ ] Show candidate posts to link +- [ ] **Follows** + - [ ] Follow tags/pools, Followed screens with unseen badges + - [ ] "Followed" nav entry +- [ ] `F` shortcut to favorite a post, tag finder in the Ctrl+K palette + +## 3. Uploads & ingestion pipeline (major rework) + +Current upload writes straight into the watched folder. The target flow matches +the original app: + +- [ ] **Backend staging storage** + - [ ] `TempUpload` model: temp_id, user, md5, filename, temp path, status + (`pending` / `visual_match` / `completed` / `error`), timestamps + - [ ] Files land in a temp folder first; only "completed" moves them into the + watched library folder + - [ ] Pending uploads list endpoint, delete/discard endpoint +- [ ] **Auto-upload / auto-match** + - [ ] Compute MD5 on upload; if it already exists in the library, mark as + duplicate and resolve automatically (no new file) + - [ ] If the MD5 matches an e621 post, auto-complete: fetch and store the post + metadata, seed rating/tags, mark as indexed + - [ ] Batch MD5 checks against e621 for staged uploads +- [ ] **IQDB / visual similarity on upload** + - [ ] Run IQDB on staged uploads and surface candidate posts + - [ ] Perceptual-hash check against the library → "Visual Similarity Detected" + state with side-by-side comparison +- [ ] **Upload UI** + - [ ] Three-column board: Pending & Unmatched / Visual Similarity Detected / + Auto-uploaded & Indexed + - [ ] Metadata modal (link to e621 post via search + custom tags/rating/notes) + - [ ] Per-file progress plus batch summary + +## 4. Staff tools + +- [ ] **Optimization modal** (image quality, GIF/APNG resolution+FPS+compression, + video bitrate/presets/two-pass/hw-accel) with original vs processed preview + and override +- [ ] **Stats dashboard** + - [ ] CPU / RAM / GPU / disk usage + - [ ] Active jobs list (this is what makes the footer "Active Workers" real) + - [ ] Live log tail + +## 5. Shell & polish + +- [ ] Toasts instead of inline messages / confirm dialogs +- [ ] Mobile drawer polish for metadata panels (design spec §layout) +- [ ] Avatar picker (choose from library) +- [ ] Profile extras (per-user browse preferences) +- [ ] Command palette: tag finder, recent searches + +## 6. Infrastructure + +- [ ] Guest blacklist refresh on a timer (`refresh_guest_blacklist` via cron/systemd) +- [ ] Production setup: build the SPA, serve via Nginx (static + `/media` + + `/library`), systemd unit for Waitress +- [ ] Automated tests (backend API + frontend components) +- [ ] Backfill e621 metadata for items downloaded before metadata was stored + (re-download or a "match" action) + +## Dependencies / notes + +- Duplicates and upload visual-similarity share the perceptual hashing layer + (`imagehash`/`imgdd` server-side + a hash cache table). +- Download progress, optimization jobs and the stats "Active Workers" count all + want the same background-task/progress primitive — design it once. +- IQDB and e621 matching depend on e621 credentials being configured; the SPA + already talks to e621 directly where possible.