Upload pipeline: staging, MD5 auto-match, IQDB, three-column board
Backend: - TempUpload model: staged files (pending / visual_match / completed / error) with resolution, e621 payload, custom metadata and IQDB data - Files land in a temp folder and only move into the watched library folder once resolved; duplicates resolve immediately without a copy - Endpoints: stage (multipart), list, retrieve, temp file, IQDB save, resolve (link to post or custom metadata), discard/dismiss - cleanup_temp_uploads command for old staged files - Replaces the old direct-to-library upload endpoint Frontend: - Upload page is now a three-column board (Pending & Unmatched / Visual Similarity Detected / Auto-uploaded & Indexed) - After upload: MD5s are batch-checked against e621 and matches auto-complete with full post metadata; remaining files run through IQDB and move to the similarity column when candidates exist - Metadata modal with IQDB candidates, post-ID linking and custom tags/rating/notes; discard and dismiss actions - e621 client gains fetchPostsByMd5 and iqdbSearch helpers Roadmap updated with the completed upload items.
This commit is contained in:
+22
-23
@@ -37,32 +37,31 @@ they land.
|
||||
- [ ] "Followed" nav entry
|
||||
- [ ] `F` shortcut to favorite a post, tag finder in the Ctrl+K palette
|
||||
|
||||
## 3. Uploads & ingestion pipeline (major rework)
|
||||
## 3. Uploads & ingestion pipeline
|
||||
|
||||
Current upload writes straight into the watched folder. The target flow matches
|
||||
the original app:
|
||||
Files now stage first and are resolved before entering the library.
|
||||
|
||||
- [ ] **Backend staging storage**
|
||||
- [ ] `TempUpload` model: temp_id, user, md5, filename, temp path, status
|
||||
(`pending` / `visual_match` / `completed` / `error`), timestamps
|
||||
- [ ] Files land in a temp folder first; only "completed" moves them into the
|
||||
watched library folder
|
||||
- [ ] Pending uploads list endpoint, delete/discard endpoint
|
||||
- [ ] **Auto-upload / auto-match**
|
||||
- [ ] Compute MD5 on upload; if it already exists in the library, mark as
|
||||
duplicate and resolve automatically (no new file)
|
||||
- [ ] If the MD5 matches an e621 post, auto-complete: fetch and store the post
|
||||
metadata, seed rating/tags, mark as indexed
|
||||
- [ ] Batch MD5 checks against e621 for staged uploads
|
||||
- [ ] **IQDB / visual similarity on upload**
|
||||
- [ ] Run IQDB on staged uploads and surface candidate posts
|
||||
- [ ] Perceptual-hash check against the library → "Visual Similarity Detected"
|
||||
state with side-by-side comparison
|
||||
- [ ] **Upload UI**
|
||||
- [ ] Three-column board: Pending & Unmatched / Visual Similarity Detected /
|
||||
- [x] **Backend staging storage**
|
||||
- [x] `TempUpload` model: user, md5, filename, temp path, status
|
||||
(`pending` / `visual_match` / `completed` / `error`), resolution
|
||||
- [x] Files land in a temp folder first; only completed uploads move into
|
||||
the watched library folder
|
||||
- [x] Pending/upload list endpoint, file serving, discard endpoint
|
||||
- [x] `cleanup_temp_uploads` command for old staged files
|
||||
- [x] **Auto-upload / auto-match**
|
||||
- [x] MD5 computed on staging; exact duplicates resolve immediately
|
||||
- [x] MD5 batch-checked against e621; matches auto-complete with post
|
||||
metadata stored and rating seeded
|
||||
- [x] **IQDB similarity on upload** (SPA-driven)
|
||||
- [x] Automatic + manual IQDB checks with candidate posts
|
||||
- [x] "Visual Similarity Detected" state with candidate picker
|
||||
- [ ] Perceptual-hash comparison against the library (shared with the
|
||||
duplicates engine, still pending)
|
||||
- [x] **Upload UI**
|
||||
- [x] Three-column board: Pending & Unmatched / Visual Similarity Detected /
|
||||
Auto-uploaded & Indexed
|
||||
- [ ] Metadata modal (link to e621 post via search + custom tags/rating/notes)
|
||||
- [ ] Per-file progress plus batch summary
|
||||
- [x] Metadata modal (link to e621 post, IQDB candidates, custom metadata)
|
||||
- [x] Per-file progress plus batch processing indicator
|
||||
|
||||
## 4. Staff tools
|
||||
|
||||
|
||||
Reference in New Issue
Block a user