Commit Graph
137 Commits
Author SHA1 Message Date
JakeBreath 7c2569522f Make thumbnail generation atomic and warm it on import
- write thumbnails to a .part file and os.replace() them, so concurrent
  requests never read a half-written JPEG
- a stale thumbnail plus a vanished source no longer raises through the
  request (getmtime on a missing file returned 500); it falls back cleanly
- ensure_thumbnail(item) warms the preview when a file is indexed, keeping
  image decoding out of the request path
2026-09-23 21:35:09 -05:00
JakeBreath a62195ffce Re-sign visual-match thumbnails on every detail fetch
Match rows stored a signed URL minted when the scan ran, so it aged out (or
used the pre-stable signing scheme) and the modal showed broken tiles even
after legacy signatures were fixed. Rows now carry item_id/j_id and the
detail serializer mints a fresh thumbnail URL per request; matches whose
item no longer exists are dropped.
2026-09-23 21:33:20 -05:00
JakeBreath c73a81a5f4 Fix 500 on legacy signed URLs
A TimestampSigner value is an HMAC over 'payload:timestamp', so a plain
Signer's HMAC check accepts it and the embedded timestamp then reached the
JSON decoder, raising JSONDecodeError (not BadSignature) and surfacing as a
500. That broke every stored visual-match thumbnail URL minted before the
stable scheme, so the J-ID match tiles never loaded on prod.

Detect the legacy shape by its extra separator and verify it with
TimestampSigner; malformed input returns None instead of raising.
2026-09-23 21:31:34 -05:00
JakeBreath af378e7d71 Update the roadmap for cacheable media and session-only completions
CI / Backend tests (push) Successful in 2m29s
CI / Frontend build & lint (push) Successful in 25s
2026-09-23 18:29:49 -05:00
JakeBreath 59f397a96c Show indexed uploads as session notifications, add a Failed column
The board rendered every persisted completed row with a dismiss button and a
dismiss-all that sent the whole column to the 1000-id bulk endpoint. Now that
the backend deletes completed rows and reports them through the status feed:

- the Auto-uploaded & Indexed column renders the live feed only; a reload or
  leaving the page forgets them, and there is nothing to dismiss (just a
  client-side clear)
- duplicates complete during staging and get their card immediately from the
  upload response
- failures get their own persisted column with per-card retry and discard
- bulk discard chunks requests at 500 ids, so backlogs over the server's
  1000-id cap are still removable in one action
2026-09-23 18:28:35 -05:00
JakeBreath 9ababb8b48 Stop storing completed uploads; announce them through a live feed
Every auto-matched, duplicate or manually resolved upload left a completed
TempUpload row on the board until it was dismissed by hand, so the rows
accumulated without bound and the bulk dismiss (capped at 1000 ids) failed
once there were more. The original app never stored these: they are
notifications, not records.

- complete_temp_upload now appends {filename, J-ID, resolution, post} to a
  bounded recent_completions feed on UploadRun and deletes the staged row
- staging duplicates never create a board record either; the create response
  carries the J-ID and preview so the SPA can show the card immediately
- status_payload returns the feed (newest first, signed thumbnails) for the
  live board; finalize_round counts deleted matches in processed
- resolve/link-bulk return synthetic completion payloads
- migration 0012 adds the field and purges the existing completed backlog
  (and any stray staged files) on deploy
2026-09-23 18:24:38 -05:00
JakeBreath 37085c5dac Make media URLs stable and cacheable, add real image thumbnails
Signed media URLs embedded the current second (TimestampSigner), so every
API response re-minted every raw/thumbnail/staged URL and the browser
re-downloaded each file on every poll or navigation. Responses also carried
no cache headers at all.

- sign with a plain Signer plus a bucket-quantized exp (7d TTL, 24h bucket),
  so a URL is byte-identical across responses and rotates once a day; legacy
  TimestampSigner URLs stay accepted for one release
- add a v=<md5> version parameter to library media URLs so replacing a file
  under the same J-ID (the optimize flow) busts caches exactly when needed
- serve_file now sends ETag/Last-Modified and a private Cache-Control and
  answers conditional requests with 304; library media gets max-age 6d +
  immutable, staged/similarity files 1h
- build cached 480px JPEG thumbnails for images (Pillow, keyed by MD5 under
  MEDIA_ROOT/thumbs) instead of serving full-size originals through the
  thumbnail endpoint; the library grid uses thumbnail_url for images too
2026-09-23 18:13:52 -05:00
JakeBreath ecac4cb8b4 Fix the release asset upload for names with spaces
CI / Backend tests (push) Successful in 2m4s
CI / Frontend build & lint (push) Successful in 23s
curl exit 3 (malformed URL) on 'J621 Setup 0.1.1.exe': percent-encode the
asset name in the query string.

Also replace assets instead of skipping them on re-runs: NSIS builds are
not bit-reproducible, so latest.yml/latest-linux.yml must reference the
installers produced by the same run. Existing assets are deleted by id
before the fresh upload.
2026-09-23 07:14:19 -05:00
JakeBreath 2eb7af0d41 Fix the CD wine build and add per-part toggles
CI / Backend tests (push) Successful in 2m13s
CI / Frontend build & lint (push) Successful in 20s
The Windows NSIS step failed under wine for two reasons: no X display
(nodrv_CreateWindow) and missing 32-bit libraries (failed to load
syswow64\ntdll.dll). The job now installs xvfb + wine32:i386 and runs the
desktop build under xvfb-run, with Gecko/Mono lookups disabled.

Also add `images` and `desktop` dispatch inputs so either half of the CD
can be skipped (e.g. desktop-only or images-only releases).
desktop-v0.1.1
2026-09-23 00:06:55 -05:00
JakeBreath 7cecfeabc6 Use a minimal registry PAT and the job token for releases
CI / Backend tests (push) Successful in 2m23s
CI / Frontend build & lint (push) Successful in 22s
Gitea's container registry rejects the automatic job token
(go-gitea/gitea#23642 is still open), so the image push keeps a PAT with
only the write:package scope; a preflight step fails clearly when the
REGISTRY_USER/REGISTRY_TOKEN secrets are missing. Release creation needs
no PAT: the desktop job asks for contents: write on the job token.
2026-09-22 23:35:55 -05:00
JakeBreath ed6178d12e Make Actions token permissions explicit
The automatic job token creates the desktop release, so the CD desktop job
asks for contents: write; the images job keeps contents: read and asks for
packages: write so the job token can stand in for the scoped registry PAT.
CI stays read-only. The registry token itself remains a write:package-only
PAT (verified login + pull).
2026-09-22 23:30:37 -05:00
JakeBreath 8f9656ac0e Add the manual CD release workflow
One dispatch builds and pushes both images and builds the desktop packages
into a Gitea release (desktop-v<version>, installers + latest*.yml attached,
idempotent on re-run). The live update feed stays a deploy-host operation:
CI has no SSH key for jakerasp, so push_desktop.sh --no-build remains the
way to publish it.

Repo secrets REGISTRY_USER/REGISTRY_TOKEN are set, so the image push uses
the Gitea registry credentials directly.
2026-09-22 23:23:05 -05:00
JakeBreath 72fc42217f Fix CI for the user-scoped runners
- ci.yml: connect to the test MariaDB as root so Django creates the test
  database itself (no client install/grant step), and drop actions/cache
  (cache: pip/npm): Gitea's cache service hangs the job on restore/save.
- publish.yml: prefer the REGISTRY_USER/REGISTRY_TOKEN secrets (as on other
  repos) and fall back to the automatic Actions token.
- AGENTS.md: note the CI layout, the runner labels and the cache caveat.
2026-09-22 23:12:56 -05:00
JakeBreath d9c1e9e521 Add CI and manual image publishing workflows
CI / Backend tests (push) Successful in 12m54s
CI / Frontend build & lint (push) Failing after 4m59s
- .gitea/workflows/ci.yml: on every push/PR, run Django checks + the full
  backend suite against MariaDB/Redis services and the frontend
  lint/type-check/build. Runs on the nitro-ci runner (ubuntu-latest).
- .gitea/workflows/publish.yml: manual dispatch; multi-arch build+push of
  both images as :latest and :<short-sha> with GIT_HASH baked in.
- push_*.sh: non-interactive registry login for CI (REGISTRY_USER/
  REGISTRY_TOKEN) and a PLATFORMS override.
2026-09-22 22:32:37 -05:00
JakeBreath e2697c0a78 Stop throttling signed media and ease the browser's e621 queue
Signed media URLs are fetched by <img>/<video> tags without an
Authorization header, so they were charged to the anonymous 120/min
bucket: past that, galleries and the fish-greeting download got 429 JSON
instead of image bytes. The raw/thumbnail/staged-file/similarity-file
actions are now exempt, and THROTTLE_ENABLED=false removes the general
anon+user limits for private/tailnet deployments (login/register/proxy
guards stay).

The SPA's e621 client also stops self-throttling so hard: 1s gap between
browsing calls (2.5s for the stricter IQDB endpoint) and a 15s cooldown
instead of 60s when e621 answers 429.
2026-09-22 22:32:37 -05:00
JakeBreath 474403ffe2 Upload updates 2026-09-21 09:01:01 -05:00
JakeBreath 98025e9e6d Prune old installers from the remote feed on push
rsync without --delete left every previous version on the server (the
screenshot showed 0.1.0 and 0.1.1 side by side). The push now removes
non-current installers over ssh first, so the remote feed mirrors the local
one whether the transfer uses rsync or tar.
2026-09-20 21:39:43 -05:00
JakeBreath 3183a3bece Prune old desktop builds from release/ on every build
build_desktop.sh now reads the version first and removes anything in
desktop/release/ that is not that version (plus the regenerated unpacked
trees), so a version bump never leaves old installers lying around — the
same rule push_desktop.sh applies to the feed.
2026-09-20 21:16:56 -05:00
JakeBreath 041a9d4471 Keep desktop builds and the feed to one version
Both scripts now read the version from desktop/package.json: the build
report and checksums only cover the current version's artifacts, the feed
copy ignores older files, and publishing prunes previous installers from
the feed (latest*.yml only ever points at the current one).
2026-09-20 21:11:19 -05:00
JakeBreath d84fdd0e98 Bump the desktop app to 0.1.1
The icon set, e621 referrer fix and setup-screen corrections shipped after
0.1.0, and the updater compares versions, so installed 0.1.0 builds would
never have seen them.
2026-09-20 21:09:42 -05:00
JakeBreath c992a63b8f Fix e621 images and the setup screen in the desktop shell
e621's CDN answers cross-site image loads that carry no Referer with a 403
(Chromium sends none from a custom-scheme page, then blocks the response as
ORB), so images never appeared in the desktop app. The main process now
attaches an e621 referrer to requests for its hosts.

The shell also answers /api, /admin, /static and /health with a 404 JSON
instead of the SPA fallback — that fallback made the setup screen's empty-URL
connection test report "Connected" against the shell itself. The setup screen
is now desktop-aware (no same-origin option, no "Use this server", clearer
copy), and the smoke test runs against a throwaway profile and covers both
regressions.
2026-09-20 21:08:56 -05:00
JakeBreath bd2417aff8 Keep a broken Redis from 500ing the whole API
Redis backs the DRF throttles, and the stock RedisCache raises inside the
throttle check when Redis is unreachable or refusing writes (a failed RDB
snapshot disables writes by default) — turning a cache problem into a
blanket 500, which is exactly what took prod down. ResilientRedisCache
treats backend failures as cache misses, logs the first one per worker, and
lets rate limits degrade until Redis is back.
2026-09-20 21:08:38 -05:00
JakeBreath 70d7a4f606 Default push_desktop.sh to the jakerasp deploy checkout
The remote feed copy now happens by default (overridable with
J621_DESKTOP_FEED_HOST or --host, skippable with --local), so a release is
one command on the build machine.
2026-09-20 20:25:37 -05:00
JakeBreath f4b534aa09 Let push_desktop.sh copy the feed to a remote deploy checkout
--host user@server:/path syncs deploy/data/desktop to the same path on the
server with rsync, falling back to tar over ssh when the server has no
rsync. A failed transfer now exits non-zero instead of claiming the feed is
live.
2026-09-20 20:22:59 -05:00
JakeBreath 1c6735cde8 Include the desktop origin in generated CORS settings
The backend only allows app://j621 through CORS_ALLOWED_ORIGINS, so
gen_env.sh now always writes that origin (plus the split-deploy frontend
when one is given) and --update keeps hand-added origins instead of
overwriting the list.
2026-09-20 19:44:56 -05:00
JakeBreath 7f64c6b635 Ship a full icon set so KDE resolves the launcher icon
The Linux packages only installed a single 1024x1024 hicolor PNG, and
Plasma's icon lookup returns nothing for a lone oversized icon — confirmed
with kiconfinder6 under Papirus-Dark. Generate 16-1024 px PNGs from the
favicon and point electron-builder at the directory so every standard
hicolor size is installed.
2026-09-20 19:39:19 -05:00
JakeBreath 69f324ead7 Add a desktop build script for manual releases
deploy/build_desktop.sh builds the Arch, Debian and Windows packages into
desktop/release/ without publishing anything, lists what it produced with
sizes and SHA-256 sums for release notes, and prints the suggested Gitea
tag. Installing locally, handing the files out and attaching them to a
Gitea release all stay manual; push_desktop.sh remains the update-feed
publisher.
2026-09-20 18:08:19 -05:00
JakeBreath e839c84bf0 Make the desktop smoke test mode-aware
The dev path (J621_DEV_SERVER) has the Vite origin and no /setup screen, so
the expectations now follow the mode instead of reporting false failures.
2026-09-20 17:34:01 -05:00
JakeBreath 7c39383655 Wire desktop updates through a generic feed
electron-builder now publishes latest-linux.yml / latest.yml and embeds
app-update.yml plus package-type, so electron-updater runs pacman -U or
dpkg -i through pkexec for packages and updates the per-user NSIS install
without elevation. The app checks only when asked (menu item), asks before
downloading and before installing, and J621_UPDATE_URL overrides the feed
for tests or forks.

deploy/push_desktop.sh builds and publishes the artifacts to
deploy/data/desktop, which the frontend nginx mounts read-only at
/desktop/. Verified detection and up-to-date handling against a local feed
with the packaged Arch build.
2026-09-20 17:32:41 -05:00
JakeBreath 84ec431441 Package the desktop app for Arch, Debian and Windows
electron-builder produces j621-desktop_*_amd64.deb,
j621-desktop-*.pkg.tar.zst (zstd, lean Arch dependencies instead of the
Electron 2 era default set) and a per-user NSIS installer cross-built with
wine. The launcher entry and Electron's desktopName now agree on
io.j621.desktop so window association works, and package metadata points
at the repository homepage and the non-commercial licence.
2026-09-20 17:18:53 -05:00
JakeBreath cf129714be Add an Electron desktop shell for the SPA
desktop/ serves the normal frontend build over a privileged app://j621
scheme, so localStorage, OPFS, Web Workers, WebCodecs and history routing
behave exactly like Chrome. Development points at the Vite dev server;
`npm run smoke` runs headless Electron and checks the bundled app.

External links open in the system browser, and download navigations
(?download=1 or media URLs) are rerouted through webContents.downloadURL,
since preventing them cancels the download. The setup screen names the
shell's origin when the connection test fails, and the deploy docs list
app://j621 for CORS_ALLOWED_ORIGINS.
2026-09-20 17:07:22 -05:00
JakeBreath 3a07481dfc Wait for all uploads, then batch MD5 -> visual -> IQDB with bulk links
Batching must not start while files are still being uploaded, and the MD5
phase must move a whole chunk at once instead of one resolve per file:

- the upload queue drains completely first (failed uploads included) before
  any matching starts;
- phase 1 asks e621 for every md5 (75 per posts.json request), builds the
  md5 -> post map from the response, and sends the matches to the new
  POST /api/uploads/link-bulk/ action, so a whole 75-file chunk moves into
  Indexed in a single board update;
- link-bulk indexes the staged file directly when the post's MD5 matches
  (identical bytes), so there is no per-file download round trip;
- phase 2 runs local visual similarity for whatever stayed pending, phase 3
  the IQDB queue.

Verified end to end with real e621 files: one md5 query for the batch, one
link-bulk call, both matching files flipped to Indexed together, then the
visual and IQDB phases. 23 library tests green (link-bulk, visual phase,
deferred visual matching).
2026-09-19 11:10:24 -05:00
JakeBreath e2bf1c457f Make upload processing phase-based: MD5 -> visual -> IQDB over the batch
Uploads were doing md5 + local visual matching inside the upload request
(backend create) while the frontend later ran its own e621 MD5 pass, so the
pipeline looked interleaved per file. Now every step is a phase applied to
the whole batch in order:

1. upload (fast: md5 + exact-duplicate check only),
2. e621 MD5 lookup, 75 md5: metatags per posts.json request,
3. local visual similarity, one file at a time via the new
   POST /api/uploads/<id>/visual-match/ action,
4. IQDB through the existing serial queue.

The board shows the active phase with its own progress bar (e621 MD5 in
peach, visual in lavender, IQDB in teal) and every step updates the staged
list as it lands. Verified from a headless run: one batched posts.json
request for 10 files, then 10 visual-match calls, then IQDB.
2026-09-19 10:25:57 -05:00
JakeBreath e62d7af42f Live-updating upload board, self-clearing tiles, original MD5 batch size
- MD5 auto-match now sends 75 md5: metatags per posts.json query, the same
  batch size the original J621-Django app used (was 20); the limit cap no
  longer truncates batches.
- every settled upload is upserted into the staged list right away, so the
  Pending / Visual Similarity / Auto-uploaded columns move as files land
  instead of waiting for the whole batch (auto-matched resolves and IQDB
  results use the same path).
- finished upload tiles fade out and remove themselves ~2s after completing;
  failures stay until cleared. Batch counters are tracked separately from the
  visible tiles so the header and progress bar stay accurate as tiles vanish.
- AGENTS.md now points at the original Django app for reference behavior.

Verified live with a headless upload run: the columns showed the new files
immediately and the 8 tiles were gone ~3s after finishing.
2026-09-19 01:18:50 -05:00
JakeBreath 9641862515 Back off from e621 rate limits and pace requests more conservatively
e621 intermittently answers 429 to the IQDB endpoint; browsers hide that
status behind CORS ('Access-Control-Allow-Origin missing'), so the SPA
cannot read it. Treat every network-level failure as a possible rate limit
and pause all e621 traffic for a minute. The cooldown is shared through
localStorage so extra tabs respect it, requests are spaced 1.5s apart
instead of 1s, user-cancelled requests do not trigger a cooldown, and the
upload queue waits the cooldown out with a countdown instead of looking
stuck.

Server side: the per-process e621 gap goes from 0.5s to 1s so two gunicorn
workers cannot together exceed e621's 2/s hard limit.
2026-09-19 00:37:32 -05:00
JakeBreath b96c311235 Fix cross-origin staging URLs, e621 UA format and IQDB queue stalls
The dev Vite proxy rewrites the request Host to 127.0.0.1:8000, so the
backend's absolute signed file URLs pointed at a different origin than the
SPA (localhost:5173). Images tolerated it, but the auth'd fetch that reads
the staging blob for IQDB was blocked ('Cross-Origin Request Blocked') and
every similarity check died before reaching e621.

apiUrl() now keeps API-built absolute URLs on the page's origin whenever the
SPA is in same-origin mode (dev proxy, deploy nginx) and leaves them
absolute when an explicit backend URL is configured. All consumers use it:
staging previews and the bulk modal, library cards, optimizer (range sniff +
worker), IQDB card, delete page, similar page.

e621 identification now follows the documented 'App/version (developer)'
form: server-side requests send 'J621/<hash> (JakeBreath)' and the browser
_client gets the same string, with the hash baked into the frontend image
(GIT_HASH build arg; guarded at runtime so the dev server still works).

IQDB stalls: requests now time out after 20s (a hung fetch used to block the
serialized e621 queue forever), and all checks run through one serial drain
so repeated 'Check similarity' clicks can no longer start overlapping runs
that re-download the same staging blobs. Auth/rate-limit/timeout/network
failures stop the queue with the reason and a retry button instead of
grinding through the rest.

Verified live: staged file URL is same-origin through the proxy and fetches
200 through it.
2026-09-19 00:30:42 -05:00
JakeBreath 2f613b7027 Stop Firefox from squashing upload tiles and show IQDB queue progress
The upload grid was its own scroll container (max-height + overflow on the
same element). Firefox sizes auto grid rows to min-content in that setup,
so every tile collapsed to its footer height and the image was clipped to a
wide strip; Chromium sizes them to max-content, which is why this only
showed up in the user's browser. auto-rows-max pins rows to max-content in
both; verified with headless Firefox screenshots and Chromium measurements
(60 tiles render 152x194 each, 1845px of content in a 639px scroller).
The box is now 70vh so it behaves as a proper fixed gallery area with its
own scrollbar instead of shrinking with the item count.

IQDB progress: the page header now shows a live 'Checking IQDB — x/y'
counter with a bar while background checks run, so the queue is visible
without opening the per-file modal (which keeps its spinner, candidates
and per-file errors).
2026-09-19 00:06:05 -05:00
JakeBreath bc7494e7be Show IQDB checks in the metadata modal and run them for visual matches
The modal held a snapshot of the staged upload, so IQDB results that landed
from the background check queue never appeared until it was closed and
reopened — the only hint a check was running was the e621 request history.
It now follows the live uploads query, so candidates, progress and errors
show up in place.

Related gaps fixed along the way:
- files flagged by the local visual-similarity check were skipped by the
  IQDB pass entirely (only 'pending' files were checked), so their modal
  could only ever show 'already in your library'; unresolved files of both
  statuses are now checked, and the check button shows on visual-match
  cards too;
- a check with no candidates posted nothing, leaving 'never checked' and
  'checked, no match' indistinguishable; results are stored even when
  empty and the modal now says which one it is;
- per-file failures surface in the modal instead of being swallowed, the
  modal shows a spinner while the query runs and a check now/re-check
  button, and auto-runs skip files already checked (and videos, since IQDB
  is image-only).

Backend production code unchanged; tests pin the empty-result recording
(18 library tests, full suite 56 green).
2026-09-18 23:42:45 -05:00
JakeBreath b076903ecd Show live progress while the bulk rating tool moves files
A 200-file bulk move is one long server-side copy+index chain per file, so
the old single request sat on a spinner the whole time (and got uncomfortably
close to the 120s proxy timeout). The modal now resolves the selection in
chunks of 8:

- footer switches to a progress bar with 'n moved / done / total / percent'
  while running, and the header explains that files are being indexed;
- the rating pills, selection actions, grid and close button are locked
  while it runs so progress can't be lost by accident;
- failures are collected with their filenames, the modal stays open for a
  summary, and 'Retry failed' re-selects only the files that are still
  pending; a clean run still auto-closes with a toast.
2026-09-18 23:09:42 -05:00
JakeBreath a761def65e Scrollable upload grid and bulk rating for the pending backlog
Upload board:
- the tile grid no longer re-sorts itself as files finish (that reshuffled
  the list under the cursor); it keeps insertion order, uses auto-fill tiles
  of ~150px so they hold a readable size, scrolls inside a 60vh area and no
  longer chains the page scroll (overscroll-contain);
- the files currently in flight are pinned in a small live strip above the
  grid (name, percent, bar) so progress stays visible while the grid is
  scrolled with hundreds of tiles.

Bulk rating: a 'bulk rate' button in the Pending & Unmatched header opens a
large modal with Safe/Questionable/Explicit pills, a tickable thumbnail grid
(Select all / Clear) and one confirm that moves every selected upload into
the library with that rating. Backed by POST /api/uploads/resolve-bulk/
(temp_ids + rating, own rows only): each staged file is resolved as a custom
entry (keeps its staged tags/notes), and already-completed or foreign ids are
reported per entry instead of failing the whole batch. Built for the
358-file backlog.

Tests: 4 bulk-resolve tests (resolution with the rating, input validation,
foreign ids untouched, mixed completed+pending) — full backend suite 53
green. Verified live end to end: staged a file, bulk-resolved it as 'q', saw
J-96 created with that rating, then removed the item, temp row and test
token.
2026-09-18 21:50:22 -05:00
JakeBreath 39307cb141 Unpaginate staged uploads and make big upload batches visible
The upload board partitions /api/uploads/ into Pending / Visual similarity /
Auto-uploaded, but the endpoint was paginated at 48 — a 69-file batch
silently lost 21 entries, and the similarity sweep (which reads the same
list back after uploading) only ever saw the first page. The staged-upload
list is now unpaginated: it is a transient per-user set, still limited to
the caller's rows and the uploader role. The page takes a plain array.

Watching progress with dozens of files was also poor:
- the queue uploads three files at a time instead of strictly one at a time;
- the Uploads section now shows a batch bar and 'n/m uploaded · x%' next to
  the count, so the overall progress never scrolls out of sight;
- entries are ordered active-first (uploading, queued, failed, done) so the
  file being uploaded is always at the top of the grid;
- tiles are larger (4 columns at lg instead of 5);
- the header reads 'Uploading n/m…' and 'Checking n file(s) against IQDB…'
  instead of a bare spinner.

Tests: staged-upload list unpaginated past 48, per-user, uploader-only
(3 new; full suite 49 green). Live-checked the bare-array response.
2026-09-18 19:58:17 -05:00
JakeBreath 1adb761c8d Fix following past 48 entries and make the e621 page size configurable
Follow lists were paginated at the API default of 48, but the SPA treats
them as complete sets: the tag/pool toggles read their state from page one
(so the 49th follow looked unfollowed and its spinner waited for a page that
could never contain it) and the Followed page rendered only 48 cards while
showing that as the count. Both follow endpoints are now unpaginated — they
are per-user sets and still restricted to the caller's rows — and the three
consumers take plain arrays.

Post visibility: the old J621-Django online view fetched limit=320 (e621's
maximum) while ours hard-coded 48, and fetchPostsByIds capped id batches at
100. The Online browser now has a 'Posts per page' setting (48/100/200/320)
in its sidebar, mirrored in Account -> Browsing preferences, stored per user
as e621_per_page and also used for pool loading; the id-batch cap is raised
to 320.

Tests: follow list shape/isolation (4) and preference validation/merge (3)
added; the full backend suite is 46 green. Live-checked the array response
shape and the preference bounds (200 accepted, 500 rejected).
2026-09-18 19:06:18 -05:00
JakeBreath d16a77907a Keep the online detail's download state on its own post
Two stale-state bugs came from react-router reusing the detail component
between posts (parent/child links hit the same /detail/<id> route):

- the previously viewed post's download panel kept rendering, so a freshly
  opened post could claim 'Downloaded to the library J-xx'. The task view is
  now gated on the task's post_id as well, and DetailPage keys the detail
  views per item — component state (download panel, delete confirmation,
  optimizer modal) cannot survive a post change any more.
- 'Your last download for this post finished' appeared on every revisit. It
  now only shows when this visit actually saw the download running (derived
  state, set during render), which still reports a re-attached download
  finishing while staying quiet on later visits; the 'In library' button
  remains the persistent indicator.

Library detail gets the same per-item key, so its delete confirmation and
optimizer modal reset between items too.

tsc, oxlint and the build are clean (the derived-state pattern was chosen
over a ref read in render / setState-in-effect, both flagged by the linter).
2026-09-18 17:52:50 -05:00
JakeBreath a17dd5a4ef Keep secrets and runtime data out of the Docker images
The build context is the repository root and there was no .dockerignore, so
'COPY backend/ ./' swept backend/venv (327 MB), backend/media (the actual
library, 224 MB), backend/logs, backend/staticfiles and backend/.env
(SECRET_KEY plus the database password) into the backend image: 1.43 GB per
architecture, including secrets headed for the registry. The frontend build
stage also copied the host's node_modules over the fresh install.

- Root .dockerignore excludes .git, virtualenvs, __pycache__, db.sqlite3,
  logs/staticfiles, .env files, media/, node_modules, dist and the deploy
  runtime state (data/, tailscale-state/).
- The backend Dockerfile now asserts the context is clean (.env, venv,
  media/library, db.sqlite3 all absent) before collectstatic, so a missing
  ignore file fails the build instead of leaking.
- Rebuilt: backend 1.43 GB -> 876 MB ('COPY backend/' is now 268 kB),
  frontend stays at 65 MB. Verified by booting the compose stack with the
  new image: migrations applied, /health ok, no .env or venv inside, and
  /app/media is the mounted (empty) volume; the scheduler runs too.
- Removed the stale local images that still contained the library and the
  dev .env.
2026-09-18 16:04:55 -05:00
JakeBreath 1170e6e9c1 Updates 2026-09-18 14:11:10 -05:00
JakeBreath b3ceac52ed fish greeting: win over distro configs that define fish_greeting inline
On CachyOS (and OMF-style setups) config.fish sources a distro file that
defines fish_greeting while the shell starts. A function defined that way
beats autoloading from functions/, so the installed greeting never ran.

install.fish now appends a guarded block to ~/.config/fish/config.fish that
sources the greeting after everything else (idempotent via a marker, skipped
with --no-config), and the README documents the symptom, the check
(functions --details fish_greeting) and the manual one-liner.

Verified in a sandbox HOME that reproduces the CachyOS setup: before the
install fish resolves the distro file and prints its message, after it
resolves ~/.config/fish/functions/fish_greeting.fish and runs ours; a second
install leaves a single block. Applied to this machine's real config as
well, where fish_greeting now resolves to the user function and the
__j621_fetch_random helper is loaded.
2026-09-18 13:48:02 -05:00
JakeBreath bcff184a64 Make extras/fish_greeting/install.fish executable
It has a '#!/usr/bin/env fish' shebang but was committed as mode 644, so
./install.fish answered 'Permission denied' on a fresh clone. Mode is now
100755 like the deploy scripts; 'fish install.fish' worked either way.
2026-09-18 13:43:12 -05:00
JakeBreath 770b1e5ee6 Scoped API tokens for the random endpoint, with a management page
Backend: a GreetingToken model stores only a SHA-256 hash of a j621r_…
key (shown once at creation) plus label, prefix, created/last-used. A
dedicated GreetingTokenAuthentication understands the usual
'Authorization: Token …' header but is registered only on RandomItemView
(alongside the normal token auth), so a greeting token authenticates
/api/random/ and is rejected with 401 everywhere else — exactly the scope
shell greetings need. Endpoints: GET/POST /api/auth/greeting-tokens/ and
DELETE /api/auth/greeting-tokens/{id}/ (own tokens only; the list never
returns keys or hashes).

Frontend: /tokens page (Account → Shell tokens card, command palette entry)
lists tokens with label, prefix, created/last-used and revoke (shared
confirm dialog). Creating one shows the key with Copy and 'Copy for fish'
buttons plus a pointer to extras/fish_greeting.

Tests: apps/accounts/tests/test_greeting_tokens.py — 9 tests covering
create-once semantics and hashing, hidden keys in listings, the scope
guarantee (random 200 with a signed URL; 401 on files, storage, me, tags
cloud, delete and the token list itself), unknown/revoked keys, cross-user
revocation, last-used tracking and label limits.

Verified live: created a token, rolled /random (signed URL), got 401 from
four other endpoints, saw the list omit secrets, revoked it (204) and the
same key then 401'd on /random. Full suite: 39 tests green.
2026-09-18 13:37:29 -05:00
JakeBreath 2d9493d9fe fish_greeting installer asks for the API origin and probes the backend
install.fish now:
- asks for the API origin (default https://j621.rainbow-herring.ts.net) and
  an optional token when run interactively, or takes --url/--token;
- writes ~/.config/j621Greeting/config.fish with mode 600 (it may hold a
  token), keeps an existing config unless --force/--url is given, and
  --no-prompt runs fully unattended;
- verifies the setup: /health must answer 'ok' and a fastfetch random roll is
  attempted, reporting the API's own message when it finds nothing; exits
  non-zero when the backend is unreachable so scripts notice.

README documents the prompts/flags, the chmod 600 config, and that the
greeting is designed to run without a token (guest mode: unsigned links,
guest-visible items only) with a token adding signed links and hidden items.

Tested with fish 4.9.3 in throwaway HOME dirs: flag-driven install with a
token, interactive install with a piped origin and no token, re-run keeping
the existing config, and an unreachable backend exiting 1.
2026-09-18 13:22:55 -05:00
JakeBreath aee29de34a Port the fish_greeting shell greeting to the new API
The original script (J621-Django/extras/fish_greeting system) downloaded
image bytes from /random/?rating=X and read the X-File-* headers. The new
API answers with JSON and a signed link, so the greeting now:

- asks /api/random/?fastfetch=1&rating=<mode> for JSON;
- parses url/j_id/filename/md5 with jq, python3, or a grep/sed fallback;
- downloads the signed link and keeps the original display path (fastfetch
  kitty/kitty-icat logos, gifsicle preprocessing for GIFs, recursion guard,
  logging with rotation, greeting_mode 0/1/2 = NSFW/SFW/Questionable);
- is configured through ~/.config/j621Greeting/config.fish or universal
  variables (J621_BASE, J621_WEB, J621_TOKEN, J621_FASTFETCH_CONFIG) instead
  of a hardcoded host, and prints the /detail/<J-ID> link on J621_WEB;
- reports the API's own 404 message when a rating has no images, and keeps
  curl quiet so failures do not spill into the greeting.

extras/fish_greeting/ contains the function, an install.fish (copies it into
the fish functions dir, creates the config once, checks dependencies) and a
README with the old-vs-new table and troubleshooting. The existing
gm-switch helper and .desktop launchers keep working (same mode file).

Tested with fish 4.9.3: syntax check on every file, guest and token runs
against the dev API (unsigned vs signed URLs), the empty-rating and
unreachable-API paths, and the grep/sed fallback with jq and python3
unavailable.
2026-09-18 13:14:41 -05:00