Commit Graph
138 Commits
Author SHA1 Message Date
JakeBreath aee29de34a Port the fish_greeting shell greeting to the new API
The original script (J621-Django/extras/fish_greeting system) downloaded
image bytes from /random/?rating=X and read the X-File-* headers. The new
API answers with JSON and a signed link, so the greeting now:

- asks /api/random/?fastfetch=1&rating=<mode> for JSON;
- parses url/j_id/filename/md5 with jq, python3, or a grep/sed fallback;
- downloads the signed link and keeps the original display path (fastfetch
  kitty/kitty-icat logos, gifsicle preprocessing for GIFs, recursion guard,
  logging with rotation, greeting_mode 0/1/2 = NSFW/SFW/Questionable);
- is configured through ~/.config/j621Greeting/config.fish or universal
  variables (J621_BASE, J621_WEB, J621_TOKEN, J621_FASTFETCH_CONFIG) instead
  of a hardcoded host, and prints the /detail/<J-ID> link on J621_WEB;
- reports the API's own 404 message when a rating has no images, and keeps
  curl quiet so failures do not spill into the greeting.

extras/fish_greeting/ contains the function, an install.fish (copies it into
the fish functions dir, creates the config once, checks dependencies) and a
README with the old-vs-new table and troubleshooting. The existing
gm-switch helper and .desktop launchers keep working (same mode file).

Tested with fish 4.9.3: syntax check on every file, guest and token runs
against the dev API (unsigned vs signed URLs), the empty-rating and
unreachable-API paths, and the grep/sed fallback with jq and python3
unavailable.
2026-09-18 13:14:41 -05:00
JakeBreath f8667c1037 Add a Random image endpoint and SPA page (with fastfetch mode)
Backend: GET /api/random/ (aliases /random and /random/) returns a random
library image with:
- rating=s,q,e filtering (comma separated, default any);
- fastfetch mode (?fastfetch=1 or any User-Agent containing "fastfetch")
  that only considers png/jpg/gif - what terminal viewers can show;
- JSON with j_id, filename, extension, rating, size, e621 id plus absolute
  url/download_url/thumbnail_url. Authenticated callers get signed URLs so
  fastfetch and image viewers can load them without headers; guests get
  unsigned URLs and never receive hidden_from_guests items.

Tests: apps/library/tests/test_random.py (8 tests) covering the response
contract, guest signatures, image-only default, the fastfetch format
restriction (flag and User-Agent), rating filters, guest visibility and the
short alias.

Frontend: /random page with rating pills, R to roll, Open/Download and a
library link, plus navigation and command palette entries; needs a backend,
hidden in local mode.

nginx: /random negotiates on Accept so browsers keep getting the SPA while
scripts get the JSON (verified with the proxy and frontend containers).

Also fixes a regression from the SSRF change: the guest download proxy
still referenced the removed 'parsed' variable on its success path, so
every proxied download would have 500'd. Redirect hops are now covered by
tests with a mocked requests.get.
2026-09-18 13:06:36 -05:00
JakeBreath f25526782c Run the periodic commands in a scheduler service (no host cron)
Adds deploy/scheduler-entrypoint.sh to the backend image (entrypoint
j621-scheduler) and a scheduler service to the four composes that have a
backend. It waits until the database and migrations are ready, runs every
job once, then keeps to the intervals:

  sync_followed_tags + sync_followed_pools   every 30 min (J621_SYNC_EVERY)
  cleanup_similarity                         hourly      (J621_CLEAN_EVERY)
  refresh_guest_blacklist                    daily       (J621_BLACKLIST_EVERY)

It shares the backend image, media volume and env file, so commands see
the same library and database; failures are logged and retried next
interval. Output goes to docker compose logs scheduler; the frontend-only
composes have no backend and therefore no scheduler.

Verified against a real stack: the scheduler waited for migrations, ran all
four commands on start (follow syncs as anonymous, similarity cleanup, and
a guest blacklist refresh that pulled the real 14-tag list from e621), and
kept looping. All six composes still validate.
2026-09-18 12:51:17 -05:00
JakeBreath 93cce6b9fd deploy/gen_env.sh: generate .env with openssl secrets
Builds deploy/.env from .env.example, generating SECRET_KEY and both
database passwords with openssl (base64/hex only, so nothing needs quoting
in the env file or the compose parser). Derives TS_HOSTNAME and
ALLOWED_HOSTS from the tailnet hostname, optionally sets
CORS_ALLOWED_ORIGINS/CSRF_TRUSTED_ORIGINS for split deployments, forces
DEBUG=False and writes the file with mode 600.

Modes: --no-prompt (defaults only), --update (refresh hostnames/auth key
while keeping the existing secrets, reading the stored FQDN from
ALLOWED_HOSTS), --force (rotate everything, with the SECRET_KEY warning in
the docs). Refuses to overwrite an existing file otherwise.

Verified: all modes, updated FQDN preservation, mode 600, and
docker compose config accepting the generated file.
2026-09-18 12:46:56 -05:00
JakeBreath 30b1a1a4b0 Tailnet-only (Serve, no Funnel) compose variants
Three more composes — compose.tailnet.yml, compose.tailnet.frontend.yml,
compose.tailnet.backend.yml — mirror the funnel set exactly but mount
serve.default/frontend/backend.tailnet.json, which drop AllowFunnel. The
sidecar still registers and serves HTTPS with a tailnet certificate, but
nothing is exposed publicly; Serve also needs no ACL change.

Project names carry a -tailnet suffix so both sets can coexist, and the
README explains that each set needs its own data directory (or host), plus
how to switch a host between funnel and tailnet by starting the other file
with the same .env.

Verified: all six composes validate with docker compose config, and the
six serve configs split cleanly into funnel (AllowFunnel present) and
tailnet-only (absent).
2026-09-18 11:21:48 -05:00
JakeBreath ce016fb221 AGENTS.md: how to run and restart the dev stack
Ports, start/restart commands, the kill-by-PID gotcha and the test command
so a fresh session can bring the environment back without guessing.
2026-09-18 00:53:31 -05:00
JakeBreath 8ebda6ab20 Docker deployment (2 images, 3 composes, Tailscale funnels) + committed security suite
Test suite (the manual audit harness, now a real test):
- backend/apps/core/tests/test_security.py: 20 transactional tests across
  guest visibility, object ownership, staged-upload/similarity privacy,
  staff role boundaries, deletion rules, encrypted credentials, throttling
  and the download allowlist. Uses temp media folders and clears cache.
  Needs a one-time GRANT on test_j621 (documented in the module + README).

Images (deploy/J621-Frontend, deploy/J621-Backend, repo root as context):
- Frontend: node build -> static nginx with SPA fallback, asset caching and
  an internal health endpoint.
- Backend: gunicorn + whitenoise (admin static collected at build), ffmpeg
  for video thumbnails, migrations applied on start, GIT_HASH build arg so
  the shell's version pill shows the commit.

Composes (distinct project names so they coexist with the dev stack):
- compose.yml (both), compose.frontend.yml, compose.backend.yml.
- A shared nginx proxy service is the only entry point (no host nginx, no
  published host ports): /api,/admin,/static,/health -> backend, everything
  else -> SPA; both upstreams resolve at request time so one config serves
  all variants.
- A Tailscale sidecar per compose shares the nginx network namespace;
  serve.default/frontend/backend.json use funnel ports 443 and 8443 only
  (10000 is the remaining allowance) with ${TS_CERT_DOMAIN} substitution.

Registry: push_frontend.sh / push_backend.sh / push_all.sh build multi-arch
images and push :latest + :<sha> to the Gitea registry, following the
existing Packs-site pattern.

Docs: deploy/README.md + .env.example, ROADMAP section 6 updated,
AGENTS.md deployment and test notes.

Verified: all three composes validate, both nginx configs pass nginx -t,
both images build, the combined stack boots against real MariaDB/Redis
(migrations applied, /health ok, whitenoise serving admin static, env=prod,
git hash baked in), the proxy serves the SPA and routes /api, and
manage.py test apps.core.tests passes 20/20.
2026-09-18 00:51:29 -05:00
JakeBreath f86eccf9a3 Security fixes: SSRF, staff role escalation, SPA-only gating, throttling, encrypted keys
Findings from the audit (50-check harness across guest/user/uploader/staff/
admin) and their fixes:

- SSRF: 'Download to Library' and the staged-upload resolve path fetched
  any http(s) URL. services.validate_remote_url now enforces the e621
  media allowlist and open_remote re-validates every redirect hop; the
  download-task create endpoint and the guest proxy use them, so internal
  addresses (127.0.0.1, LAN, metadata) are rejected with 400.
- Privilege escalation: staff could promote users to staff and demote
  other staff. Role changes across the staff boundary now require an
  admin, matching the account-deletion rules; the Users page hides what
  the backend would refuse.
- SPA-only gating: /api/storage/ and /api/duplicates/* were readable by
  any authenticated account (absolute paths, duplicate groups) while the
  SPA only shows them to uploaders. They now require CanUpload.
- Throttling (REST_FRAMEWORK, env-overridable, counted in Redis):
  anon 120/min, user 600/min, login 5/min, register 20/hour, guest e621
  proxy 60/hour. Login now goes through a throttled view.
- e621 API keys are encrypted at rest with a Fernet key derived from
  SECRET_KEY (apps/accounts/crypto.py); a data migration encrypts existing
  rows and the column widens first. Reads decrypt transparently, legacy
  plaintext still works, and a changed SECRET_KEY reads as 'not
  configured' instead of leaking. Rotating SECRET_KEY now invalidates
  stored keys as well as signed media URLs.
- Hardening: the server refuses to start with DEBUG=False while SECRET_KEY
  is still the development default.

Verified: corrected harness 50/50 (guest visibility, IDOR, signed-URL
tamper/expiry, staged-upload/similarity privacy, role matrix, SSRF),
login throttles at the 6th attempt with 429, anon polling unaffected, the
guest proxy still reaches allowlisted hosts, live e621 auth works with the
decrypted key, and DB rows hold only ciphertext.
2026-09-18 00:21:14 -05:00
JakeBreath a904abdf20 Run the SPA without a backend (local mode)
The app can now operate backend-agnostically: a production build still
asks on first start, but /setup also offers 'Continue without a backend'
(stored as the sentinel 'none'), and the shell adapts:

- Local mode shows only the e621-facing pages: Online (search, post view,
  favorites, blacklist editor, direct downloads) and Pools. Library,
  uploads, duplicates, stats, users, follows and similarity are hidden
  from the nav and palette and render a 'backend needed' state when
  reached directly; /detail/<e621 id> still works while /detail/J-x asks
  for a backend.
- e621 credentials are stored in this browser (j621.e621) and the
  Account page becomes a credentials-only screen; the store reads/writes
  locally instead of /api/auth/e621/.
- The header replaces the status pill and login/user area with an e621
  credentials button and a 'Setup Backend' button; the footer shows
  'Local mode — e621 features only' with the same entry point.
- In-library lookups (badges/browse markers) are skipped without a
  backend; 'Download to client' links straight to the e621 file instead
  of the backend proxy; follow buttons and palette follow toggles are
  hidden; api() fails fast with a clear message if something slips
  through.

Mode logic lives in lib/backend.ts (URL / '' same-origin / 'none') with
its matrix verified in Node; tsc, oxlint and the build are clean.
2026-09-18 00:00:48 -05:00
JakeBreath 3bd5bc73dd Fix the order:hot default: seed the toggle and clear stale defaults
Two bugs made the toggle look broken even though the preference was
stored correctly (JakeBreathild had online_hot_default false):

- The Account card never seeded online_hot_default into its form state,
  so the checkbox always rendered checked via the '?? true' fallback.
  It now starts from the saved value.
- Turning the toggle off did not change Online when the URL still
  carried tags=order:hot (e.g. Ctrl+Shift+R reloading the old URL).
  order:hot on its own is the default, not a deliberate search, so it is
  now removed when the account has the toggle off; 'order:hot canine' or
  any other search is still left untouched. The decision moved into
  hotDefault.ts (set-hot / clear-hot / keep) with the matrix verified in
  Node.
2026-09-17 23:49:19 -05:00
JakeBreath dc79349d9e Online defaults to order:hot, with an Account toggle
New online_hot_default preference (on by default, so guests and accounts
that never saved preferences get it): opening /online without a search
replaces the URL with ?tags=order:hot — e621's metatag for the order the
Hot page uses — so it is visible in the search field and shareable.
Existing searches are never touched: they live in the URL, so a refresh
or back/forward keeps them, while a fresh visit (nav pill, first time,
after a long time away) gets the hot default again. Turning the toggle
off opens Online on the site-wide newest posts as before.

The toggle sits in Account -> Browsing preferences and saves with the
rest of the settings; the backend validates the new boolean
(400 for non-boolean input).
2026-09-17 23:46:14 -05:00
JakeBreath 2df001b477 Let staff delete accounts from the Users page
DELETE /api/users/{id}/ with guards: nobody deletes the account they are
signed in as (400); staff can delete regular/uploader accounts only,
while admins can also delete staff and admins (403 for staff targets
otherwise, and the last admin can never be deleted). Deleting a user
removes their follows, tokens and staged uploads — including the staged
files on disk — while library items survive and simply lose their owner
(uploaded_by is SET_NULL), as does download/match/similarity history.

The Users page gets a per-row delete button behind the shared confirm
dialog, hidden wherever the backend would refuse (own row, or a
staff/admin target when the actor is not an admin).

Verified against the dev server: staff 204 for a regular account, 400
for self, 403 for an admin; admin 204; a plain account gets 403. After
deleting a user that owned J-81 and had a staged file, the file was gone
and J-81 survived with a null owner.
2026-09-17 23:35:56 -05:00
JakeBreath 99f617d296 Footer storage/backend display and a staff role that actually grants staff
Footer:
- Left is now 'Backend Storage:' with a capacity bar (blue, peach at 80%,
  red at 95% per DESIGN.md) and a used/total/free tooltip; the watched
  folder path is no longer printed. /api/status/ returns a compact storage
  summary instead of the path (the full storage page still shows paths to
  authenticated users).
- Centre shows the backend API origin (empty = same origin). Staff get a
  link to /setup to point the browser elsewhere; everyone else sees it as
  plain text. The Account 'Backend connection' card is gone — this is
  installation plumbing, not a per-user setting.
- Design spec updated to match.

Staff role:
- The custom role did nothing on several endpoints that only accepted
  Django's is_staff/is_superuser. One canonical check now exists:
  User.is_app_staff (superuser, Django staff, or the staff role), used by
  the stats/users APIs, item object permissions, can_delete, upload/
  similarity/download/match querysets, and the management commands
  (which also pick staff-role accounts for e621 sync/match and file
  ownership).

Verified with a role-only staff account (is_staff/is_superuser false):
stats/users 200, all 32 downloads + 2 scans visible, others' items
editable; the same account as role=user gets 403 for all of those.
2026-09-17 23:24:24 -05:00
JakeBreath a93500154c Ask where the backend lives on first start (runtime setup)
Replaces the build-time VITE_API_BASE knob with a runtime setup screen so
one build works same-origin and cross-origin:

- frontend/src/lib/backend.ts stores the API origin in localStorage
  (empty = same origin). DEFAULT_BACKEND_URL is the clearly marked,
  easily edited prefilled default — the matrix.org equivalent; set it to
  your public API origin.
- Production builds show /setup before anything else on first start,
  with a connection test against /health (or leave it blank for this
  server). The route stays reachable from Account -> Backend connection;
  switching backends clears the previous backend's token and reloads.
- Input normalisation: scheme defaulted (https, http for localhost),
  trailing slashes trimmed; a failed cross-origin test points at
  CORS_ALLOWED_ORIGINS.
- Dev keeps defaulting to the same-origin Vite proxy; /setup can be
  visited manually.

Verified: normalisation cases in Node, /health returns CORS headers for
an allowed origin, tsc/oxlint/build clean.
2026-09-17 22:56:31 -05:00
JakeBreath 16907c39ca Support cross-origin frontends alongside same-origin setups
- django-cors-headers with env-driven CORS_ALLOWED_ORIGINS,
  CORS_ALLOW_ALL_ORIGINS, CORS_ALLOW_CREDENTIALS and CSRF_TRUSTED_ORIGINS;
  same-origin traffic is unaffected and a disallowed origin gets no CORS
  headers. Token auth needs no cookies, so credentials stay off by default.
- TRUST_PROXY_HEADERS=true lets a TLS-terminating proxy supply
  X-Forwarded-Proto/Host for correct absolute URLs.
- API media URLs (raw/thumbnail/upload/similarity/staged previews) are now
  absolute, built from the request host, so <img>/<video>/fetch() keep
  working when the SPA is served from another origin. Signed URLs are still
  per-user; nothing is stored in the DB.
- The SPA gains VITE_API_BASE (build-time, empty = same-origin) applied by
  a small apiUrl() helper used for XHR/fetch and the few URL fallbacks.

Verified with a throwaway instance: preflight and GET responses carry the
allowed origin, foreign origins get nothing, media GETs include CORS for
cross-origin fetch(), and payload URLs use the request host (dev :8000
unchanged).
2026-09-17 22:50:12 -05:00
JakeBreath 98a674d55e AGENTS.md: deployment will be Docker-based
Recorded the decision so no future session re-introduces systemd/cron
unit files: scheduling belongs to the container setup.
2026-09-17 22:37:40 -05:00
JakeBreath bb87f563a9 Per-user browse preferences
Adds a preferences JSON field on the user plus GET/POST
/api/auth/preferences/ (merge semantics, validated keys), surfaced in
/auth/me/ and typed on the frontend.

The Account page gains a Browsing preferences card: landing page,
default rating filter, default sort, items per page and thumbnail size.
Signed-in users also sync these while browsing (the Library sidebar's
rating/sort/per-page controls and the new thumbnail slider), debounced;
on load the account's values seed the local UI state, so settings follow
the user across browsers. Guests keep the existing localStorage
behaviour. The thumbnail size drives the media grids (Library, Online,
pool detail) between 140 and 320px columns.

Verified the API against the dev server: merge keeps untouched keys,
invalid values 400, values round-trip through /auth/me/.
2026-09-17 22:34:38 -05:00
JakeBreath 3c49d2be2e Self-service avatar picker in Account
Users can now set their own profile picture instead of asking staff:
POST /api/auth/avatar/ accepts a J-ID (or blank to clear) and reuses the
same item resolution as the staff endpoint. The Account page gains a
profile picture card with a searchable, paginated library grid — any
item works (the thumbnail is used), the current avatar is marked, and
the choice is confirmed before saving. Refreshing the signed-in user
updates the shell avatar immediately.

Verified against the dev server: set, clear, unknown J-ID -> 400,
anonymous -> 401.
2026-09-17 22:31:05 -05:00
JakeBreath d8ba442e72 Collapse metadata panels into bottom sheets on mobile
DESIGN.md asks for metadata panels to slide up from the bottom under
768px. A BottomSheet component provides the trigger pill and the sheet
(backdrop blur, scroll lock, Escape to close) and ResponsivePanel swaps
between it and the existing desktop <aside>, so panel content is mounted
once either way.

Applied to the Library/Online/Similar detail asides and to long pool
descriptions. Upload needed nothing: its metadata editor is already a
full-screen modal that stacks cleanly on small screens.
2026-09-17 22:26:38 -05:00
JakeBreath e97c3b9da0 Toast action results and confirm destructive actions in one dialog
Inline banners and per-row status text reported action results all over
the app; they are replaced by a small toast stack (bottom-right, Level 3
floating well styling) that only speaks for actions: successes fade,
errors stay until dismissed, and form-field validation stays inline.

Destructive actions no longer use bespoke inline confirm steps (the
library detail's Confirm delete button) or fire immediately (duplicate
copies/items, delete page selections, temp cleanup, upload discard, job
cancellation): they all go through one promise-based confirm dialog
(confirmAction) with a danger variant, Escape/backdrop to cancel.
2026-09-17 22:19:54 -05:00
JakeBreath 1053e3ee55 AGENTS.md: remember the e621 OpenAPI spec and the project constraints
Future sessions are told to fetch and grep https://e621.wiki/openapi.yaml
before touching e621 endpoints, with the response-shape gotchas we hit
(bare arrays vs wrapped objects, pool search parameters, the form-encoded
PATCH for user settings). The durable constraints — token auth, no
server-side media processing, no imgdd, no chat — are recorded too so a
compacted session cannot regress them.
2026-09-17 22:03:09 -05:00
JakeBreath a152063d76 Keep animated images animated in the optimizer
- Animated WebP (J-82) was being flattened to its first frame: browsers
  have no animated WebP encoder, so the modal now sniffs the file header
  (4 KB range request), explains the limitation and disables Process
  instead of overwriting the file with a single frame.
- APNGs saved as .png took the still-image path and lost their frames;
  the header sniff looks for the acTL chunk, routes them to the animation
  pipeline (all frames + delays) and switches the UI to the animation
  options. The worker double-checks the header too, so no path can
  flatten an APNG.
- New dependency-free imageformat module, verified against real files
  (J-82 animated, static WebP/PNG, and a generated APNG named .png).
2026-09-17 21:57:02 -05:00
JakeBreath 38e64140ed Serve /health and keep 404 scanner noise out of the log file
Something on the network polls /health (and /v1/models) every 30 seconds;
the former now answers 200 with a database check instead of a 404, and
django.request is limited to ERROR in the logging config so scanner 404s
stop filling backend/logs/j621.log. Real server errors still log, and the
dev server keeps printing every request to its console.
2026-09-17 21:50:20 -05:00
JakeBreath 487d14c617 Keep the UI attached to background jobs across navigation
Jobs already run on the server — leaving the page or closing the tab does
not stop them — but the SPA lost its link to them because the task id
lived in component state. The online detail page now looks up the newest
task for the post: an active one resumes the progress bar and cancel
button, and a finished one shows "your last download for this post
finished — J-xx". The downloads list accepts a post_id filter for that
lookup.

The footer's Active Workers count is also a link to the staff stats
dashboard, which is the global view of running jobs.
2026-09-17 21:45:24 -05:00
JakeBreath 27cbfd882c Add job cancellation to the stats dashboard
- Active jobs on /stats get a cancel button wired to the existing
  download/match cancel endpoints, showing "cancelling..." and an inline
  error when the task already finished.
- Cancelling now sets the status immediately, so a task whose runner died
  in a restart stops showing as "downloading".
- Download streams use a bounded read timeout (10 s connect / 60 s read):
  a stalled socket fails within a minute (previously it could block
  forever), and a task cancelled while stalled is marked cancelled rather
  than error.
- The stats job list reaps stale download/match tasks, so phantom jobs
  never appear on the dashboard.
2026-09-17 21:41:54 -05:00
JakeBreath b024fc52d7 Add the staff stats dashboard
Backend: GET /api/stats/ (staff only) gathers psutil CPU/memory counters,
nvidia-smi GPU stats, the cached disk numbers and the running/finished
download + match jobs. Root logging now also writes a rotating file
(backend/logs/j621.log) so the dashboard can tail it, and psutil joins the
requirements. The storage payload computation is shared with the existing
storage endpoint.

Frontend: a /stats route + Stats nav entry for staff, polling every 2 s —
per-core CPU bars, memory and swap, GPUs (utilization, VRAM, temperature),
disk with the media/temp breakdown, active jobs with progress bars,
recently finished jobs with summaries, and the log tail with level colours
and an auto-scroll toggle. Section 4 of the roadmap is complete.
2026-09-17 21:36:33 -05:00
JakeBreath 0fcc4e518a Widen the optimize modal so the encoder diagnostics fit on one line 2026-09-17 21:28:17 -05:00
JakeBreath 8ae8080426 Name the browser in the encoder diagnostics and offer a container switch
The optimizer now shows which browser is running (e.g. "Firefox 141")
next to the encoder probe results, explains the common Firefox case
("Firefox does not implement H.264/HEVC encoding"), and offers a
one-click switch to the container that has working encoders.
2026-09-17 21:13:49 -05:00
JakeBreath 01b9a6181a Probe real encoder output, order codecs by it, and add a quality slider
Your machine's H.264/HEVC encoders report support and then emit no video
samples at all, which no configure-time check can see. The optimizer now
test-encodes three frames per codec (cached for 30 days), shows the result
as diagnostics in the modal (AVC x / VP9 (hardware) / ...), feeds the
working codecs to the worker so Auto tries them first, and tells you to
switch to WebM when a container has no usable encoder.

Video quality is now a 10-100 slider with a predicted bitrate and size
for this clip (mirroring Mediabunny's mapping), which makes the trade-off
visible instead of guessing from presets.
2026-09-17 21:11:46 -05:00
JakeBreath 33ae790bd0 Detect a missing video track and retry with another encoder
Your symptom (audio kept, video 0x0) means the hardware H.264 encode
silently produced no video samples while the audio was copied. The result
check now parses the moov box and requires a 'vide' handler, so an
audio-only output is treated as a failed attempt: the pipeline walks all
codec x hardware/software configurations (starting with hardware when
enabled) and only reports an error when every one fails, naming what each
attempt returned. Resizing is also skipped entirely unless a smaller
height was requested, keeping the scaler out of the pipeline.
2026-09-17 21:04:58 -05:00
JakeBreath e7657a3dd0 Fix the OPFS output lifecycle (locked stream on close)
StreamTarget closes the underlying writer when the output is finalized —
that is also what commits an OPFS file — so closing it ourselves threw
"Can not close locked stream". finalize() now simply reads the committed
file back, and every failure path (invalid attempt, encode error,
validation failure) aborts the write and deletes the temporary entry.
2026-09-17 21:01:15 -05:00
JakeBreath bd812bb601 Write video output to a real OPFS file instead of assembling chunks
The hand-rolled chunk assembly was verified correct in Node but still
produced a broken MP4 in the browser, so stop relying on it: the muxer now
streams into an Origin Private File System file (random access is exactly
what MP4 needs), the worker returns the OPFS File directly, and the entry
is deleted after a successful apply (stale ones are purged hourly). The
chunk collector remains only as a fallback for browsers without OPFS.

The result is validated before it reaches the UI: MP4s must contain a
moov box and WebM files must start with the EBML magic, so a broken muxer
output surfaces as an error instead of a 0x0 preview.
2026-09-17 20:58:26 -05:00
JakeBreath 9162cf23ce Fix the streaming assembler duplicating overlapped bytes
When the muxer patched a byte range inside an already-written chunk (the
mdat header, for example), the merge trimmed the right side of the old
segment but left its full blob on the left, duplicating megabytes and
shifting every box offset — the MP4 still reported its duration but had
no usable video track (0x0 in the browser, "moov atom not found" in
ffprobe).

The collector/assembler now lives in its own module and was verified in
Node with a real transmux of J-81: the assembled file matches the source
(h264 1280x720, 500 frames, AAC, 20.84 s) byte for byte in structure.
2026-09-17 20:50:08 -05:00
JakeBreath 038773e79e Use bitrate-based quality presets for video encoding
Qualitative presets made Mediabunny use quantizer (CRF-like) encoding, but
hardware H.264 encoders commonly ignore the per-frame quantizer and fall
back to a very low default bitrate, producing files far smaller than the
preset implies. Passing preferBitrate makes the preset map to an explicit
bitrate so hardware and software paths agree.
2026-09-17 20:45:47 -05:00
JakeBreath fed1e8bb5a Fix chunk assembly order and surface video duration in the optimizer
- The streaming assembler applied chunks sorted by position, so header
  patches written last could be overwritten by earlier data. Chunks now
  apply in arrival order (newest write wins per byte range) and are only
  laid out by position afterwards.
- The Optimize modal now reads the media metadata itself: both previews
  show resolution and duration, and the processed video is flagged in red
  when it comes out shorter than the original (a truncated encode is no
  longer something you have to guess by file size).
2026-09-17 20:45:12 -05:00
JakeBreath 58f030878e Fix duplicate React keys and the MP4 encoder probe
- MatchCard and IqdbCard are siblings in the library detail aside and
  both used key={item.j_id}, so React warned about duplicate children
  (J-81 twice). Their keys are now unique per card.
- The video pipeline no longer decides "this browser can't encode" from
  a single getFirstEncodableVideoCodec probe with source dimensions.
  It now probes Conversion.init with the real (even) output size across
  codec candidates and hardware preferences, uses the first valid
  configuration, and reports exactly what was tried when nothing works.
  It also fails fast with a clear message if VideoEncoder is missing in
  the worker.
2026-09-17 20:41:16 -05:00
JakeBreath cad659b75f Add a blacklist editor that saves to e621
The Online sidebar's "Your blacklist" section is editable now: typing a
tag appends it and each entry gets an x to remove it. Changes are written
straight to the e621 account (PATCH /users/{id}.json with
user[blacklisted_tags]); the store keeps the fetched profile for the id,
updates the list optimistically and reloads it from e621. The Followed
page's blacklist cloud is rebuilt afterwards via the new
/api/follows/cloud/?refresh=1 force flag.

Verified against the live API with a reversible add/verify/restore test.
2026-09-17 20:35:23 -05:00
JakeBreath f7e2242e5f Fix the client optimizer: PNG init, GIF compositing, streaming video
- PNG failed because wasm-bindgen's init only accepts a URL string (or a
  module/buffer): passing { module_or_path } broke every PNG optimization.
- GIF/APNG showed flashing colors because patches were written with
  putImageData, which ignores the transparency that means "keep the
  previous frame". Patches now blend through drawImage with correct
  disposal handling, and quantization keeps a transparent palette entry.
- Video no longer downloads the whole file into memory: Mediabunny reads
  it with range requests (UrlSource) and the muxer streams into Blob
  chunks (StreamTarget) that are assembled with last-write-wins range
  merging. BufferTarget held the entire output in memory, which crashed
  the tab on large files.
2026-09-17 20:10:31 -05:00
JakeBreath bab9904fc8 Client-side optimization pipeline with apply-to-J-ID
Backend:
- POST /api/files/J-x/optimize/ applies a browser-processed file: replaces
  every copy (renaming when the extension changes), recomputes MD5, size
  and perceptual hashes, seeds guest visibility; 400 when identical,
  409 when the result matches another item, owner/staff only.

Frontend (no server-side processing by design):
- optimize.worker.ts + pipelines: Mediabunny/WebCodecs for video with a
  prefer-hardware hint and per-browser codec detection; MozJPEG/OxiPNG/
  libwebp (jSquash) for images; gifuct-js+gifenc and UPNG for GIF/APNG.
- OptimizeModal: per-file-type options, original vs processed previews
  with sizes/savings, progress bar with ETA, then Apply (overwrite).
- Optimize button on the library detail for the uploader/staff.
2026-09-17 20:03:22 -05:00
JakeBreath 90e61bb333 Palette: keep focus in the search input so Enter only searches
Clicking the + follow toggle used to move focus onto that button, so the
next Enter re-triggered the follow instead of searching the highlighted
tag. The toggle now prevents the mouse-down focus steal, and the footer
hint spells out that Enter searches while + follows.
2026-09-17 19:38:49 -05:00
JakeBreath f74cf323b5 Add the F favorite shortcut and the Ctrl+K tag finder
The command palette now searches e621 tags as you type (debounced
/tags.json suggestions with category chips and post counts), lets you
search a tag online, follow/unfollow it inline and open it on e621, and
remembers recent searches in localStorage. New navigation commands cover
Pools, Followed and Similar. On the online detail, F toggles the current
post's favorite.
2026-09-17 19:36:30 -05:00
JakeBreath bbb33905c5 Roadmap: the e621 match scan covers the metadata backfill item 2026-09-17 19:31:38 -05:00
JakeBreath f9348ea9ba Roadmap: correct stale entries (filter statuses, where follows happen, hashing note) 2026-09-17 19:31:21 -05:00
JakeBreath c061d2681b Fix Download to Library crashing with a NameError
run_download_task sets e621_match_status from MediaItem, but the module
only imported DownloadTask — every download that carried e621 metadata
failed right after indexing, leaving the file in the library unlinked.
Verified the runner end-to-end with a stubbed fetch.
2026-09-17 19:28:19 -05:00
JakeBreath 1c2cb8d468 Add an ephemeral similarity check page
- /similar (nav: Similar): drop a file to get the exact MD5 match, the
  perceptual matches against the library, and e621 IQDB candidates
  (auto-run for images when credentials are configured). Read-only —
  nothing enters the library.
- SimilarityCheck model + /api/similarity/ (create/list/retrieve/delete)
  with signed preview URLs and an expires_at timestamp.
- Temp files are wiped on startup (AppConfig.ready, file-only so no
  database access during initialization), lazily past
  SIMILARITY_TTL_MINUTES (default 30, env-overridable), on delete, and
  by manage.py cleanup_similarity.
- uploadFile() takes a target path; .env.example documents the TTL.
2026-09-17 18:22:11 -05:00
JakeBreath cea34422ec Add the Jake Labs Non-Commercial Software Licence
Source-available, non-commercial licence (attribution + copyleft) at the
repo root as LICENSE, with the year/name filled in, ASCII punctuation,
a termination-on-breach clause with a 30-day cure, and a clearer
'no licence' wording in section 0. README gains a Licence section.
2026-09-17 15:14:05 -05:00
JakeBreath e95dc5f265 Clear the query cache when the signed-in account changes
Logging out or switching accounts kept the previous user's React Query
cache (follows, feed, cloud, e621 pages), so the new account briefly
saw the old one's followed tags/pools until each query refetched. The
query client now lives in lib/queryClient.ts and login/register/logout
clear it alongside the e621 credential store.
2026-09-17 14:41:56 -05:00
JakeBreath 62561a13f3 Show the Followed tag and pool cards in four columns on wide screens 2026-09-17 14:39:04 -05:00
JakeBreath 4c0b8cb1d8 Simplify the Followed page layout
- Remove the follow-a-tag / follow-a-pool forms; follows happen from tag
  chips (+ on detail views) and the Follow pool button on pool pages, so
  the empty states now point there instead.
- Move the blacklisted-tag cloud to a full-width horizontal panel at the
  bottom of the page.
- Tag and pool card grids cap at two columns so the cover cards are
  bigger.
2026-09-17 14:37:59 -05:00
JakeBreath 26f5069aa2 Fix pool post cards rendering as a colored sliver
PostCard's Link is inline by default; wrapped in a div on the pool detail
page it stopped being blockified like a grid child, so its rating-tinted
border collapsed into a vertical line on the left. The card is now
block-level, and the pool grid only wraps blacklisted posts (for the red
ring), so normal cards render exactly like the Online grid.
2026-09-17 14:32:24 -05:00