Two bugs made the toggle look broken even though the preference was
stored correctly (JakeBreathild had online_hot_default false):
- The Account card never seeded online_hot_default into its form state,
so the checkbox always rendered checked via the '?? true' fallback.
It now starts from the saved value.
- Turning the toggle off did not change Online when the URL still
carried tags=order:hot (e.g. Ctrl+Shift+R reloading the old URL).
order:hot on its own is the default, not a deliberate search, so it is
now removed when the account has the toggle off; 'order:hot canine' or
any other search is still left untouched. The decision moved into
hotDefault.ts (set-hot / clear-hot / keep) with the matrix verified in
Node.
New online_hot_default preference (on by default, so guests and accounts
that never saved preferences get it): opening /online without a search
replaces the URL with ?tags=order:hot — e621's metatag for the order the
Hot page uses — so it is visible in the search field and shareable.
Existing searches are never touched: they live in the URL, so a refresh
or back/forward keeps them, while a fresh visit (nav pill, first time,
after a long time away) gets the hot default again. Turning the toggle
off opens Online on the site-wide newest posts as before.
The toggle sits in Account -> Browsing preferences and saves with the
rest of the settings; the backend validates the new boolean
(400 for non-boolean input).
DELETE /api/users/{id}/ with guards: nobody deletes the account they are
signed in as (400); staff can delete regular/uploader accounts only,
while admins can also delete staff and admins (403 for staff targets
otherwise, and the last admin can never be deleted). Deleting a user
removes their follows, tokens and staged uploads — including the staged
files on disk — while library items survive and simply lose their owner
(uploaded_by is SET_NULL), as does download/match/similarity history.
The Users page gets a per-row delete button behind the shared confirm
dialog, hidden wherever the backend would refuse (own row, or a
staff/admin target when the actor is not an admin).
Verified against the dev server: staff 204 for a regular account, 400
for self, 403 for an admin; admin 204; a plain account gets 403. After
deleting a user that owned J-81 and had a staged file, the file was gone
and J-81 survived with a null owner.
Footer:
- Left is now 'Backend Storage:' with a capacity bar (blue, peach at 80%,
red at 95% per DESIGN.md) and a used/total/free tooltip; the watched
folder path is no longer printed. /api/status/ returns a compact storage
summary instead of the path (the full storage page still shows paths to
authenticated users).
- Centre shows the backend API origin (empty = same origin). Staff get a
link to /setup to point the browser elsewhere; everyone else sees it as
plain text. The Account 'Backend connection' card is gone — this is
installation plumbing, not a per-user setting.
- Design spec updated to match.
Staff role:
- The custom role did nothing on several endpoints that only accepted
Django's is_staff/is_superuser. One canonical check now exists:
User.is_app_staff (superuser, Django staff, or the staff role), used by
the stats/users APIs, item object permissions, can_delete, upload/
similarity/download/match querysets, and the management commands
(which also pick staff-role accounts for e621 sync/match and file
ownership).
Verified with a role-only staff account (is_staff/is_superuser false):
stats/users 200, all 32 downloads + 2 scans visible, others' items
editable; the same account as role=user gets 403 for all of those.
Replaces the build-time VITE_API_BASE knob with a runtime setup screen so
one build works same-origin and cross-origin:
- frontend/src/lib/backend.ts stores the API origin in localStorage
(empty = same origin). DEFAULT_BACKEND_URL is the clearly marked,
easily edited prefilled default — the matrix.org equivalent; set it to
your public API origin.
- Production builds show /setup before anything else on first start,
with a connection test against /health (or leave it blank for this
server). The route stays reachable from Account -> Backend connection;
switching backends clears the previous backend's token and reloads.
- Input normalisation: scheme defaulted (https, http for localhost),
trailing slashes trimmed; a failed cross-origin test points at
CORS_ALLOWED_ORIGINS.
- Dev keeps defaulting to the same-origin Vite proxy; /setup can be
visited manually.
Verified: normalisation cases in Node, /health returns CORS headers for
an allowed origin, tsc/oxlint/build clean.
- django-cors-headers with env-driven CORS_ALLOWED_ORIGINS,
CORS_ALLOW_ALL_ORIGINS, CORS_ALLOW_CREDENTIALS and CSRF_TRUSTED_ORIGINS;
same-origin traffic is unaffected and a disallowed origin gets no CORS
headers. Token auth needs no cookies, so credentials stay off by default.
- TRUST_PROXY_HEADERS=true lets a TLS-terminating proxy supply
X-Forwarded-Proto/Host for correct absolute URLs.
- API media URLs (raw/thumbnail/upload/similarity/staged previews) are now
absolute, built from the request host, so <img>/<video>/fetch() keep
working when the SPA is served from another origin. Signed URLs are still
per-user; nothing is stored in the DB.
- The SPA gains VITE_API_BASE (build-time, empty = same-origin) applied by
a small apiUrl() helper used for XHR/fetch and the few URL fallbacks.
Verified with a throwaway instance: preflight and GET responses carry the
allowed origin, foreign origins get nothing, media GETs include CORS for
cross-origin fetch(), and payload URLs use the request host (dev :8000
unchanged).
Adds a preferences JSON field on the user plus GET/POST
/api/auth/preferences/ (merge semantics, validated keys), surfaced in
/auth/me/ and typed on the frontend.
The Account page gains a Browsing preferences card: landing page,
default rating filter, default sort, items per page and thumbnail size.
Signed-in users also sync these while browsing (the Library sidebar's
rating/sort/per-page controls and the new thumbnail slider), debounced;
on load the account's values seed the local UI state, so settings follow
the user across browsers. Guests keep the existing localStorage
behaviour. The thumbnail size drives the media grids (Library, Online,
pool detail) between 140 and 320px columns.
Verified the API against the dev server: merge keeps untouched keys,
invalid values 400, values round-trip through /auth/me/.
Users can now set their own profile picture instead of asking staff:
POST /api/auth/avatar/ accepts a J-ID (or blank to clear) and reuses the
same item resolution as the staff endpoint. The Account page gains a
profile picture card with a searchable, paginated library grid — any
item works (the thumbnail is used), the current avatar is marked, and
the choice is confirmed before saving. Refreshing the signed-in user
updates the shell avatar immediately.
Verified against the dev server: set, clear, unknown J-ID -> 400,
anonymous -> 401.
DESIGN.md asks for metadata panels to slide up from the bottom under
768px. A BottomSheet component provides the trigger pill and the sheet
(backdrop blur, scroll lock, Escape to close) and ResponsivePanel swaps
between it and the existing desktop <aside>, so panel content is mounted
once either way.
Applied to the Library/Online/Similar detail asides and to long pool
descriptions. Upload needed nothing: its metadata editor is already a
full-screen modal that stacks cleanly on small screens.
Inline banners and per-row status text reported action results all over
the app; they are replaced by a small toast stack (bottom-right, Level 3
floating well styling) that only speaks for actions: successes fade,
errors stay until dismissed, and form-field validation stays inline.
Destructive actions no longer use bespoke inline confirm steps (the
library detail's Confirm delete button) or fire immediately (duplicate
copies/items, delete page selections, temp cleanup, upload discard, job
cancellation): they all go through one promise-based confirm dialog
(confirmAction) with a danger variant, Escape/backdrop to cancel.
- Animated WebP (J-82) was being flattened to its first frame: browsers
have no animated WebP encoder, so the modal now sniffs the file header
(4 KB range request), explains the limitation and disables Process
instead of overwriting the file with a single frame.
- APNGs saved as .png took the still-image path and lost their frames;
the header sniff looks for the acTL chunk, routes them to the animation
pipeline (all frames + delays) and switches the UI to the animation
options. The worker double-checks the header too, so no path can
flatten an APNG.
- New dependency-free imageformat module, verified against real files
(J-82 animated, static WebP/PNG, and a generated APNG named .png).
Jobs already run on the server — leaving the page or closing the tab does
not stop them — but the SPA lost its link to them because the task id
lived in component state. The online detail page now looks up the newest
task for the post: an active one resumes the progress bar and cancel
button, and a finished one shows "your last download for this post
finished — J-xx". The downloads list accepts a post_id filter for that
lookup.
The footer's Active Workers count is also a link to the staff stats
dashboard, which is the global view of running jobs.
- Active jobs on /stats get a cancel button wired to the existing
download/match cancel endpoints, showing "cancelling..." and an inline
error when the task already finished.
- Cancelling now sets the status immediately, so a task whose runner died
in a restart stops showing as "downloading".
- Download streams use a bounded read timeout (10 s connect / 60 s read):
a stalled socket fails within a minute (previously it could block
forever), and a task cancelled while stalled is marked cancelled rather
than error.
- The stats job list reaps stale download/match tasks, so phantom jobs
never appear on the dashboard.
Backend: GET /api/stats/ (staff only) gathers psutil CPU/memory counters,
nvidia-smi GPU stats, the cached disk numbers and the running/finished
download + match jobs. Root logging now also writes a rotating file
(backend/logs/j621.log) so the dashboard can tail it, and psutil joins the
requirements. The storage payload computation is shared with the existing
storage endpoint.
Frontend: a /stats route + Stats nav entry for staff, polling every 2 s —
per-core CPU bars, memory and swap, GPUs (utilization, VRAM, temperature),
disk with the media/temp breakdown, active jobs with progress bars,
recently finished jobs with summaries, and the log tail with level colours
and an auto-scroll toggle. Section 4 of the roadmap is complete.
The optimizer now shows which browser is running (e.g. "Firefox 141")
next to the encoder probe results, explains the common Firefox case
("Firefox does not implement H.264/HEVC encoding"), and offers a
one-click switch to the container that has working encoders.
Your machine's H.264/HEVC encoders report support and then emit no video
samples at all, which no configure-time check can see. The optimizer now
test-encodes three frames per codec (cached for 30 days), shows the result
as diagnostics in the modal (AVC x / VP9 (hardware) / ...), feeds the
working codecs to the worker so Auto tries them first, and tells you to
switch to WebM when a container has no usable encoder.
Video quality is now a 10-100 slider with a predicted bitrate and size
for this clip (mirroring Mediabunny's mapping), which makes the trade-off
visible instead of guessing from presets.
Your symptom (audio kept, video 0x0) means the hardware H.264 encode
silently produced no video samples while the audio was copied. The result
check now parses the moov box and requires a 'vide' handler, so an
audio-only output is treated as a failed attempt: the pipeline walks all
codec x hardware/software configurations (starting with hardware when
enabled) and only reports an error when every one fails, naming what each
attempt returned. Resizing is also skipped entirely unless a smaller
height was requested, keeping the scaler out of the pipeline.
StreamTarget closes the underlying writer when the output is finalized —
that is also what commits an OPFS file — so closing it ourselves threw
"Can not close locked stream". finalize() now simply reads the committed
file back, and every failure path (invalid attempt, encode error,
validation failure) aborts the write and deletes the temporary entry.
The hand-rolled chunk assembly was verified correct in Node but still
produced a broken MP4 in the browser, so stop relying on it: the muxer now
streams into an Origin Private File System file (random access is exactly
what MP4 needs), the worker returns the OPFS File directly, and the entry
is deleted after a successful apply (stale ones are purged hourly). The
chunk collector remains only as a fallback for browsers without OPFS.
The result is validated before it reaches the UI: MP4s must contain a
moov box and WebM files must start with the EBML magic, so a broken muxer
output surfaces as an error instead of a 0x0 preview.
When the muxer patched a byte range inside an already-written chunk (the
mdat header, for example), the merge trimmed the right side of the old
segment but left its full blob on the left, duplicating megabytes and
shifting every box offset — the MP4 still reported its duration but had
no usable video track (0x0 in the browser, "moov atom not found" in
ffprobe).
The collector/assembler now lives in its own module and was verified in
Node with a real transmux of J-81: the assembled file matches the source
(h264 1280x720, 500 frames, AAC, 20.84 s) byte for byte in structure.
Qualitative presets made Mediabunny use quantizer (CRF-like) encoding, but
hardware H.264 encoders commonly ignore the per-frame quantizer and fall
back to a very low default bitrate, producing files far smaller than the
preset implies. Passing preferBitrate makes the preset map to an explicit
bitrate so hardware and software paths agree.
- The streaming assembler applied chunks sorted by position, so header
patches written last could be overwritten by earlier data. Chunks now
apply in arrival order (newest write wins per byte range) and are only
laid out by position afterwards.
- The Optimize modal now reads the media metadata itself: both previews
show resolution and duration, and the processed video is flagged in red
when it comes out shorter than the original (a truncated encode is no
longer something you have to guess by file size).
- MatchCard and IqdbCard are siblings in the library detail aside and
both used key={item.j_id}, so React warned about duplicate children
(J-81 twice). Their keys are now unique per card.
- The video pipeline no longer decides "this browser can't encode" from
a single getFirstEncodableVideoCodec probe with source dimensions.
It now probes Conversion.init with the real (even) output size across
codec candidates and hardware preferences, uses the first valid
configuration, and reports exactly what was tried when nothing works.
It also fails fast with a clear message if VideoEncoder is missing in
the worker.
The Online sidebar's "Your blacklist" section is editable now: typing a
tag appends it and each entry gets an x to remove it. Changes are written
straight to the e621 account (PATCH /users/{id}.json with
user[blacklisted_tags]); the store keeps the fetched profile for the id,
updates the list optimistically and reloads it from e621. The Followed
page's blacklist cloud is rebuilt afterwards via the new
/api/follows/cloud/?refresh=1 force flag.
Verified against the live API with a reversible add/verify/restore test.
- PNG failed because wasm-bindgen's init only accepts a URL string (or a
module/buffer): passing { module_or_path } broke every PNG optimization.
- GIF/APNG showed flashing colors because patches were written with
putImageData, which ignores the transparency that means "keep the
previous frame". Patches now blend through drawImage with correct
disposal handling, and quantization keeps a transparent palette entry.
- Video no longer downloads the whole file into memory: Mediabunny reads
it with range requests (UrlSource) and the muxer streams into Blob
chunks (StreamTarget) that are assembled with last-write-wins range
merging. BufferTarget held the entire output in memory, which crashed
the tab on large files.
Backend:
- POST /api/files/J-x/optimize/ applies a browser-processed file: replaces
every copy (renaming when the extension changes), recomputes MD5, size
and perceptual hashes, seeds guest visibility; 400 when identical,
409 when the result matches another item, owner/staff only.
Frontend (no server-side processing by design):
- optimize.worker.ts + pipelines: Mediabunny/WebCodecs for video with a
prefer-hardware hint and per-browser codec detection; MozJPEG/OxiPNG/
libwebp (jSquash) for images; gifuct-js+gifenc and UPNG for GIF/APNG.
- OptimizeModal: per-file-type options, original vs processed previews
with sizes/savings, progress bar with ETA, then Apply (overwrite).
- Optimize button on the library detail for the uploader/staff.
Clicking the + follow toggle used to move focus onto that button, so the
next Enter re-triggered the follow instead of searching the highlighted
tag. The toggle now prevents the mouse-down focus steal, and the footer
hint spells out that Enter searches while + follows.
The command palette now searches e621 tags as you type (debounced
/tags.json suggestions with category chips and post counts), lets you
search a tag online, follow/unfollow it inline and open it on e621, and
remembers recent searches in localStorage. New navigation commands cover
Pools, Followed and Similar. On the online detail, F toggles the current
post's favorite.
- /similar (nav: Similar): drop a file to get the exact MD5 match, the
perceptual matches against the library, and e621 IQDB candidates
(auto-run for images when credentials are configured). Read-only —
nothing enters the library.
- SimilarityCheck model + /api/similarity/ (create/list/retrieve/delete)
with signed preview URLs and an expires_at timestamp.
- Temp files are wiped on startup (AppConfig.ready, file-only so no
database access during initialization), lazily past
SIMILARITY_TTL_MINUTES (default 30, env-overridable), on delete, and
by manage.py cleanup_similarity.
- uploadFile() takes a target path; .env.example documents the TTL.
Logging out or switching accounts kept the previous user's React Query
cache (follows, feed, cloud, e621 pages), so the new account briefly
saw the old one's followed tags/pools until each query refetched. The
query client now lives in lib/queryClient.ts and login/register/logout
clear it alongside the e621 credential store.
- Remove the follow-a-tag / follow-a-pool forms; follows happen from tag
chips (+ on detail views) and the Follow pool button on pool pages, so
the empty states now point there instead.
- Move the blacklisted-tag cloud to a full-width horizontal panel at the
bottom of the page.
- Tag and pool card grids cap at two columns so the cover cards are
bigger.
PostCard's Link is inline by default; wrapped in a div on the pool detail
page it stopped being blockified like a grid child, so its rating-tinted
border collapsed into a vertical line on the left. The card is now
block-level, and the pool grid only wraps blacklisted posts (for the red
ring), so normal cards render exactly like the Online grid.
- /pools: search by name, category/active filters, sort options and
pagination per the OpenAPI spec, with covers taken from each pool's
first post in one batched post call; blacklisted covers fall back to a
placeholder and deleted pools get an archive marker.
- /pools/<id>: DText description, post grid kept in the pool's own order
with chunked loading, in-library badges, a blacklist reveal toggle and
a Follow pool button wired into the follows API.
- e621 client gains fetchPools/fetchPool; Pools nav entry added.
The chip toggle swaps its icon for a tiny current-color spinner from
the click until the follows list reflects the new state, covering the
refetch gap so '+' never flashes back before the checkmark.
TagChip gains a followable mode that renders a small toggle inside the
chip: '+' follows the tag, '✓' (click to unfollow) once followed, and a
red marker with the reason as its title when e621 rejects it. Enabled on
the library detail's e621 metadata card (matched posts) and on online
post detail tags; guests see plain chips and the label click keeps
working beside the toggle.
Backend (new apps.follows):
- FollowedTag/FollowedPool/FollowedPost models; per-user follows with
unseen tracking, plus FollowCloud for the cached blacklist cloud.
- Two periodic commands sharing one fetch path: sync_followed_tags and
sync_followed_pools fetch each followed tag/pool's newest posts (one
e621 search per unique follow), store unseen feed rows, refresh covers
and pool metadata; both fall back to anonymous e621 access.
- API: /api/follows/tags|pools (follow, unfollow, mark seen), a merged
feed with per-follow filtering, and /api/follows/cloud/ which rebuilds
the blacklisted-tag cloud in a daemon thread when its 10 min cache is
stale (polling returns building/ready).
- e621 client now supports anonymous reads; trimmed posts carry preview
URLs for covers and feed tiles.
Frontend:
- /followed page: follow forms, cover cards with unseen badges and
Mark seen, merged feed with filter/unseen toggle, and a blacklist
cloud panel that polls while building. Followed nav entry added.
- New IQDB card on local item pages: fetches the signed raw file, POSTs it
to e621's /iqdb_queries.json with the user's credentials, and lists
candidates as tiles (thumbnail, rating, score%) with exact-MD5 markers.
- Selecting a candidate opens a detail panel (rating, score, favs, size,
tag preview) and linking is an explicit confirm that reuses the e621
match endpoint; videos show a note since IQDB is image-only.
- Guests don't see the card; linking follows the uploader/staff rule.
A homescreen-style age gate shown before auth or any route: J621 brand
mark, the explicit 18+ check, an Enter action remembered per browser in
localStorage (j621.age-verified), and a blocked state if the visitor
chooses Leave.
- MediaItem gains e621_match_status (unknown/matched/not_found/deleted)
and e621_checked_at, backfilled for existing matched items.
- Server-side e621 client (apps/library/e621.py) using the user's stored
credentials, throttled to 2 req/s, with typed errors.
- Matching service: MD5 lookup, manual post linking (flags MD5
mismatches), unlink, metadata refresh, deleted-post detection.
- Detail actions POST /api/files/J-x/match/ and /unlink/ (uploader or
staff only).
- Background library scans: MatchTask + /api/matches/ with missing/all
scopes, progress polling, cancel and stale-task reaping; the scan
counts toward the footer's Active Workers. Same pass available as
manage.py match_e621 for cron.
- Library gains not_found/deleted status filters; the detail page adds
an e621 match card (check / link by post ID / unlink) and the metadata
card warns when a post was deleted on e621.
Items flagged hidden_from_guests (blacklisted tags) returned 404 for
<img> requests since tags cannot send the auth header. The API now
exposes signed raw_url/thumbnail_url fields (mirroring upload previews
and avatars), and the SPA uses them in the gallery, detail view,
duplicates and delete screens, and upload visual matches.
Backend:
- MediaItem gains search_tags (custom + e621 tags, lowercase) and
has_custom_data, maintained on save with a data migration backfill
- File list search accepts search_type=filename|tags|both (tag search is
word-AND across the flattened tag text) and status=matched|custom|
unknown filters
- New /api/tags/cloud/ endpoint (cached 2 min per guest/auth, invalidated
on item changes and deletions) returning the most-used tags, honouring
guest visibility
Frontend:
- Library sidebar: Filename/Tags/Both selector, status pill toggles
(persisted), and a clickable tag cloud that runs a tag search
- Roadmap updated
Staff and uploaders (for their own items) get a Delete action on
/detail/J-<id> with an inline confirmation; on success it returns to the
library and refreshes files, duplicates and storage.
Backend:
- Perceptual hashes (aHash/dHash/pHash/wHash via imagehash, no imgdd)
stored on items, computed on upload/download and by the new
compute_visual_hashes command
- Duplicates API: exact duplicates (multi-location items), visual matches
for one item, union-find similarity groups with pagination
- Delete API with ownership/staff checks, per-item and per-copy deletion,
watched-folder path validation; storage overview and temp cleanup;
file list accepts j_ids batches
- Staged uploads are flagged visual_match with their library matches
(threshold via VISUAL_MATCH_THRESHOLD)
- Staff users API: list with upload counts, set role and avatar by J-ID;
User.avatar FK with signed avatar URLs
- Download threads close their DB connection and stale tasks are reaped,
keeping behaviour Gunicorn-friendly
Frontend:
- /duplicates: exact duplicate groups with per-copy delete, visual
similarity controls, search similar to a J-ID, paginated groups with
selection, bulk delete and dismiss
- /delete: storage cards, delete by J-ID with preview grid, temp cleanup
- /users: staff directory with role selects and avatar J-ID inputs
- Nav + command palette entries; top-bar avatar; upload cards and the
metadata modal show library visual matches
- Status polling drops from 30s to 5s, and download start/finish/cancel
invalidates it immediately, so the footer's Active Workers reflects
running download tasks in near real time
- The e621 time in the status pill is now a button: it opens a dropdown
with the session's request history (clock time, endpoint, duration,
colour-coded) plus totals; closes on outside click or Escape
- Metrics store keeps the last 50 requests
Backend:
- DownloadTask model + background thread runner: streams the file with
progress (%, bytes, speed) and a cancel flag, then indexes it, names it
J-<id>.<ext> and applies the e621 metadata
- DownloadTaskViewSet (create/retrieve/cancel) replaces the synchronous
endpoint; the status footer's worker counts now reflect download jobs
- Client download proxy (/api/online/file/) streams an e621 original to
the browser with Content-Disposition: attachment, restricted to the
configured e621 CDN hosts so it cannot be used as an open proxy
Frontend:
- Online detail: progress bar with percentage, transferred size, speed
and cancel while downloading; success links to the new J-ID
- New 'Download to client' button available to everyone (guests too)
The modal preview used a fixed 200px column with object-contain, so wide
images rendered as a slim letterboxed rectangle with dead space. The
preview now sizes naturally (max 320px wide / 55vh tall, aspect ratio
preserved) and the modal column follows it.
- Files upload as soon as they are dropped or selected; a sequential
queue processes them and picks up files added while uploading
- The progress list is now a grid: 5 columns with about three rows
visible (scrolls beyond) showing thumbnails, per-file progress bars
and status; 'clear finished' remains, object URLs are revoked
- Videos show an icon instead of a thumbnail
- 'dismiss all' clears every indexed record at once
- Indexed cards show the actual file preview: completed records now get a
signed library media URL (raw for images, thumbnail for videos) so
<img>/<video> tags can load it, including items hidden from guests
- Media raw/thumbnail endpoints accept the signature for anonymous
requests and fall back to the normal guest-filtered path otherwise
- Guest blacklist keeps a persistent Redis mirror: an expired TTL or an
unreachable e621 keeps the last successful list instead of falling
back to the small local list
JakeBreath