J621/AGENTS.md

Use the venv on backend/venv/

DO NOT mess with the system's python

The original J621 (Django) app is kept for reference at
    /mnt/Disco/Proyects/Python3.13/J621-Django/
Consult it when matching original behavior (batch sizes for e621 lookups,
per-page sets, upload flow) instead of guessing.

For Frontend work:

read both Frontend Design related Markdown files on the frontend/ folder

For anything touching e621 endpoints or payloads (posts, pools, tags, users,
favorites, IQDB, blacklists, uploads), consult the e621 OpenAPI spec first:
    https://e621.wiki/openapi.yaml

Download it once per session (it is ~450 KB YAML), then grep it for the exact
path/parameter/schema names instead of guessing:

    curl -sL https://e621.wiki/openapi.yaml -o /tmp/opencode/e621-openapi.yaml
    grep -n "^  /pools.json" /tmp/opencode/e621-openapi.yaml

Notes learned from the spec (verify before trusting):
- Index endpoints often return a bare array (e.g. GET /pools.json) while the
  show endpoint returns the object directly (e.g. GET /pools/{id}.json) — not
  always wrapped in a named key.
- Pools index supports search[order] (id_asc, id_desc, name, created_at,
  post_count), search[name_matches], search[category], search[is_active].
- Own settings (including user[blacklisted_tags]) are updated with
  PATCH /users/{id}.json as form-encoded data; /users/me.json returns the user
  object at the top level.

Project constraints (do not regress):
- Backend is Django 6 + DRF with token auth (no session auth on the API);
  MariaDB/Redis come from docker-compose.yml.
- Deployment will be Docker-based (compose); do not add systemd/cron unit
  files for scheduling — use the container setup for timers/workers.
- deploy/ is the deployment source of truth: J621-Frontend (SPA on static
  nginx) and J621-Backend (gunicorn + whitenoise, ffmpeg, migrations on
  start), three compose variants (both / frontend-only / backend-only) behind
  a shared nginx proxy service, and a Tailscale sidecar per compose.
  serve.*.json are the public Funnel variants (only ports 443 / 8443 / 10000);
  serve.*.tailnet.json + compose.tailnet*.yml are the same stacks without
  AllowFunnel (tailnet-only, no public exposure). Images are pushed to the
  Gitea registry with deploy/push_*.sh (multi-arch, :latest + :sha, GIT_HASH
  baked in for the version pill); deploy/gen_env.sh generates .env with
  openssl secrets (--update keeps SECRET_KEY, --force rotates it).
- Periodic commands (follow syncs, similarity cleanup, guest blacklist
  refresh) run in the composes' `scheduler` service — the backend image with
  the j621-scheduler entrypoint, intervals via J621_*_EVERY. No host cron.
- CI/CD lives in .gitea/workflows: ci.yml runs on every push/PR (Django checks
  + the full backend suite against MariaDB/Redis service containers, frontend
  lint/type-check/build); cd.yml is manual and builds/pushes both images
  multi-arch plus the desktop packages (attached to the Gitea release
  `desktop-v<version>`). Jobs run on the user-scoped runners: `ubuntu-latest`
  on nitro-ci, `desktop` on msi-mortar-ci. Do not add actions/cache
  (`cache: pip`/`npm`) to these workflows: Gitea's cache service hangs the job
  on restore/save. The live desktop update feed (deploy/data/desktop) is still
  published with `deploy/push_desktop.sh --no-build` from a machine with SSH
  to the deploy host — CI has no key for that.
- Security/permission tests live in backend/apps/core/tests and need a
  one-time grant: GRANT ALL ON `test_j621`.* TO 'j621'@'%';

Dev environment (this machine):
- MariaDB/Redis run from the repo root: `docker compose up -d` (host ports
  3307 and 6380 — other local projects own 3305/6379/3406).
- Backend dev server: `backend/venv/bin/python manage.py runserver 0.0.0.0:8000`.
  Django's system checks touch the database, so start MariaDB first; run it
  from backend/ with the venv python.
- Frontend dev server: `npm run dev` in frontend/ — Vite listens on
  [::1]:5173 (http://localhost:5173) and proxies /api to 127.0.0.1:8000.
- Restarting: kill dev servers by PID from `pgrep -af`; never
  `pkill -f "manage.py runserver"` (the pattern matches its own shell).
- Suite: `backend/venv/bin/python manage.py test apps.core.tests` (~80 s).
- Same-origin and cross-origin frontends both work, and the SPA can also run
  with no backend at all: a production build asks on first start (/setup) and
  stores the answer in localStorage — a URL, "" for same origin, or "none"
  for local mode. Local mode shows only the e621-facing pages (Online, Pools)
  with credentials kept in this browser (j621.e621) and a "Setup Backend"
  button in the shell. DEFAULT_BACKEND_URL in frontend/src/lib/backend.ts is
  the prefilled default; the backend allows extra origins via
  CORS_ALLOWED_ORIGINS (plus CSRF_TRUSTED_ORIGINS for the admin), and API
  media URLs are absolute (built from the request host), so signed files load
  cross-origin too. TRUST_PROXY_HEADERS=true is required behind a
  TLS-terminating proxy.
- No server-side media processing: the home server cannot handle it.
  Compression/optimization runs client-side (WebCodecs + WASM in a worker)
  and the server only applies the result via POST /api/files/J-x/optimize/.
- Do not use imgdd; perceptual hashing is imagehash server-side.
- Chat/messaging features are out of scope.

Security hardening (do not weaken):
- e621 API keys are stored Fernet-encrypted with a key derived from
  SECRET_KEY (apps/accounts/crypto.py); rotating SECRET_KEY invalidates them
  (and all signed media URLs), so users must re-enter the key.
- API throttles live in REST_FRAMEWORK (env-overridable): anon 120/min,
  user 600/min, login 5/min, register 20/hour, e621_proxy 60/hour. Signed
  media URLs (raw/thumbnail/staged-file/similarity-file actions) are exempt
  on purpose: <img>/<video> tags fetch them without an Authorization header,
  so a gallery would otherwise drain the anonymous bucket and get 429 JSON
  instead of images. THROTTLE_ENABLED=false removes the anon+user limits for
  private/tailnet deployments (the login/register/proxy guards stay).
- Only admins (superusers) may grant/revoke the staff role or delete
  staff/admin accounts; staff manage regular/uploader accounts only.
- Storage, duplicates, delete, temp-clear, uploads and downloads require
  upload rights (CanUpload), not merely authentication.
- Server-side fetching only happens for allowlisted e621 media hosts via
  services.validate_remote_url / open_remote (every redirect hop is
  re-validated); do not call requests.get on user-supplied URLs elsewhere.
- A repeatable audit harness (permission matrix, IDOR, guest visibility,
  signed URLs, SSRF, throttles) was used to verify this; formalising it as
  a test suite is still open in the roadmap.