JakeBreath
f86eccf9a3
Findings from the audit (50-check harness across guest/user/uploader/staff/ admin) and their fixes: - SSRF: 'Download to Library' and the staged-upload resolve path fetched any http(s) URL. services.validate_remote_url now enforces the e621 media allowlist and open_remote re-validates every redirect hop; the download-task create endpoint and the guest proxy use them, so internal addresses (127.0.0.1, LAN, metadata) are rejected with 400. - Privilege escalation: staff could promote users to staff and demote other staff. Role changes across the staff boundary now require an admin, matching the account-deletion rules; the Users page hides what the backend would refuse. - SPA-only gating: /api/storage/ and /api/duplicates/* were readable by any authenticated account (absolute paths, duplicate groups) while the SPA only shows them to uploaders. They now require CanUpload. - Throttling (REST_FRAMEWORK, env-overridable, counted in Redis): anon 120/min, user 600/min, login 5/min, register 20/hour, guest e621 proxy 60/hour. Login now goes through a throttled view. - e621 API keys are encrypted at rest with a Fernet key derived from SECRET_KEY (apps/accounts/crypto.py); a data migration encrypts existing rows and the column widens first. Reads decrypt transparently, legacy plaintext still works, and a changed SECRET_KEY reads as 'not configured' instead of leaking. Rotating SECRET_KEY now invalidates stored keys as well as signed media URLs. - Hardening: the server refuses to start with DEBUG=False while SECRET_KEY is still the development default. Verified: corrected harness 50/50 (guest visibility, IDOR, signed-URL tamper/expiry, staged-upload/similarity privacy, role matrix, SSRF), login throttles at the 6th attempt with 429, anon polling unaffected, the guest proxy still reaches allowlisted hosts, live e621 auth works with the decrypted key, and DB rows hold only ciphertext.
3.6 KiB
3.6 KiB
Use the venv on backend/venv/
DO NOT mess with the system's python
For Frontend work:
read both Frontend Design related Markdown files on the frontend/ folder
For anything touching e621 endpoints or payloads (posts, pools, tags, users, favorites, IQDB, blacklists, uploads), consult the e621 OpenAPI spec first: https://e621.wiki/openapi.yaml
Download it once per session (it is ~450 KB YAML), then grep it for the exact path/parameter/schema names instead of guessing:
curl -sL https://e621.wiki/openapi.yaml -o /tmp/opencode/e621-openapi.yaml
grep -n "^ /pools.json" /tmp/opencode/e621-openapi.yaml
Notes learned from the spec (verify before trusting):
- Index endpoints often return a bare array (e.g. GET /pools.json) while the show endpoint returns the object directly (e.g. GET /pools/{id}.json) — not always wrapped in a named key.
- Pools index supports search[order] (id_asc, id_desc, name, created_at, post_count), search[name_matches], search[category], search[is_active].
- Own settings (including user[blacklisted_tags]) are updated with PATCH /users/{id}.json as form-encoded data; /users/me.json returns the user object at the top level.
Project constraints (do not regress):
- Backend is Django 6 + DRF with token auth (no session auth on the API); MariaDB/Redis come from docker-compose.yml.
- Deployment will be Docker-based (compose); do not add systemd/cron unit files for scheduling — use the container setup for timers/workers.
- Same-origin and cross-origin frontends both work, and the SPA can also run with no backend at all: a production build asks on first start (/setup) and stores the answer in localStorage — a URL, "" for same origin, or "none" for local mode. Local mode shows only the e621-facing pages (Online, Pools) with credentials kept in this browser (j621.e621) and a "Setup Backend" button in the shell. DEFAULT_BACKEND_URL in frontend/src/lib/backend.ts is the prefilled default; the backend allows extra origins via CORS_ALLOWED_ORIGINS (plus CSRF_TRUSTED_ORIGINS for the admin), and API media URLs are absolute (built from the request host), so signed files load cross-origin too. TRUST_PROXY_HEADERS=true is required behind a TLS-terminating proxy.
- No server-side media processing: the home server cannot handle it. Compression/optimization runs client-side (WebCodecs + WASM in a worker) and the server only applies the result via POST /api/files/J-x/optimize/.
- Do not use imgdd; perceptual hashing is imagehash server-side.
- Chat/messaging features are out of scope.
Security hardening (do not weaken):
- e621 API keys are stored Fernet-encrypted with a key derived from SECRET_KEY (apps/accounts/crypto.py); rotating SECRET_KEY invalidates them (and all signed media URLs), so users must re-enter the key.
- API throttles live in REST_FRAMEWORK (env-overridable): anon 120/min, user 600/min, login 5/min, register 20/hour, e621_proxy 60/hour.
- Only admins (superusers) may grant/revoke the staff role or delete staff/admin accounts; staff manage regular/uploader accounts only.
- Storage, duplicates, delete, temp-clear, uploads and downloads require upload rights (CanUpload), not merely authentication.
- Server-side fetching only happens for allowlisted e621 media hosts via services.validate_remote_url / open_remote (every redirect hop is re-validated); do not call requests.get on user-supplied URLs elsewhere.
- A repeatable audit harness (permission matrix, IDOR, guest visibility, signed URLs, SSRF, throttles) was used to verify this; formalising it as a test suite is still open in the roadmap.