The build context is the repository root and there was no .dockerignore, so
'COPY backend/ ./' swept backend/venv (327 MB), backend/media (the actual
library, 224 MB), backend/logs, backend/staticfiles and backend/.env
(SECRET_KEY plus the database password) into the backend image: 1.43 GB per
architecture, including secrets headed for the registry. The frontend build
stage also copied the host's node_modules over the fresh install.
- Root .dockerignore excludes .git, virtualenvs, __pycache__, db.sqlite3,
logs/staticfiles, .env files, media/, node_modules, dist and the deploy
runtime state (data/, tailscale-state/).
- The backend Dockerfile now asserts the context is clean (.env, venv,
media/library, db.sqlite3 all absent) before collectstatic, so a missing
ignore file fails the build instead of leaking.
- Rebuilt: backend 1.43 GB -> 876 MB ('COPY backend/' is now 268 kB),
frontend stays at 65 MB. Verified by booting the compose stack with the
new image: migrations applied, /health ok, no .env or venv inside, and
/app/media is the mounted (empty) volume; the scheduler runs too.
- Removed the stale local images that still contained the library and the
dev .env.
Adds deploy/scheduler-entrypoint.sh to the backend image (entrypoint
j621-scheduler) and a scheduler service to the four composes that have a
backend. It waits until the database and migrations are ready, runs every
job once, then keeps to the intervals:
sync_followed_tags + sync_followed_pools every 30 min (J621_SYNC_EVERY)
cleanup_similarity hourly (J621_CLEAN_EVERY)
refresh_guest_blacklist daily (J621_BLACKLIST_EVERY)
It shares the backend image, media volume and env file, so commands see
the same library and database; failures are logged and retried next
interval. Output goes to docker compose logs scheduler; the frontend-only
composes have no backend and therefore no scheduler.
Verified against a real stack: the scheduler waited for migrations, ran all
four commands on start (follow syncs as anonymous, similarity cleanup, and
a guest blacklist refresh that pulled the real 14-tag list from e621), and
kept looping. All six composes still validate.
Builds deploy/.env from .env.example, generating SECRET_KEY and both
database passwords with openssl (base64/hex only, so nothing needs quoting
in the env file or the compose parser). Derives TS_HOSTNAME and
ALLOWED_HOSTS from the tailnet hostname, optionally sets
CORS_ALLOWED_ORIGINS/CSRF_TRUSTED_ORIGINS for split deployments, forces
DEBUG=False and writes the file with mode 600.
Modes: --no-prompt (defaults only), --update (refresh hostnames/auth key
while keeping the existing secrets, reading the stored FQDN from
ALLOWED_HOSTS), --force (rotate everything, with the SECRET_KEY warning in
the docs). Refuses to overwrite an existing file otherwise.
Verified: all modes, updated FQDN preservation, mode 600, and
docker compose config accepting the generated file.
Three more composes — compose.tailnet.yml, compose.tailnet.frontend.yml,
compose.tailnet.backend.yml — mirror the funnel set exactly but mount
serve.default/frontend/backend.tailnet.json, which drop AllowFunnel. The
sidecar still registers and serves HTTPS with a tailnet certificate, but
nothing is exposed publicly; Serve also needs no ACL change.
Project names carry a -tailnet suffix so both sets can coexist, and the
README explains that each set needs its own data directory (or host), plus
how to switch a host between funnel and tailnet by starting the other file
with the same .env.
Verified: all six composes validate with docker compose config, and the
six serve configs split cleanly into funnel (AllowFunnel present) and
tailnet-only (absent).
Test suite (the manual audit harness, now a real test):
- backend/apps/core/tests/test_security.py: 20 transactional tests across
guest visibility, object ownership, staged-upload/similarity privacy,
staff role boundaries, deletion rules, encrypted credentials, throttling
and the download allowlist. Uses temp media folders and clears cache.
Needs a one-time GRANT on test_j621 (documented in the module + README).
Images (deploy/J621-Frontend, deploy/J621-Backend, repo root as context):
- Frontend: node build -> static nginx with SPA fallback, asset caching and
an internal health endpoint.
- Backend: gunicorn + whitenoise (admin static collected at build), ffmpeg
for video thumbnails, migrations applied on start, GIT_HASH build arg so
the shell's version pill shows the commit.
Composes (distinct project names so they coexist with the dev stack):
- compose.yml (both), compose.frontend.yml, compose.backend.yml.
- A shared nginx proxy service is the only entry point (no host nginx, no
published host ports): /api,/admin,/static,/health -> backend, everything
else -> SPA; both upstreams resolve at request time so one config serves
all variants.
- A Tailscale sidecar per compose shares the nginx network namespace;
serve.default/frontend/backend.json use funnel ports 443 and 8443 only
(10000 is the remaining allowance) with ${TS_CERT_DOMAIN} substitution.
Registry: push_frontend.sh / push_backend.sh / push_all.sh build multi-arch
images and push :latest + :<sha> to the Gitea registry, following the
existing Packs-site pattern.
Docs: deploy/README.md + .env.example, ROADMAP section 6 updated,
AGENTS.md deployment and test notes.
Verified: all three composes validate, both nginx configs pass nginx -t,
both images build, the combined stack boots against real MariaDB/Redis
(migrations applied, /health ok, whitenoise serving admin static, env=prod,
git hash baked in), the proxy serves the SPA and routes /api, and
manage.py test apps.core.tests passes 20/20.
JakeBreath