JakeBreath
70d7a4f606
The remote feed copy now happens by default (overridable with J621_DESKTOP_FEED_HOST or --host, skippable with --local), so a release is one command on the build machine.
210 lines
9.1 KiB
Markdown
210 lines
9.1 KiB
Markdown
# J621 deployment (Docker + Tailscale)
|
|
|
|
Three compose variants, all behind an **nginx** service and a **Tailscale**
|
|
sidecar. Nothing is published on the host's ports and no system nginx or
|
|
reverse proxy is involved: the sidecar shares the nginx service's network
|
|
namespace and Tailscale Serve/Funnel exposes it.
|
|
|
|
## Funnel set (public HTTPS)
|
|
|
|
| Compose | Services | Entry point | Serve config |
|
|
| --- | --- | --- | --- |
|
|
| `compose.yml` (default) | mariadb, redis, backend, frontend, nginx, tailscale | `https://<host>.<tailnet>.ts.net` → nginx → backend + SPA | `serve.default.json` |
|
|
| `compose.frontend.yml` | frontend, nginx, tailscale | `https://<host>.<tailnet>.ts.net` → nginx → SPA only | `serve.frontend.json` |
|
|
| `compose.backend.yml` | mariadb, redis, backend, nginx, tailscale | `https://<host>.<tailnet>.ts.net:8443` → nginx → API | `serve.backend.json` |
|
|
|
|
Funnel can only expose ports **443**, **8443** and **10000**, so the separate
|
|
backend uses 8443. Change it in `serve.backend.json` (and `.env`) if you
|
|
prefer 10000.
|
|
|
|
## Tailnet-only set (no Funnel)
|
|
|
|
The same three stacks without `AllowFunnel` in the serve config: the sidecar
|
|
still registers the node and serves over HTTPS with a tailnet certificate,
|
|
but only devices on your tailnet can reach it — nothing is exposed to the
|
|
public internet.
|
|
|
|
| Compose | Serve config | Reachable at |
|
|
| --- | --- | --- |
|
|
| `compose.tailnet.yml` (both) | `serve.default.tailnet.json` | `https://<host>.<tailnet>.ts.net` |
|
|
| `compose.tailnet.frontend.yml` | `serve.frontend.tailnet.json` | `https://<host>.<tailnet>.ts.net` |
|
|
| `compose.tailnet.backend.yml` | `serve.backend.tailnet.json` | `https://<host>.<tailnet>.ts.net:8443` |
|
|
|
|
```bash
|
|
docker compose -f compose.tailnet.yml up -d
|
|
# or compose.tailnet.frontend.yml / compose.tailnet.backend.yml
|
|
```
|
|
|
|
Notes:
|
|
- Project names are `…-tailnet` so both sets can be installed side by side.
|
|
- Serve needs no tailnet policy change (Funnel requires the `funnel` node
|
|
attribute in your ACLs), so this set works as soon as the sidecar joins.
|
|
- Both sets use the same `deploy/data` directory (library, database, logs).
|
|
Run one set per data directory — two MariaDB instances on one data dir would
|
|
corrupt it. Giving the tailnet set its own `TS_HOSTNAME` (or running it on a
|
|
second host) is the way to run both.
|
|
- Switching a host from funnel to tailnet (or back) is just starting the other
|
|
compose file with the same `.env`.
|
|
|
|
## Environment file
|
|
|
|
`gen_env.sh` builds `deploy/.env` from `.env.example`, generating `SECRET_KEY`
|
|
and both database passwords with `openssl`:
|
|
|
|
```bash
|
|
./gen_env.sh # asks for the Tailscale auth key + hostname(s)
|
|
./gen_env.sh --no-prompt # secrets and defaults only; fill TS_AUTHKEY later
|
|
./gen_env.sh --update # refresh hostnames/authkey, keep the existing secrets
|
|
./gen_env.sh --force # regenerate everything, including SECRET_KEY
|
|
```
|
|
|
|
It derives `TS_HOSTNAME` and `ALLOWED_HOSTS` from the tailnet hostname you
|
|
give it, and can set `CORS_ALLOWED_ORIGINS`/`CSRF_TRUSTED_ORIGINS` when you
|
|
provide the frontend's hostname for a split deployment. `--update` is the safe
|
|
way to add hostnames later; `--force` rotates SECRET_KEY, which invalidates
|
|
signed media URLs and stored e621 API keys. The file is written with mode 600
|
|
and is git-ignored.
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
cd deploy
|
|
./gen_env.sh # or: cp .env.example .env and edit it yourself
|
|
docker compose up -d # default: everything on one host, public funnel
|
|
# or
|
|
docker compose -f compose.frontend.yml up -d
|
|
docker compose -f compose.backend.yml up -d
|
|
# tailnet-only (no public funnel): compose.tailnet*.yml, see below
|
|
```
|
|
|
|
The images are pulled from the Gitea registry; append `--build` (or run the
|
|
push scripts) if you build locally. Data lives in `deploy/data/`
|
|
(`media/`, `logs/`, `mariadb/`, `redis/`) and the Tailscale node state in
|
|
`deploy/tailscale-state/`; both are git-ignored.
|
|
|
|
## First start
|
|
|
|
The SPA asks where the backend is (`/setup`) in production builds:
|
|
|
|
- **Default compose** — leave the field blank (same origin) and everything
|
|
works through nginx.
|
|
- **Frontend-only + backend-only** — point the (cross-origin) SPA at the
|
|
backend's funnel URL, e.g. `https://j621-backend.<tailnet>.ts.net:8443`,
|
|
and give the backend `CORS_ALLOWED_ORIGINS=https://<frontend-host>.<tailnet>.ts.net`
|
|
(plus `CSRF_TRUSTED_ORIGINS` for the admin).
|
|
- **Desktop app** — the Electron shell (`desktop/`) is served from the
|
|
`app://j621` origin, which `gen_env.sh` includes in `CORS_ALLOWED_ORIGINS`
|
|
automatically (add it by hand if you wrote `.env` yourself). The shell's
|
|
setup screen prints the exact origin when the connection test fails.
|
|
- Without a backend at all, choose **"Continue without a backend"** to run in
|
|
local mode (e621 browsing only).
|
|
|
|
Any origin and port is fine — the backend builds its media URLs from the
|
|
forwarded host/proto (`TRUST_PROXY_HEADERS=true` is set in all composes).
|
|
|
|
## Images
|
|
|
|
Two Dockerfiles, both built from the repository root:
|
|
|
|
```bash
|
|
docker build -f deploy/J621-Frontend -t j621-frontend .
|
|
docker build --build-arg "GIT_HASH=$(git rev-parse --short HEAD)" \
|
|
-f deploy/J621-Backend -t j621-backend .
|
|
```
|
|
|
|
`J621-Frontend` builds the SPA and serves it as static files.
|
|
`J621-Backend` is Django + gunicorn (migrations run on start) and includes
|
|
ffmpeg for video thumbnails. The `GIT_HASH` build arg is baked into the
|
|
backend image so the shell's version pill shows the commit (images have no
|
|
`.git` directory); the push scripts pass it automatically.
|
|
|
|
Both build from the repository root, which is filtered by `.dockerignore`:
|
|
`backend/venv`, `backend/media`, `backend/logs`, `backend/staticfiles`,
|
|
`backend/.env`, `frontend/node_modules`, `frontend/dist` and the deploy
|
|
runtime state never enter the images — the database, media and logs come
|
|
from the compose volumes and `deploy/.env` at runtime. The backend build
|
|
also asserts that (`.env`, `venv`, `media/library`, `db.sqlite3` absent), so
|
|
a missing ignore file fails the build instead of shipping secrets. Rebuild
|
|
(and `docker image prune`) if you built before that guard existed.
|
|
|
|
Push multi-arch images to the Gitea registry:
|
|
|
|
```bash
|
|
./push_all.sh # or push_frontend.sh / push_backend.sh [sha]
|
|
```
|
|
|
|
They tag `:latest` and `:<commit-sha>` and expect `docker login
|
|
gitea.rainbow-herring.ts.net` to succeed.
|
|
|
|
Build the desktop installers without publishing anything:
|
|
|
|
```bash
|
|
./build_desktop.sh # Arch + Debian + Windows -> desktop/release/
|
|
./build_desktop.sh --linux # Arch + Debian only
|
|
./build_desktop.sh --win # Windows installer only
|
|
```
|
|
|
|
Hand the files out or attach them to a Gitea release manually — the script
|
|
prints sizes and SHA-256 sums for the release notes.
|
|
|
|
Push the desktop builds and their update metadata to the frontend's feed:
|
|
|
|
```bash
|
|
./push_desktop.sh # build Linux packages + copy the feed to jakerasp
|
|
./push_desktop.sh --win # also cross-build the NSIS installer (wine)
|
|
./push_desktop.sh --local # publish locally only, skip the remote copy
|
|
./push_desktop.sh --host other:/path/to/J621
|
|
```
|
|
|
|
The remote copy defaults to `jakerasp:/home/jake/servers/J621`, or
|
|
`$J621_DESKTOP_FEED_HOST` when set. Artifacts land in `deploy/data/desktop/`,
|
|
which the frontend nginx mounts read-only and serves at `/desktop/`. The
|
|
remote copy uses rsync when both ends have it, tar over ssh when the server
|
|
does not. The desktop app's "Check for updates…" menu item reads
|
|
`latest-linux.yml` / `latest.yml` from there (see `desktop/README.md`).
|
|
Backend-only composes have no frontend, so no feed.
|
|
|
|
## Scheduled jobs
|
|
|
|
Compose files with a backend also run a **`scheduler`** service — the same
|
|
backend image with a different entrypoint, so no host cron is involved:
|
|
|
|
| Job | Default interval | Env override |
|
|
| --- | --- | --- |
|
|
| `sync_followed_tags` + `sync_followed_pools` | every 30 minutes | `J621_SYNC_EVERY` |
|
|
| `cleanup_similarity` | hourly | `J621_CLEAN_EVERY` |
|
|
| `refresh_guest_blacklist` | daily | `J621_BLACKLIST_EVERY` |
|
|
|
|
It waits for the database and migrations before its first run, runs every job
|
|
once on start, then keeps to the intervals (failures are logged and retried
|
|
next round). Output goes to `docker compose logs scheduler`. Intervals are
|
|
seconds, set in `deploy/.env`. The frontend-only composes have no backend, so
|
|
no scheduler.
|
|
|
|
## Tests
|
|
|
|
The security/permission suite lives in `backend/apps/core/tests/`:
|
|
|
|
```bash
|
|
cd ../backend
|
|
./venv/bin/python manage.py test apps.core.tests
|
|
```
|
|
|
|
It needs a one-time grant on the database user (test databases are created
|
|
from scratch):
|
|
|
|
```sql
|
|
GRANT ALL ON `test_j621`.* TO 'j621'@'%';
|
|
```
|
|
|
|
## Notes
|
|
|
|
- The nginx service is the only entry point: `/api`, `/admin`, `/static` and
|
|
`/health` go to the backend, everything else to the SPA. Both upstreams are
|
|
resolved at request time, so the same config works in all three variants.
|
|
- The compose healthchecks use `/nginx-health` (nginx itself), `/health`
|
|
(Django + database) and the frontend's static server, so `depends_on:
|
|
condition: service_healthy` gates the sidecar on a working stack.
|
|
- `deploy/data/media/library` is the watched folder; drop files there (or use
|
|
uploads) and run `manage.py scan_files` inside the backend container.
|