Run the periodic commands in a scheduler service (no host cron)

Adds deploy/scheduler-entrypoint.sh to the backend image (entrypoint
j621-scheduler) and a scheduler service to the four composes that have a
backend. It waits until the database and migrations are ready, runs every
job once, then keeps to the intervals:

  sync_followed_tags + sync_followed_pools   every 30 min (J621_SYNC_EVERY)
  cleanup_similarity                         hourly      (J621_CLEAN_EVERY)
  refresh_guest_blacklist                    daily       (J621_BLACKLIST_EVERY)

It shares the backend image, media volume and env file, so commands see
the same library and database; failures are logged and retried next
interval. Output goes to docker compose logs scheduler; the frontend-only
composes have no backend and therefore no scheduler.

Verified against a real stack: the scheduler waited for migrations, ran all
four commands on start (follow syncs as anonymous, similarity cleanup, and
a guest blacklist refresh that pulled the real 14-tag list from e621), and
kept looping. All six composes still validate.
This commit is contained in:
2026-09-18 12:51:17 -05:00
parent 93cce6b9fd
commit f25526782c
10 changed files with 204 additions and 5 deletions
+62
View File
@@ -0,0 +1,62 @@
#!/bin/sh
# Periodic maintenance for a J621 deployment (runs in the "scheduler"
# service; no host cron involved).
#
# sync_followed_tags + sync_followed_pools every SYNC_EVERY seconds (30 min)
# cleanup_similarity every CLEAN_EVERY seconds (1 h)
# refresh_guest_blacklist every BLACKLIST_EVERY (24 h)
#
# Waits for the database and migrations to be ready, runs everything once,
# then keeps checking. A failing command is logged and retried next interval,
# so a temporary e621 outage is not fatal.
set -u
SYNC_EVERY="${SYNC_EVERY:-1800}"
CLEAN_EVERY="${CLEAN_EVERY:-3600}"
BLACKLIST_EVERY="${BLACKLIST_EVERY:-86400}"
log() { echo "[scheduler] $(date -Iseconds) $*"; }
run() {
log "running: $*"
if "$@"; then
log "done: $*"
else
log "FAILED (retrying at the next interval): $*"
fi
}
command -v python >/dev/null || { log "python not found"; exit 1; }
log "waiting for the database and migrations..."
until python manage.py migrate --check >/dev/null 2>&1; do
sleep 10
done
log "database ready; intervals: sync=${SYNC_EVERY}s cleanup=${CLEAN_EVERY}s blacklist=${BLACKLIST_EVERY}s"
now=$(date +%s)
last_sync=$((now - SYNC_EVERY))
last_clean=$((now - CLEAN_EVERY))
last_blacklist=$((now - BLACKLIST_EVERY))
while true; do
now=$(date +%s)
if [ $((now - last_sync)) -ge "$SYNC_EVERY" ]; then
last_sync=$now
run python manage.py sync_followed_tags
run python manage.py sync_followed_pools
fi
if [ $((now - last_clean)) -ge "$CLEAN_EVERY" ]; then
last_clean=$now
run python manage.py cleanup_similarity
fi
if [ $((now - last_blacklist)) -ge "$BLACKLIST_EVERY" ]; then
last_blacklist=$now
run python manage.py refresh_guest_blacklist
fi
sleep 30
done