Run the periodic commands in a scheduler service (no host cron)
Adds deploy/scheduler-entrypoint.sh to the backend image (entrypoint j621-scheduler) and a scheduler service to the four composes that have a backend. It waits until the database and migrations are ready, runs every job once, then keeps to the intervals: sync_followed_tags + sync_followed_pools every 30 min (J621_SYNC_EVERY) cleanup_similarity hourly (J621_CLEAN_EVERY) refresh_guest_blacklist daily (J621_BLACKLIST_EVERY) It shares the backend image, media volume and env file, so commands see the same library and database; failures are logged and retried next interval. Output goes to docker compose logs scheduler; the frontend-only composes have no backend and therefore no scheduler. Verified against a real stack: the scheduler waited for migrations, ran all four commands on start (follow syncs as anonymous, similarity cleanup, and a guest blacklist refresh that pulled the real 14-tag list from e621), and kept looping. All six composes still validate.
This commit is contained in:
@@ -0,0 +1,62 @@
|
||||
#!/bin/sh
|
||||
# Periodic maintenance for a J621 deployment (runs in the "scheduler"
|
||||
# service; no host cron involved).
|
||||
#
|
||||
# sync_followed_tags + sync_followed_pools every SYNC_EVERY seconds (30 min)
|
||||
# cleanup_similarity every CLEAN_EVERY seconds (1 h)
|
||||
# refresh_guest_blacklist every BLACKLIST_EVERY (24 h)
|
||||
#
|
||||
# Waits for the database and migrations to be ready, runs everything once,
|
||||
# then keeps checking. A failing command is logged and retried next interval,
|
||||
# so a temporary e621 outage is not fatal.
|
||||
set -u
|
||||
|
||||
SYNC_EVERY="${SYNC_EVERY:-1800}"
|
||||
CLEAN_EVERY="${CLEAN_EVERY:-3600}"
|
||||
BLACKLIST_EVERY="${BLACKLIST_EVERY:-86400}"
|
||||
|
||||
log() { echo "[scheduler] $(date -Iseconds) $*"; }
|
||||
|
||||
run() {
|
||||
log "running: $*"
|
||||
if "$@"; then
|
||||
log "done: $*"
|
||||
else
|
||||
log "FAILED (retrying at the next interval): $*"
|
||||
fi
|
||||
}
|
||||
|
||||
command -v python >/dev/null || { log "python not found"; exit 1; }
|
||||
|
||||
log "waiting for the database and migrations..."
|
||||
until python manage.py migrate --check >/dev/null 2>&1; do
|
||||
sleep 10
|
||||
done
|
||||
log "database ready; intervals: sync=${SYNC_EVERY}s cleanup=${CLEAN_EVERY}s blacklist=${BLACKLIST_EVERY}s"
|
||||
|
||||
now=$(date +%s)
|
||||
last_sync=$((now - SYNC_EVERY))
|
||||
last_clean=$((now - CLEAN_EVERY))
|
||||
last_blacklist=$((now - BLACKLIST_EVERY))
|
||||
|
||||
while true; do
|
||||
now=$(date +%s)
|
||||
|
||||
if [ $((now - last_sync)) -ge "$SYNC_EVERY" ]; then
|
||||
last_sync=$now
|
||||
run python manage.py sync_followed_tags
|
||||
run python manage.py sync_followed_pools
|
||||
fi
|
||||
|
||||
if [ $((now - last_clean)) -ge "$CLEAN_EVERY" ]; then
|
||||
last_clean=$now
|
||||
run python manage.py cleanup_similarity
|
||||
fi
|
||||
|
||||
if [ $((now - last_blacklist)) -ge "$BLACKLIST_EVERY" ]; then
|
||||
last_blacklist=$now
|
||||
run python manage.py refresh_guest_blacklist
|
||||
fi
|
||||
|
||||
sleep 30
|
||||
done
|
||||
Reference in New Issue
Block a user