Back off from e621 rate limits and pace requests more conservatively

e621 intermittently answers 429 to the IQDB endpoint; browsers hide that
status behind CORS ('Access-Control-Allow-Origin missing'), so the SPA
cannot read it. Treat every network-level failure as a possible rate limit
and pause all e621 traffic for a minute. The cooldown is shared through
localStorage so extra tabs respect it, requests are spaced 1.5s apart
instead of 1s, user-cancelled requests do not trigger a cooldown, and the
upload queue waits the cooldown out with a countdown instead of looking
stuck.

Server side: the per-process e621 gap goes from 0.5s to 1s so two gunicorn
workers cannot together exceed e621's 2/s hard limit.
This commit is contained in:
2026-09-19 00:37:32 -05:00
parent b96c311235
commit 9641862515
3 changed files with 65 additions and 5 deletions
@@ -21,6 +21,7 @@ import { api, authHeaders, apiUrl, errorMessage, uploadFile } from "@/lib/api";
import { cn } from "@/lib/cn";
import {
E621Error,
e621CooldownRemainingMs,
e621ErrorMessage,
effectiveCredentials,
fetchPost,
@@ -823,6 +824,15 @@ export default function UploadPage() {
setIqdbError(null);
try {
while (iqdbQueueRef.current.length > 0) {
// Wait out a rate-limit cooldown before touching e621 again.
const cooldown = e621CooldownRemainingMs();
if (cooldown > 0) {
setIqdbError(
`e621 rate limit — resuming in ${Math.ceil(cooldown / 1000)}s`,
);
await new Promise((resolve) => setTimeout(resolve, cooldown + 250));
setIqdbError(null);
}
const tempId = iqdbQueueRef.current.shift() as string;
const cached =
queryClient.getQueryData<TempUpload[]>(["uploads"]) ?? [];