Image Gen Failover Chain: In Priority Order

The openai image-gen plugin is the working path. It is also the one that fails in the most ways — rate limits, timeouts, 5xx waves, bad keys, expired keys. When it fails mid-batch, an operator has four moves, and the order matters: trying a different prompt “to see if it works” or jumping straight to text-only because “we are in a hurry” both make the outage worse. This guide is the four-step failover chain the ABS deploy actually runs, in priority order, with the failure shape that triggers each step and the verify that gates the move to the next one.

The chain is intentionally tight: “step N+1 if and only if step N failed” — not “step N+1 or step N if I have time.” Operators who skip steps lose the audit trail and end up shipping a text card when a cached PNG would have shipped, or burning retries on a known-bad prompt. Walk the chain, in order, every time, and the post-mortem two days later is short: which step stopped, what the error was, and what shipped as the next-best fallback.

The most common failure shape on ABS is a '429' rate_limit from a hot tenant key during the morning batch window. The second most common is a 4xx bad_request after a model rename upstream. The chain handles both: rate-limit falls to step 2 (cache reuse, no upstream call), auth fails falls to step 2 (also no upstream call), transient 5xx falls to step 3 (cache-bust on the page so origin retry catches a healthy POP). Step 4 (the text card) is rare — it ships once or twice a month at most, and usually only when steps 1-3 all fail within the same “10”-minute window.

The four-step chain

When the openai plugin is degraded, ABS’s failover chain is:

1. openai plugin — the working path. If generate() returns a path, use it.
2. Cached PNG set — pre-generated covers for the most-shipping slugs. Reuse without re-generation.
3. Cache-bust query string — append ?<random> to the image URL; CF serves a fresh request that may succeed at the edge.
4. Text-only social card — drop the image; ship a “600”x”600” typographic card with the guide title.

Each step has a trigger. The operator doesn’t skip steps; the chain runs in order.

Step 1: openai plugin (the working path)

from plugins.image_gen.openai import OpenAIImageGenProvider
r = OpenAIImageGenProvider().generate(prompt, aspect_ratio="landscape")

If r['success'] is True and r['image'] is a path, you’re done. Skip to verification.

Failure shapes that trigger step 2 or 3:

• r['success'] is False with error containing 'rate_limit' or '429'. The provider is rate-limited. Step 3 won’t help; fall to step 2.
• r['success'] is False with error containing 'timeout' or '5xx'. Transient. Step 3 may help (the cache-bust causes a fresh fetch from origin, hitting retry path).
• r['success'] is False with error containing 'auth' or '401'. The key is bad or expired. Skip to step 2 (cache reuse); don’t burn time on retry.

Step 2: cached PNG set

/srv/abs-site/public/images/ already has hundreds of covers from prior batches. If a recent batch shipped an image with the same LOCKED style + similar subject, it’s reusable.

The cache lookup:

ls -la /srv/abs-site/public/images/ | grep -E "<slug|prior-slug"

If you find a prior-slug-2026-7-14.png that visually matches the new guide, you can copy it to the new slug:

cp /srv/abs-site/public/images/prior-slug-2026-7-14.{png,webp} \
   /srv/abs-site/public/images/new-slug-2026-7-14.{png,webp}

Trade-off: an irrelevant image on a guide page is uglier than no image. The cached set only helps when the visual match is good. Don’t substitute a totally unrelated cover “because it’s a file.”

Step 3: cache-bust query string

If step 1 failed but a transient error, the cache may have an old MISS on the image URL. Append ?<random> to bust the cache:

<!-- Before -->
<img src="/images/cover.png" alt="..." />

<!-- After cache-bust -->
<img src="/images/cover.png?v=20260714t1130" alt="..." />

CF serves a fresh edge POP that will retry origin. This often unblocks step 1 on a subsequent call (just wait “60” seconds and retry the openai plugin).

The cache-bust is for the HTML page itself; the image file at /srv/abs-site/public/images/<slug>.png doesn’t change. The query is to CF’s edge POP, not the file.

Step 4: text-only social card

Last resort: ship a “600”x”600” typographic card with the guide title rendered as text. The card is built locally with PIL; no API call needed.

from PIL import Image, ImageDraw, ImageFont
img = Image.new('RGB', (600, 600), color='#1a1a1a')
draw = ImageDraw.Draw(img)
draw.text((40, 280), "Guide Title Goes Here",
          fill='#f0e6d2', font=ImageFont.load_default(36))
img.save('/srv/abs-site/public/images/text-card.png')

The text-only card is not great UX, but it ships. It’s also auditable — the operator sees no AI image and can decide to revisit later.

What NOT to do as failover

• Don’t fall back to a different prompt or model without surfacing the change. The operator needs to know what shipped.
• Don’t retry openai more than twice before escalating. Two failures is signal enough.
• Don’t skip the cache-bust step “to save time.” It costs “30” seconds.
• Don’t ship a text-only card with the AI image comment (“image temporarily unavailable”) — that’s worse than no comment.

The check at each step

Step	What to verify before moving on
1	Generated PNG opens, visual match OK
2	Cached PNG matches the guide’s subject
3	Cache-bust URL serves a cf-cache-status: HIT within “60” seconds
4	Text card renders the title and renders correctly

Verification checklist

#	Signal	Move to
1	openai.generate() returns success	Done — ship
2	openai.generate() fails / retry fails	Step 2 (cache)
3	Cache miss on image post-bust	Step 3 (cache-bust)
4	Cache-bust returns HIT but no image	Step 4 (text card)
5	Operator on-call	Escalate; don’t continue degrading

Sources

• OpenAI API status
• Cloudflare cache purge docs
• ABS companion: Image Gen: OpenAI Backend (Real Key, Not Codex OAuth)
• ABS companion: WebP and PNG on the Same Deploy (P42)

Last verified: 2026-07-”14”. Live chain verified during today’s batches “5”-“8” when openai was rate-limited once.