Cloudflare AI Gateway and Turnstile: Fronting a Friend's Local Qwen Endpoint

July 02, 2026·8 min read

This blog has a handful of LLM-powered features: Neon GPT (a no-login chat), the world generator in Blocks, and the live slide remixer in the arcade's PowerPoint Karaoke game. None of them call a commercial API. The model is a Qwen instance running on a friend's hardware, served through LiteLLM, and every request from this site reaches it through a Cloudflare AI Gateway that I own.

That arrangement creates a specific problem: the endpoint burns my friend's electricity and GPU time, and the frontend is a public static site with no accounts. Anyone can open dev tools, find the API call, and script it. The answer ended up being three layers: an AI Gateway for visibility and control, a thin Cloudflare Pages Function so no keys ever ship to the browser, and Turnstile to make sure a real browser on my site is asking.

The Building Blocks

Cloudflare AI Gateway is a proxy that sits between your application and an LLM provider. You create a gateway in the Cloudflare dashboard, point your app at the gateway URL instead of the provider, and every request flows through Cloudflare on its way upstream. In exchange you get analytics (request counts, tokens, latency, errors), full request/response logging, response caching, rate limiting, and retry/fallback behavior, all configured per-gateway without touching application code. It supports the big commercial providers, but it will also front any OpenAI-compatible endpoint, which is what makes it useful here: the "provider" behind my gateway is a friend's machine.

Cloudflare Turnstile is Cloudflare's CAPTCHA replacement. The widget runs in the browser (there's an invisible mode with no puzzle to solve), produces a short-lived token, and your server verifies that token against the siteverify API. A passing token means a real browser executed the challenge on a page with your sitekey. It's free, and it's the entire reason curl can't talk to my LLM proxy.

LiteLLM is an open-source proxy that exposes 100+ model backends behind one OpenAI-compatible API. My friend runs it in front of a locally hosted Qwen3 model (a 27B running an NVFP4 quant). From my side of the wire, it looks exactly like the OpenAI chat completions API: POST /v1/chat/completions, same request shape, same response shape. That compatibility is what lets the AI Gateway and my Pages Function treat a hobby GPU box like any other provider.

The Architecture

Browser (Neon GPT, Blocks, arcade games)
  │  POST /api/llm         + X-LLM-Session header
  ▼
Cloudflare Pages Function  (origin allowlist → rate limit → session check → sanitize)
  │  Authorization: Bearer <gateway key>
  │  cf-aig-authorization: Bearer <AI Gateway token>
  ▼
Cloudflare AI Gateway      (logging, analytics, caching, kill switch)
  │
  ▼
LiteLLM (friend's box)     (OpenAI-compatible proxy)
  │
  ▼
Qwen3 27B (local GPU)

And the session side-channel that feeds the X-LLM-Session header:

Browser ── invisible Turnstile widget ──> turnstile token
  │  POST /api/llm-session { turnstileToken }
  ▼
Pages Function ── siteverify ──> Cloudflare
  │  (valid → mint HMAC session token, 10 min TTL)
  ▼
Browser caches token, attaches it to every /api/llm call

Nothing secret ever reaches the browser. The gateway URL, the upstream key, the AI Gateway token, the Turnstile secret, and the session-signing secret all live as encrypted Cloudflare environment variables.

Setting Up the AI Gateway

In the Cloudflare dashboard: AI → AI Gateway → Create Gateway. That gives you a gateway endpoint. Because the upstream here is a custom OpenAI-compatible server rather than a named provider, requests address it in passthrough style. The gateway forwards to the configured upstream and records everything that goes through.

Two settings matter for this build:

1. Authenticated Gateway. By default anyone who discovers your gateway URL can send requests through it (and to your upstream). Turning on authentication makes the gateway reject any request that doesn't carry a cf-aig-authorization header with a token you generate in the dashboard.
2. Logs. Request/response logging is per-gateway. For a setup where someone else pays the GPU bill, the log view is the accountability layer. I can see exactly what traffic I sent him, token counts included.

The server-side call that goes through the gateway lives in a Pages Function (functions/api/llm.js):

const base = env.LLM_GATEWAY_URL.replace(/\/+$/, "")
const upstreamHeaders = { "Content-Type": "application/json", Authorization: `Bearer ${env.LLM_GATEWAY_KEY}` }
// When routing through a Cloudflare AI Gateway (authenticated), add its gateway token.
if (env.LLM_AIG_TOKEN) upstreamHeaders["cf-aig-authorization"] = `Bearer ${env.LLM_AIG_TOKEN}`

upstream = await fetch(`${base}/chat/completions`, {
  method: "POST",
  headers: upstreamHeaders,
  body: JSON.stringify(payload),
})

Two credentials, two different locks: Authorization is the LiteLLM key the upstream expects, cf-aig-authorization is the AI Gateway's own token. The LLM_AIG_TOKEN check means the same code runs with or without a gateway in the path. Unset it and the function calls the provider directly.

One Qwen-specific detail in the payload: Qwen3 is a reasoning model, and by default it will happily spend hundreds of tokens thinking before it answers. For interactive features that's latency you can feel, so thinking is off unless a caller explicitly asks for it:

const payload = {
  model: env.LLM_MODEL || body.model || "qwen-3.6-27b-nvfp4",
  messages: safeMessages,
  temperature: clamp(numOr(body.temperature, 0.9), 0, 2),
  max_tokens: clamp(numOr(body.max_tokens, 256), 1, 2048),
  stream: false,
  // Qwen3 is a reasoning model, so disable thinking for fast, direct answers
  chat_template_kwargs: { enable_thinking: body.think === true },
}

The Pages Function Proxy

Cloudflare Pages automatically deploys anything in /functions as a serverless function, so /api/llm is one file with no infrastructure. It does four jobs before anything reaches the gateway.

Origin gating. Only requests originating from this site (or localhost during dev) are allowed:

if (!origin || !allowed.has(origin)) return json({ error: "Forbidden" }, 403, headers)

Rate limiting. Cloudflare's rate limiting binding is wired per-IP, written so it no-ops until the binding is configured:

if (env.RATE_LIMITER && typeof env.RATE_LIMITER.limit === "function") {
  const ip = request.headers.get("CF-Connecting-IP") || origin || "anon"
  const { success } = await env.RATE_LIMITER.limit({ key: ip })
  if (!success) return json({ error: "Rate limited, slow down." }, 429, headers)
}

Input capping. Whatever the client sends gets clamped before it can cost anything: last 16 messages only, 6,000 characters per message, max_tokens capped at 2048, roles whitelisted:

const safeMessages = messages.slice(-16).map((m) => ({
  role: ["system", "user", "assistant"].includes(m.role) ? m.role : "user",
  content: String(m.content == null ? "" : m.content).slice(0, 6000),
}))

Response normalization. Every caller on the site gets the same tiny shape back, { content, usage }, regardless of what the upstream returns, with a fallback to reasoning_content for the times Qwen answers inside its thinking block.

Turnstile as the Abuse Gate

Origin checks stop drive-by scripts, but Origin is just a header, and anything outside a browser can forge it. Turnstile is the piece that can't be forged, and it's wired as a two-endpoint flow so the widget only runs occasionally instead of on every request.

Client side, an invisible widget is rendered off-screen once and executed on demand (src/lib/turnstile.js):

widgetId = window.turnstile.render(host, {
  sitekey: SITEKEY,
  size: "flexible",
  action: "share",
  callback: (token) => { pendingResolver(token); pendingResolver = null },
})
// later:
window.turnstile.execute(widgetId, { action: "share" })

The token goes to /api/llm-session, which verifies it server-side and mints a short-lived session:

async function verifyTurnstile(secret, token, ip) {
  const form = new URLSearchParams()
  form.append("secret", secret)
  form.append("response", token)
  if (ip) form.append("remoteip", ip)
  const r = await fetch("https://challenges.cloudflare.com/turnstile/v0/siteverify", { method: "POST", body: form })
  const j = await r.json()
  return !!j.success
}

A passing verification returns an HMAC-signed token, <expiry>.<nonce>.<signature>, signed with WebCrypto, valid for 10 minutes:

async function mintToken(secret) {
  const exp = Date.now() + TTL_SECONDS * 1000
  const nonce = b64url(crypto.getRandomValues(new Uint8Array(8)))
  const payload = `${exp}.${nonce}`
  const sig = await sign(secret, payload) // HMAC-SHA256 via crypto.subtle
  return `${payload}.${sig}`
}

The shared client (src/utils/llm.js) caches that session token, attaches it to every /api/llm call as X-LLM-Session, and transparently re-mints when it expires or gets a 401. /api/llm verifies the signature and expiry on each request. No state, no KV, just math:

if (env.TURNSTILE_SECRET && env.LLM_SESSION_SECRET) {
  const ok = await verifySession(request.headers.get("X-LLM-Session"), env.LLM_SESSION_SECRET)
  if (!ok) return json({ error: "Unauthorized, no valid session" }, 401, headers)
}

The chain of custody: you can't get a session without passing Turnstile, you can't pass Turnstile without a real browser on an allowed origin, and you can't call the LLM without a session. A forger with curl fails at step one.

One design decision worth copying: the whole gate fails open. If TURNSTILE_SECRET or LLM_SESSION_SECRET is unset, /api/llm-session hands back a placeholder token and /api/llm skips the check. The client flow is identical in dev (where there's no Turnstile) and prod, and misconfiguring a secret degrades to "no bot gate" instead of "site-wide outage."

Neon GPT

Neon GPT is the most direct consumer: a no-login chat UI at /tools/chat that talks straight through this stack. There's nothing special about its API usage. It calls the same askLLM() helper as everything else:

const reply = await askLLM(payload, { maxTokens: 1400, temperature: 0.7, timeout: 45000 })

The user never logs in, never sees a CAPTCHA, and never learns the endpoint. The invisible Turnstile run happens on first message, the session rides along for 10 minutes, and re-minting is invisible too. The same helper powers the Blocks world generator and the arcade games, so every AI feature on the site inherits the gate for free.

Design Decisions

Why an AI Gateway between two hobby systems? Because the alternative is my code calling my friend's box with zero visibility for either of us. The gateway gives both sides an audit trail neither has to build: request logs, token counts, latency, error rates. It's also a kill switch. If something on my site goes haywire, the gateway can rate-limit or shut off the traffic without him touching his server. And since the gateway is authenticated, even the gateway URL leaking doesn't expose his endpoint.

Why Turnstile instead of accounts? The features are toys: a chat page and some arcade games. Forcing login would kill them, and standing up auth infrastructure to protect a friend's GPU is disproportionate. Turnstile is free, invisible in this configuration, and moves the cost of abuse from "free curl loop" to "defeating a browser challenge every 10 minutes."

Why the HMAC session layer instead of verifying Turnstile per request? Turnstile tokens are single-use, and running the widget before every LLM call would add latency to each message. Exchanging one Turnstile pass for a 10-minute signed session amortizes the challenge across a whole play session. Verification is a stateless HMAC check: no session store, no KV reads, just crypto.subtle.verify in the function.

Why a Pages Function at all, when the AI Gateway is already a proxy? Keys. The browser can't hold the LiteLLM key or the gateway token, so something server-side has to inject them. Once that function exists, it's also the right place for origin checks, input caps, and response normalization, things the gateway doesn't know about my app.

Why LiteLLM on the far end? It makes the local model boring. Qwen behind LiteLLM speaks the same protocol as everything else, so if the box is down, swapping LLM_GATEWAY_URL to any other OpenAI-compatible endpoint is a config change, not a code change.

Conclusion

The whole thing is two Pages Functions, one client helper, one dashboard gateway, and zero servers on my side. A public static site gets LLM features backed by a friend's local model, with the keys server-side, the traffic observable by both parties, and the abuse surface reduced to "must be a real browser, on my site, at most this fast, at most this big."

If you're fronting any self-hosted model for public use, the pattern generalizes: OpenAI-compatible upstream, AI Gateway for observability and the kill switch, a thin function for secrets and caps, and Turnstile exchanged for a signed session so the challenge cost is paid once per session instead of per request.

Tags:

Enjoyed this post? Give it a clap!