Running Your Own LLM Endpoint with Cloudflare Tunnels and Ollama (Part 3)

July 12, 2026·9 min read

This is the third stop in a series that started with a friend's GPU. Part 1 pointed OpenCode at his local Qwen. Part 2 wired this blog's AI features to that same box through a Cloudflare AI Gateway, with Turnstile keeping bots off his GPU. It all worked great, right up until his machine went dark for a weekend and every AI feature on the site started returning shrugs.

His box can go down whenever it wants, and I do not get a vote. So I wanted my own copy of the model running on my own hardware, ready to take over when his machine naps. This post is how I did that with Ollama and a Cloudflare Tunnel, locked down so I am the only one who gets to spend my electricity.

The Goal

Three things, in plain terms:

1. Serve a Qwen model from the desktop under my desk, the one with the RTX 4080 Super in it.
2. Reach it from blog.chrishouse.io, which runs on Cloudflare's edge and has no idea what my home network looks like. No port forwarding, no static IP, no exposing my home address to the internet.
3. Flip between my box and my friend's box on demand, without a redeploy, and see instantly whether the one I picked is actually alive.

The whole thing had to keep the existing rule from Part 1: no keys in the browser, and nobody but my own site gets to use the GPU.

The Building Blocks

Ollama runs local models and hands you an OpenAI-compatible endpoint at http://localhost:11434/v1. That compatibility is the whole trick. From the blog's point of view a box under my desk looks identical to any other POST /v1/chat/completions provider, which is exactly what Part 1's proxy already speaks.

Cloudflare Tunnel (the cloudflared daemon) makes an outbound-only connection from my PC to Cloudflare's edge and publishes a local port at a public hostname. My home IP stays hidden, my router stays closed, and llm.chrishouse.io resolves to a machine that never accepted an inbound connection in its life.

Cloudflare Access sits in front of that hostname and rejects anyone without a valid service token. A public tunnel to an unauthenticated Ollama is a free GPU for the entire internet, so this part is not optional.

GrowthBook holds a single feature flag, llm-backend, whose value is house or fever. The blog reads it to decide which upstream to call. Same tool that already runs the flag demos on this site, now doing something I actually depend on.

The Architecture

Browser (Neon GPT)
  │  POST /api/llm   { messages, backend }
  ▼
Cloudflare Pages Function  (origin gate → session check → sanitize → pick backend)
  │
  ├── backend = "fever" ─► Cloudflare AI Gateway ─► LiteLLM ─► friend's Qwen   (Part 1)
  │
  └── backend = "house" ─► https://llm.chrishouse.io
                              │  CF-Access-Client-Id / CF-Access-Client-Secret
                              ▼
                          Cloudflare Access   (403 without the token)
                              │
                              ▼
                          cloudflared tunnel  (Windows service)
                              │  http host header rewritten to localhost:11434
                              ▼
                          Ollama ─► Qwen (my RTX 4080 Super)

The flag decides which branch runs. The proxy holds both sets of credentials server-side and picks one per request. The browser never learns either endpoint.

Standing Up Ollama

Installing Ollama and pulling a model is two commands, and I already had a few models on disk. The part worth writing down is which model to serve, because I got this wrong first.

Qwen3 is a reasoning model. Left alone it burns a few hundred tokens thinking before it says hello. Part 1 disabled that with chat_template_kwargs: { enable_thinking: false }, and my friend's LiteLLM honored it. Ollama's OpenAI-compatible endpoint does not. I tried every switch (think:false, the /no_think prompt token, the enable_thinking kwarg) and Ollama's /v1 route ignored all of them. The answer kept landing in a reasoning field with content sitting there empty, which the blog then rendered as a blank message.

So I served a model that does not think out loud:

ollama pull qwen2.5-coder:14b

At a Q4 quant it is about 9 GB, which sits entirely in the 4080 Super's 16 GB of VRAM with room to spare, and it answers in well under a second. Coder-tuned, but for a general chat widget it holds a conversation fine. A quick sanity check straight against Ollama:

curl -s http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen2.5-coder:14b","messages":[{"role":"user","content":"say hi"}],"stream":false}'

Real content, finish_reason: stop, no empty replies. Good enough to put behind a tunnel.

The Tunnel

cloudflared needs a login (a browser SSO handshake against your Cloudflare account), then a named tunnel and a DNS route:

cloudflared tunnel login
cloudflared tunnel create theblog-llm
cloudflared tunnel route dns theblog-llm llm.chrishouse.io

Then a config file. This is where the second gotcha lives. Recent Ollama refuses any request whose Host header is not localhost, a sensible anti-drive-by measure. Through the tunnel Ollama sees Host: llm.chrishouse.io and answers with a fast, confusing 403. The fix is one line that rewrites the host header back to what Ollama expects:

tunnel: theblog-llm
credentials-file: C:\Users\Chris\.cloudflared\<tunnel-uuid>.json
ingress:
  - hostname: llm.chrishouse.io
    service: http://localhost:11434
    originRequest:
      httpHostHeader: localhost:11434
  - service: http_status:404

cloudflared tunnel run theblog-llm and the hostname goes live. To survive reboots it runs as a Windows service. One warning there: a bare cloudflared service install registers the service with no run arguments, so it starts, prints its help text, and exits half a second later. I set an explicit command on the service so it runs my tunnel with my config:

"C:\Users\Chris\tools\cloudflared.exe" --config "C:\Users\Chris\.cloudflared\config.yml" tunnel run theblog-llm

Automatic startup, comes back after a reboot, done.

Locking It Down with Access

At this point llm.chrishouse.io is a public URL pointed at an LLM with no authentication. Anyone who finds it gets free inference on my GPU. Cloudflare Access closes that with a service token, which is a client ID and secret built for machine-to-machine calls.

I created the service token, a self-hosted Access application on the hostname, and a policy that allows only that token. The blog's proxy presents the token on every call to the house backend:

const headers = { "Content-Type": "application/json", Authorization: `Bearer ${key}` }
if (env.LLM_ACCESS_CLIENT_ID_HOUSE && env.LLM_ACCESS_CLIENT_SECRET_HOUSE) {
  headers["CF-Access-Client-Id"] = env.LLM_ACCESS_CLIENT_ID_HOUSE
  headers["CF-Access-Client-Secret"] = env.LLM_ACCESS_CLIENT_SECRET_HOUSE
}

The proof is two curls. Without the token, Access stops the request at the edge before it ever reaches my house:

$ curl -s -o /dev/null -w '%{http_code}\n' https://llm.chrishouse.io/v1/chat/completions -d '{}'
403

$ curl -s https://llm.chrishouse.io/v1/chat/completions \
    -H "CF-Access-Client-Id: <id>" -H "CF-Access-Client-Secret: <secret>" \
    -d '{"model":"qwen2.5-coder:14b","messages":[{"role":"user","content":"reply OK"}],"stream":false}'
{"choices":[{"message":{"content":"OK"}}], ...}

403 for the world, 200 for the blog. That is the whole security model in two lines of output.

GrowthBook Is the Toggle

The switch itself is a single GrowthBook flag named llm-backend, and its value is either house or fever. That flag is the whole point. Nothing in the browser decides which model you talk to, and I never touch an environment variable to change it. I flip the flag and the entire site follows.

A small endpoint reads the flag and writes it through the GrowthBook Admin API, so the choice is shared state instead of a setting stuck in one person's browser tab. Change it once and everyone hitting the site moves to the same backend at the same time. When my friend's box comes back online, one flip sends everyone back to it.

Behind the flag, the proxy still has to turn the word house or fever into a real endpoint with the right credentials. I kept the naming boring: the base environment variables are the fever backend from Part 1, and a parallel set with a _HOUSE suffix is my box. A tiny resolver maps the flag value to one of them:

resolve(flag):
    if flag is "house":            # my box
        endpoint = the *_HOUSE vars (local Qwen via the tunnel)
        auth     = Cloudflare Access service token
    else:                          # fever, the Part 1 path
        endpoint = the base vars (feverdreams gateway)
        auth     = Cloudflare AI Gateway token

    return endpoint, model, auth headers

Same two lines of config, two different locks. Fever carries the AI Gateway token from Part 1, house carries the Access service token, and the flag is the only thing that decides which one runs. Adding my box was a config change, and the Part 1 path never moved.

Where You Can Test It

Go poke at Neon GPT. Up in the header there is a House / Fever toggle sitting next to the connection dot.

• House routes your message to the Qwen on my desk, through the tunnel, through Access.
• Fever routes it to my friend's box, the Part 1 path.

Flip the toggle and two things happen. It writes the GrowthBook flag, then it fires a tiny health-check ping at the backend you picked and repaints the dot: green if that model answered, red if it did not. So if you switch to House and the dot goes red, my desktop is off, or Ollama crashed, or I am playing a game and told it to stop hogging the GPU. The chat degrades to a clear "that model is offline" message instead of hanging.

It is a real switch on a real thing. Be nice to my electric bill.

Design Decisions

Why a tunnel instead of port forwarding? Port forwarding means opening my router to the internet and handing out my home IP, then hoping nothing else on my network has a bad day. The tunnel is outbound-only, hides the IP, and gives me a real hostname with a real certificate for free. Less attack surface and less DNS busywork.

Why a service token instead of a login page? The only caller is a server, my Pages Function. Human login flows are the wrong shape for that. A service token is two secret headers, it lives in the Pages environment next to the other keys, and it never involves a browser. Access checks it at the edge, so a bad request dies in Cloudflare's datacenter and never touches my house.

Why a feature flag instead of just editing an env var? Because an env var edit is a redeploy, and redeploys are slow when a model is down and the chat is broken right now. The flag moves everyone in one API call, and I can flip it from my phone. It also means the fallback is a product decision I can make live, not a code path I have to ship.

Why a non-reasoning model? Because Ollama's /v1 endpoint would not let me turn Qwen3's thinking off, and a reasoning model that spends its whole token budget thinking returns an empty chat bubble. A coder-tuned 14B that fits fully in VRAM and answers immediately is the better trade for a chat toy, and it sidesteps the whole fight.

Why keep my friend's box as the default? His is a beefier 27B and it is usually up. Mine is the safety net for when it is not. Default to the better model, fall back to the reliable one, flip in a second when reality disagrees.

The Gotchas, So You Skip Them

A short list of things that ate my afternoon:

• Ollama 403 through the tunnel. Host header. Rewrite it to localhost:11434 in the ingress config.
• Empty chat replies. Reasoning model. Ollama's /v1 ignores every thinking switch, so serve a non-reasoning model or read the reasoning field as a fallback.
• Service starts then dies. cloudflared service install leaves no run arguments. Set the service command explicitly.
• A zombie worker. wrangler pages dev runs the real worker as a separate workerd process. A dev server I killed days ago left one holding the port, so every "fresh" restart was quietly serving stale environment variables and swearing my new config did not exist. Kill the orphaned workerd, confirm the port is actually free, then start over. I lost an hour to this one and I want that hour back.

Conclusion

The shape from Part 1 held up. An OpenAI-compatible upstream, a thin Pages Function for secrets and sanitizing, and now a second upstream that happens to be a computer in my house. Ollama makes the model boring, the tunnel makes it reachable without opening my network, Access makes it mine alone, and one GrowthBook flag turns "which model" into a switch instead of a deploy.

If you have a decent GPU sitting idle and a public site that wants an LLM, this is a weekend of work and zero dollars a month. Serve the model, tunnel it out, lock it behind a token, and give yourself a flag so the day your dependency goes dark, you are one toggle away from your own hardware picking up the slack.

Tags:

Enjoyed this post? Give it a clap!