For today’s blog, I just want to jot down a quick note. I set up a local LLM server, managed it through LiteLLM, and exposed it to the outside world via a domain on a Cloudflare Tunnel. Then I noticed that the web api kept getting a 524 error.
When I dug through the logs, I found that the request from the web API to the local LLM server (through Cloudflare) was actually getting a 403. The reason it surfaced as a 524 is that my .NET API wasn’t written well: after receiving the 403, it just kept waiting instead of failing fast. As a result, Cloudflare gave up and returned a 524 after the request exceeded its timeout of roughly 100-125 seconds.
I go to Cloudflare Dashboard > your domain > Security > Analytics (Traffic / Events), you’ll see that the request was being blocked by a Managed Rule.
I think, Cloudflare classified our traffic from .NET API as an AI crawler and blocked it. It might be a bug on their side, but we still have to fix it first. So I added a Security Rule (a custom WAF rule) to make it let our traffic through for now. The steps are as follows:
📌Go to the Cloudflare Dashboard > your domain > Security > Security Rules.
📌Then add our own rule. I’m bypassing the checks for my domain plus specific paths, and in this case I used two: /chat/completions and /embeddings.
(http.host eq "ai.exampledomain.dev" and http.request.uri.path in {"/chat/completions" "/embeddings"})
📌Then, for the Action, select Skip, and tick this option as well:
- All remaining custom rules
- All managed rules
- All Super Bot Fight Mode Rules
-> User Agent Blocking
-> Browser Integrity Check
-> Security Level
-> Rate limiting rules (Previous version)
-> Managed rules (Previous version)
📌Now let’s wait and see the result. It’s no longer being blocked.
finally: Cloudflare’s UI has changed a lot. I spent a while looking for where to add a domain to the Tunnel and couldn’t find it, 555 . But the visualization is really nice, and I like it.
