I'm getting blocked when requesting the Blog RSS Feed

Support
1

I’m using FreshRSS to accumulate multiple blogs into one place. Ever since the Cloudflare Settings got changed, I’m getting a cURL error 22: The requested URL returned error: 403 [https://pretendo.network/blog/feed.xml] in my FreshRSS protocols.

Note, I’m requesting the feed URL once per day - and my root server doesn’t request anything else from Pretendo. So I’m in the opinion that Cloudflare could be blocking me.

I don’t want to publicly say my ipv4 address, but I’m willing to give it out privately to resolve this issue.

(If this is the wrong category, then I’m sorry - I didn’t know where to put it and “Support” sounded correct.)

2

It does indeed sound like you’re getting blocked by Cloudflare. I can reproduce this by running curl https://pretendo.network/blog/feed.xml, which 403s.

I’m not sure whether we can loosen the Cloudflare restrictions at this time due to concerns over DDoS attacks.

I’ve heard of a trick before where you can use Google Feedburner to proxy an RSS feed that’s protected by Cloudflare, and Google’s bot might get past the block. Maybe try that? (Edit: I just tried it myself and it doesn’t seem to be working…)

3

You could set up a WAF exclusion rule for curl’s user-agent string. If you have Bot Fight enabled, though, there’s no getting around that without a paid subscription. I’m not the biggest fan of it since it has a bad tendency to block legitimate services, and I write my own WAF block rules instead. I use this in conjunction with Comodo’s ModSec ruleset, and the AbuseIPDB blocklist for ConfigServer Firewall. If your hosting grants you root, these might be worth looking into.

I’m also guessing this is why Feedburner didn’t work. Google’s crawler is whitelisted, but their satellite services usually get snared by Bot Fight. If you have a CF subscription, you should have access to Super Bot Fight, which will let you set up custom exclusions.

1 Like
4

Thanks for the tips, I’ll send this along to our Cloudflare admins :+1:

2 Likes
5

This is likely due to one of our custom WAF rules we have setup on Cloudflare to prevent abuse. Unfortunately I can’t go into much more detail as that would require detailing what our rules look like. At this time we do not wish to add any additional bypass rules as that just opens additional holes

2 Likes
6

Yeah, we enabled Bot Fight mode a few days ago.
We have one particular bad actor (the pissed off S-kid variety) who has a bad habit of launching DDoS attacks.
I believe we have Super Bot Fight? I was able to allow consoles through.
I think I’ll allow the RSS feed’s URL through and add a 24h cache so attacks won’t reach the origin server

I’ll let you know when I get set that up (away from my PC rn)

3 Likes
7

Were you able to allow the feed URL yet? Because if so, then I’m still getting blocked.

8

When I try to access the URL in the OP, it downloads a feed.xml file normally without any errors - what are you trying to do with the URL?

9

The difference is in requesting it with a Web browser vs with an automated tool like curl. Cloudflare only blocks it in the second case with their black-box bot-detection algorithm.

1 Like