I run NewsBlur[0] and I’ve been battling this issue of NewsBlur fetching 403s across the web for months now. My users are revolting and asking for refunds. I’ve tried emailing dozens of site owners and publishers and only two of them have done the work of whitelisting their RSS feed. It’s maddening and is having a real negative effect on NewsBlur.
NewsBlur is an open-source RSS news reader (full source available at [1]), something we should all agree is necessary to support the open web! But Cloudflare blocking all of my feed fetchers is bizarre behavior. And we’re on the verified bots list for years, but it hasn’t made a difference.
Let me know what I can do. NewsBlur publishes a list of IPs that it uses for feed fetching that I've shared with Cloudflare but it hasn't made a difference.
I'm hoping Cloudflare uses the IP address list that I publish and adds them to their allowlist so NewsBlur can keep fetching (and archiving) millions of feeds.
RSS is an essential component to modern web publishing and it feels scary to see how one company’s inconsideration might harm its already fragile future. One day cloudflare will get big enough to be subject to antitrust regulation and this instance will be a strong data point working against them.
Clearly they are not 100% consenting, or at best one of them (the content publisher) is misconfiguring/misunderstanding their setup. They enabled RSS on their service, then setup a rule to require human verification for accessing that RSS feed.
It's like a business advertising a singles only area, then hiring a security company and telling them to only allow couples in the building.
If Cloudflare was honest and upfront about the tradeoffs being made and the fact that it's still going to require configuration and maintenance work they'd have significantly less customers.
Newsblur was the first SaaS I could afford as a student. I have been subscriber for something like 20 years now. And I will keep doing it to the grave. Best money ever spent.
Can't recommend Newsblur enough. I have been a customer since Fastladder was shut down. I love their integration of being able to use pinboard.in within the web interface to bookmark articles. An essential part of my web productivity flow.
I used to get my internet from a small local ISP, and ip blacklisting basically means no one in our zipcode could have reliable internet.
These days, the 10-20% of us with an unobstructed sky view switched to starlink and didn’t look back.
The thing is, both ISPs use CGNAT, but there’s no way cloudflare is going to block Musk like they do the mom and pop shop.
Anyway, apparently residential proxy networks work pretty well if you hit a spurious ip block. I’ve had good luck with apple private relay too.
I’m hoping service providers realize how useless and damaging ip blocking is to their reputations, but I’m not holding my breath. Sometimes I think the endgame is just routing 100% of residential traffic through 8.8.8.8.
NewsBlur is an open-source RSS news reader (full source available at [1]), something we should all agree is necessary to support the open web! But Cloudflare blocking all of my feed fetchers is bizarre behavior. And we’re on the verified bots list for years, but it hasn’t made a difference.
Let me know what I can do. NewsBlur publishes a list of IPs that it uses for feed fetching that I've shared with Cloudflare but it hasn't made a difference.
I'm hoping Cloudflare uses the IP address list that I publish and adds them to their allowlist so NewsBlur can keep fetching (and archiving) millions of feeds.
[0]: https://newsblur.com
[1]: https://github.com/samuelclay/NewsBlur