[Users] [Bug 4170] Problem accessing RSS feeds for sites using Cloudflare with TOR proxy

[noreply at thewildbeast.co.uk]

https://www.thewildbeast.co.uk/claws-mail/bugzilla/show_bug.cgi?id=4170

--- Comment #2 from Removed after GDPR request <removed-gdpr at example.com> ---
I am familiar with CF's captcha challenge and how it is enabled as I manage a
few websites which also use this. The page which you show [1] explains how to
enable it selectively by creating a page rule in CF's control panel. However
there are a few more things to consider here. You may be familiar or not with
some of them but I will still list them as this may be helpful info:

- Tor browser does *not* use Cloudflare's browser plugin which helps bypass
CF's own challenge. As Cloudflare is a form of centralized instance which
decrypts traffic, that would be against the whole concept of the Tor browser.

- Most websites using CF use its free plan on which the number of page rules is
very restricted (only 3), so it is unlikely to waste a rule just to block RSS
access through Tor, especially considering that:

- It is in website's interest to have their RSS feed crawlable (better SEO), so
there is no reason to explicitly block the RSS feed. Also when a particular
website presents a challenge it is for all pages, not just for the feed.

- It is possible to enable the captcha challenge not per URL pattern but by IP
address range which may be either a single IP address, a CIDR notation or also
there is a built in preset called "Tor", i.e. it is possible to present
challenge only to Tor network users - this is what many do and the proof about
it is the fact that the same page opens when Tor network is not used

- CF can automatically present the challenge based on "suspicious traffic",
regardless of settings for particular website. That may be based on previous
history from a particular IP address/range which CF has logged.

- Opening the same CF managed website in Tor web browser and in a regular (e.g.
Chrome) web browser which is configured to use Tor proxy doesn't give the same
result, even if it is configured similar to the Tor browser, even if multiple
different IP addresses are tested in both browsers. Tor browser will open the
URL, the other browser will hit a captcha. How does CF decide when to present
the challenge? - I don't know. I think it may be a combination based on the
HTTP request headers. Although I have tried to copy literally the HTTP request
headers of Tor browser and use them in curl, curl also hits the challenge.
However I am not sure if this is the right thing to do because of this special
__cfduid cookie which I am not quite sure how to handle in curl. In any case
this has nothing to do with JavaScript as in both browsers (Tor and Chrome) JS
is completely disabled.

Considering this info I was hoping that someone who is more experienced may
look into it. As the network (Tor IP range) is the same, perhaps fine tuning
the rest of the parameters will if not remove completely at least reduce the
number of 403 responses (the captcha challenge hits).

-- 
You are receiving this mail because:
You are the assignee for the bug.