[Users] Conditional requests for RSSyl

[Ivan Krylov]

Hello again,

Currently, RSSyl always makes unconditional requests for feed content.
While feed_update() in src/plugins/rssyl/libfeed/feed.c does accept a
last_update argument:

>> if( last_update != -1 ) {
>> 	curl_easy_setopt(eh, CURLOPT_TIMECONDITION,
>> 	                 CURL_TIMECOND_IFMODSINCE);
>> 	curl_easy_setopt(eh, CURLOPT_TIMEVALUE,
>>	                 (long)last_update);
>> }

...the function is only called with -1 as the time condition, from
rssyl_fetch_feed() -> rssyl_fetch_feed_thr() in
src/plugins/rssyl/rssyl_update_feed.c:

>> ctx->response_code = feed_update(ctx->feed, -1);

I know two (relatively small) independent feed operators who banned
users of RSSyl as a result: once by User-Agent and once by IP address.

Based on information from [1,2], I prepared a series of patches to let
RSSyl collect and store the Last-Modified: and ETag: header values,
send them back when fetching RSS feeds, and skip parsing when receiving
a 304 reply. Claws-Mail is a new codebase for me, so while the patches
didn't break my own feeds, they could probably be improved. Currently,
even the Last-Modified: header is stored as a string in order to avoid
parsing in case the server cares too much about time zones. Any advice
on where it might be a good place to handle the Retry-After: header
that some feed operators return together with an HTTP 429 error code?

I'm sure there's a flip side to this, e.g., some servers that don't
track their cache keys properly and therefore reply with a 304 to
conditional requests when the content did really change. Perhaps the
attached patches could be combined with the user interface parts of bug
1973 [3] to let the user choose whether to perform conditional requests?

-- 
Best regards,
Ivan

[1]
https://rachelbythebay.com/fs/help.html

[2]
https://utcc.utoronto.ca/~cks/space/blog/web/SyndicationFeedsRatelimitedNow

[3]
https://www.claws-mail.org/bugzilla/show_bug.cgi?id=1973
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-libfeed-store-and-send-ETag-Last-Modified.patch
Type: text/x-patch
Size: 6154 bytes
Desc: not available
URL: <http://lists.claws-mail.org/pipermail/users/attachments/20250316/5498ce2c/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-RSSyl-persist-and-reuse-ETag-Last-Modified.patch
Type: text/x-patch
Size: 5221 bytes
Desc: not available
URL: <http://lists.claws-mail.org/pipermail/users/attachments/20250316/5498ce2c/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-libfeed-change-User-Agent-to-Claws-Mail-version.patch
Type: text/x-patch
Size: 1423 bytes
Desc: not available
URL: <http://lists.claws-mail.org/pipermail/users/attachments/20250316/5498ce2c/attachment-0002.bin>