4

When reading my RSS feed with the Thunderbird feed reader, some entries are duplicated. Google Reader does not have the same problem.

Here is the faulty feed: http://plcoder.net/rss.php?rss=Blog

There is a problem, but where?

I added a GUID, but the problem remains. Other feeds do not duplicate like mine, so I will do rework on this module and replace this old good code.

Conclusion: I completely reworked the RSS generator code, and it's OK. I think I was using a very old version of RDF.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Cédric Girard
  • 3,358
  • 7
  • 37
  • 52
  • Why the "offtopic" tags added on my question? I try to solve a development software problem about RSS specification. And I put my own rss feed because the bug is on my CMS. Could anyone explain me why offtopic? – Cédric Girard Oct 08 '08 at 14:10
  • Because at first, second, ***and third*** glance it looks like a support question about Thunderbird (and would thus belong at [Super User](https://superuser.com/tour)). My interpretation after the ***fourth*** glance is that you have some software that generates the XML content for some RSS feed (or maybe some script or interceptor software that modifies an RSS feed). Whatever it is, can you make it clear what you are trying to do (but ***without*** "Edit:", "Update:", or similar - the question should appear as if it was written today)? – Peter Mortensen Jun 06 '21 at 14:07
  • For instance, the very first sentence in the question could answer the question *"Why is this not a support question about using RSS feeds in Thunderbird?"*. For example, *"My program generates an RSS feed, deploys it to hosting provider X, but it does not work as expected with Thunderbird's built-in RSS reader."* (presuming that is the case) – Peter Mortensen Jun 06 '21 at 14:18

4 Answers4

7

Try adding a <guid> tag to each item, giving it a permalink. i.e.:

<item rdf:about="http://plcoder.net/?doc=2134&amp;amp;titre=mon-pc-se-la-pete">
  <link>http://plcoder.net/?doc=2134&amp;amp;titre=mon-pc-se-la-pete</link>
  <guid>http://plcoder.net/?doc=2134&amp;amp;titre=mon-pc-se-la-pete</guid>
  ...
</item>

Without a GUID, if any of the content in the post changes, your RSS aggregator might think that it is a new post. With the GUID, even if the content of that item changes, your RSS aggregator should just update the post, instead of treating it as a new item.

Kip
  • 107,154
  • 87
  • 232
  • 265
  • The `pubDate` also influences the RSS aggregator. An item with the same `guid` but a different `pubDate` will be treated as a change and gets repeated sometimes. – David Silva-Barrera Jan 15 '22 at 21:05
2

At least with Thunderbird 2.0.0.21, the problem is that Thunderbird doesn't seem to respect GUID tags, but it does respect the channel's pubDate-tag. Thus if pubDate is more recent than with the last reading, Thunderbird will read all entries (it seems).

I don't know what would happen if the channel's pubDate-tag is missing though...

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
1

I have experienced these issues with some of my own feeds. I started off with a list of entries like this:

Item A
Item B
Item C

The client downloads them and everything is fine. Then I add a new item, so the feed reads as:

Item D
Item A
Item B

D shows up in the reader.

But then I decide I don't want that item, so the list reverts to:

Item A
Item B
Item C

When Thunderbird reads this, it'll count C as a new item. I am using a GUID element, so I doubt that's the problem. I think it's got more to do with Thunderbird's parser not taking older elements into consideration.

The long-winded workaround is to "remember" what items you've already published and have since been pushed off the end of the list by new items. You'll basically need to keep a current list of items in the feed and when you delete items from it, cut it short until there are new items to replace it.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Oli
  • 235,628
  • 64
  • 220
  • 299
1

This is explained in Thunderbird's documentation (under "Troubleshooting FAQ"):

Q: Why are feed messages sometimes duplicated?

A: Feed messages with identical content, but different unique ids, are not detected as duplicates. See this post for many more details.

The linked post for reference:

  1. Atom feeds (mandatory) have a unique ID; RSS feeds (not mandatory) usually have a unique GUID. For RSS feeds without a GUID, an attempt is made to create a unique ID from mandatory parts of the feed item.
  2. All downloaded feed messages have a record with this ID stored in feeditems.rdf and exist there as long as they exist in the publisher's file, with that ID. If the publisher removes a message with the ID from their file, after 24 hours the feeditems.rdf cache is also purged (on get messages biff).
  3. If a publisher reuses an ID after it has been purged, you will get a duplicate (if the content is identical). This is an abuse of the intent behind unique IDs and is the publisher's error.
  4. If a publisher reuses an ID before it is purged, and the content is different, you will not see the new content, as it will be treated as a duplicate. Thunderbird does not use the <updated> tag currently and its misuse by publishers may make it difficult to implement.
  5. If you view the source (Ctrl + U) of two apparent duplicates, you will note the Message-Id header. If two apparent duplicates have different Message-Id values, then they are not duplicates regardless of potential identical content. Thunderbird does not distinguish duplicate content.

If you want extreme debugging, change the Feeds.logging.console preference setting to debug or trace and restart, to see what happens during feed processing.

If you unsubscribe a feed URL, this will clear the feeditems.rdf cache for that feed. If you subsequently resubscribe you will get duplicates of all current items in the publisher's file that also exist in your feed folder.

Compaction has no effect on feed processing; it just removes marked for deletion items from the file. If you delete a folder/move it to trash, it is unsubscribed. Starting with Thunderbird 29, if you drag/drop a folder from one feed account to another feed account, the subscription is retained (but not feeditems). For very old profiles/feed accounts (pre Thunderbird 17), it can be a good idea to create a new feed account and drag folders there (Thunderbird 29 and up), as a fresh feeds.rdf database is created; the penalty is a one-time duplicate possibility.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
user598527
  • 175
  • 13