2

We are developing a mail client written in Java. It has same functionalities like Outlook or Thunderbird, etc. It communicates with the mail server directly. Additionally, our business rules demand that we store all messages in our database and messages should be kept synchronised always. I know that is not very suitable for IMAP, but we must keep everything in our database. Question arises, how to track an IMAP message moved from folder A to folder B? How can we get informed about that? If you remove a message from A, it is deleted from A and it created newly in B, as a result: The UID value of the message is changed. Can we rely on the MessageID found in the headers? I checked some mails servers and see that the message id in the headers remain unchanged. But i have read somewhere, that the messageids can be empty depending on the mail server.

  • Are the MessageID in headers always set, can be cases or mailservers that they leave it blank?
  • Are the MessageID value in headers unique in an IMAP folder?
  • Is it possible that it gets changed when message is moved or folders UIDVALIDITY changed?

  • What about setting a custom header during fetch? When I add a non-standart header name value pair, will it be kept on the mail server or is it possible that non-standart mail heraders will be deleted by mail server? Is it a bad idea applying a non-standart header value?

    IMAPMessage m;
m.setHeader("myHeader", "myValue");

  • There were some suggestions in stackoverflow, it is said to generate a hash including messageId and other parameters such as sender, subject etc, is it a safe approach? We can get conflicts if there is no unique MessageID is provided or no MessageID is provided.

benchpresser
  • 2,171
  • 2
  • 23
  • 41
  • 1
    No, you can't count on messageids being unique or present. You can't add a header as imap messages are immutable. – Max Oct 19 '15 at 14:20
  • 1
    Your best bet is to go with your suggestions you mention at the end - create a hash that combines sender, receiver, subject, date and as many other unique fields as possible. – Fallso Oct 19 '15 at 14:43
  • 1
    And beware, it's pretty simple to create a copy of a message (eg, IMAP COPY command), so even all of that can still lead to duplicates. Messages themselves aren't unique. – Max Oct 19 '15 at 14:45
  • what about setting custom flags? do they remain same when message moved between folders? – benchpresser Oct 21 '15 at 15:17

1 Answers1

0

There are three things you can do.

First, message-id. You can rely on the message-id being present and unique these days if your mode of failure is good enough. In your case, if the message-id is not there and a message is moved, is the failure just that you waste space in the database and/or download the message twice? The wasted space will be small these days.

Second, x-gm-msgid. That's a gmail-specific feature, a 63-bit number that never changes. If two messages have the same x-gm-msgid, they are the same.

Third, the COPYUID response code tells you about moves, but only applies when you do the moving, not when someone else does.

Put together, these should give you a fairly good understanding of how the user's mailboxes change.

arnt
  • 8,949
  • 5
  • 24
  • 32
  • message-id is not always unique. Check the case: you have an email address foo@company.com. You belong to a mail group group@company.com. And someone sends a mail to you and to the group. As a result you will recevie the mail twice. They will both have same message-ids because message-id is set from the sender while sending. If we have same message-ids with two different mails the structure is no more useful. – benchpresser Oct 21 '15 at 15:21
  • @benchpresser are those substantively different? I've seen quite a few MUAs whose threading algorithms treat them as interchangeable and show you just one copy... – arnt Oct 21 '15 at 20:26
  • is it not a good idea to set a custom mail flag and track this value(userflag)? it remains same when message moved in outlook, thunderbird etc. is it commonly supported by mailservers? – benchpresser Oct 21 '15 at 21:08
  • 1
    A couple of widely used servers support a limited number of flags per mailbox, so if you're thinking of having hundreds of differently named flags in the same mailbox, that's going to break quickly. If your app uses a dozen different flags it'll work well. – arnt Oct 22 '15 at 08:43
  • thanks that is very important for me, i planned to keep the folder names as custom flags, each message would have its folder name as a custom flag and a message is moved, i would learn that is a movement by checking the custom flag. Number of custom flags would be equal to folder count. And each message would have only one custom flag. How can i get this limit? – benchpresser Oct 22 '15 at 09:02
  • 1
    The only way to learn the limit is to create a lot of flags. Make a temporary mailbox, make a message in it, loop around storing ever-new flags on the message until one of the UID STORE commands fails with a NO response, see how many flags the test message has, finally delete the message and mailbox. My guess is that it'll be 32 or 128 for the servers that limit this, minus a bit for the system-defined flags. But it's been a few years since I looked at any of that source code... – arnt Oct 22 '15 at 11:10
  • is it supprted by exchange-server? – benchpresser Oct 22 '15 at 15:43