10

In IMAP, message numbering is based on when a message is placed into a folder (i.e. first message in the folder is 1, second message in is 2, etc.). However, this ordering has no regard for a message's received date. What I'm concerned about are messages older than message number 1 of the folder, i.e. a message that's been moved into a folder by a user manually (and not by when it was received into the system).

I cannot just get every message's receivedDate, as this is horrendoulsy inefficent.

I thought I could do a JavaMail search to get all messages with a received date older than the first message's received date, (and likewise then do a search to get all message with a recieved date newer than the first message's received date... which would be many, but while I need to process ALL of the older ones, I only need to process a few of the newer ones, (and by process, I mean download its headers).

However, when I am using the search, it just doesn't seem to work right.

        Message[] messages = csqFolder.getMessages();           

        if (messages != null && messages.length > 0) {

            Date receivedDate = messages[0].getReceivedDate();
            log.trace("Message 1 receivedDate: <" + receivedDate.toString() + ">");
            SearchTerm msgsOlderThanOETFirst = 
                new ReceivedDateTerm(DateTerm.LT, receivedDate);

            SearchTerm msgsNewerThanFirst = 
                new ReceivedDateTerm(DateTerm.GT, receivedDate);

            Message[] oldMsgs = csqFolder.search(msgsOlderThanOETFirst, messages);
            log.trace("Size of oldMsgs: <" + oldMsgs.length + ">");
            Message[] newMsgs = csqFolder.search(msgsNewerThanFirst, messages);
            log.trace("Size of newMsgs: <" + newMsgs.length + ">");

However, when I run these searches, it does not seem to give the correct results.

On a folder with 8 messages, 7 of which have a received date time of around 12:00 pm on Aug 5, but the first message in the folder which is at 4:00 pm on Aug 5, we will see the following:

Message 1 receivedDate: <Fri Aug 05 16:46:57 CDT 2011>
Size of oldMsgs: <0>
Size of newMsgs: <7>

However, ALL of the remaining seven messages are older than the first message... they should ALL be in oldMsgs.

That said, IF I put a message from the previous day (Aug 4) in that folder, then the search will coreectly work... for that ONE message. It's like the search only works on a day-by-day basis, not down to the second...

I should note that in the term msgsOlderThanOETFirst, I originally was using LE (hence the name), however, that would flip the above resutls -- ALL the messages would then be found in oldMsgs.

Can anyone else confirm if this behavior is true, and if so, would it point to a bug in Java?

I tried to look into the source code of search but I think I was getting a command line version of it and not the kind that the javamail package uses....

I'm using MS Exchange 2007 with SP1, and JavaMail 1.4.3.

Thanks for any suggestions.

UPDATE: I don't think my reading of the comparator order is wrong. Look the answer given in the following thread: java imap fetch messages since a date

Suppose we use that answer, and we have our boundaries from 7/15/2011 to 9/15/2011. Then we have the search compare a message with a given date of 8/4/2011. Then we would have the following:

                                              8/4/2011  <     9/15/2011
SearchTerm olderThen = new ReceivedDateTerm(ComparisonTerm.LT, someFutureDate);
SearchTerm newerThen = new ReceivedDateTerm(ComparisonTerm.GT, somePastDate);
                                              8/4/2011  >     7/15/2011

This would evaluate to true in both cases, which is what we expect, since the date is in the desired range.

Likewise, my code is as follows:

    SearchTerm msgsOlderThanOETFirst =    8/4/2011  < 8/5/2011
            new ReceivedDateTerm(DateTerm.LT, receivedDate); --> TRUE
    SearchTerm msgsNewerThanFirst =       8/4/2011  > 8/5/2011
            new ReceivedDateTerm(DateTerm.GT, receivedDate); --> FALSE

The above evaluations to TRUE and FALSE are what I am expecting, and will receive. However, If we take a given message at 8/5/2011 12:00:00, we get the following:

    SearchTerm msgsOlderThanOETFirst =    8/5/2011 12:00:00  < 8/5/2011 16:46:00
            new ReceivedDateTerm(DateTerm.LT, receivedDate); --> TRUE?
    SearchTerm msgsNewerThanFirst =       8/4/2011 12:00:00  > 8/5/2011 16:46:00
            new ReceivedDateTerm(DateTerm.GT, receivedDate); --> FALSE?

Except, no -- we don't get that -- instead, we get the reverse..... it's easy to think yourself in circles at this point, but... I've double-checked this several times. Is there something wrong with JavaMail, or am I utterly confused? Please correct my confustion if it is the latter!

Community
  • 1
  • 1
Jon
  • 952
  • 1
  • 11
  • 17
  • In your code sample you are getting the messages for the variable folder but performing a search on the csqFolder. Is this just a typo on your example? – bstick12 Aug 10 '11 at 10:30
  • You could try using a differnt implementation of the JavaMail API for which the source is available so you could debug the issue e.g. http://jarvana.com/jarvana/archive-details/org/apache/geronimo/specs/geronimo-javamail_1.4_spec/1.7.1/geronimo-javamail_1.4_spec-1.7.1.jar. No guarantees of the same behavior between implementations tho. – bstick12 Aug 10 '11 at 10:35
  • Yes that was a typo. Fixed. Well, I'm leaning toward the notion that this is a bug right now, and if it is, it doesn't pay for me to try and debug. If a fix for this was put in an official release of JavaMail, I could use it, however. – Jon Aug 10 '11 at 13:19
  • Can you provide an output where put display the Recieved Date of each of the messages in the Message[] – bstick12 Aug 10 '11 at 14:00
  • I do have such an output. In my test of 7 messages, it shows all the messages that I am expecting to be in OldMsg to actually be in newMsgs. And obviously oldMsgs is empty. What are you looking for specifically? I can't give you that exact output anymore without creating a new test (messages in example have since been deleted). – Jon Aug 10 '11 at 16:00
  • Can you loop thru the message array and log the recievedDate from each of the messages in the arrary. – bstick12 Aug 11 '11 at 08:21
  • Yes, I did do this. The messages displayed are what I expected -- they're from the folder in question, but, not according the search term. I didn't show this when I made the question, although those messages no longer exist, so I can't go back and display that output. However, there was a roughly 5 hour gap in time between the oldest and newest message. – Jon Aug 11 '11 at 15:16

1 Answers1

8

According to JM developer Bill Shannon, this is a limitation of the IMAP protocol:

from    Bill Shannon 
to  Jonathan Hanson 
cc  javamail_ww@oracle.com
date    Wed, Aug 10, 2011 at 11:55 AM
subject Re: Bug with searching by ReceivedDateTerms
mailed-by   oracle.com
    Important mainly because of your interaction with messages in the conversation.

hide details 11:55 AM (16 minutes ago)

The searching is being done by the IMAP server, and you're running into a limitation of the IMAP protocol.
Jon
  • 952
  • 1
  • 11
  • 17
  • 1
    The limitation being that IMAP SEARCH BEFORE only accepts dates (without time), not date+time, right? (To quote RFC 3501: "BEFORE - Messages whose internal date (disregarding time and timezone) is earlier than the specified date.") – David Dec 18 '14 at 13:20
  • 1
    What is the solution of this problem? I really want to filter mails based on time, I want mails received only in last 2 mins or so, how can to do it? – Paresh Nov 21 '16 at 12:54
  • The solution is to not use IMAP searching, it is rubbish. The best solution would be to get IMAP protocol updated or else use a different protocol (like MAPI -- which isn't available in JavaMail). If you must use IMAP as is, you're better off working under the understanding that IMAP folders function as a FIFO queue. Keep a memory map of your messages and track new ones every 2 min. – Jon Apr 06 '17 at 20:14