How to fix inaccurate DateTime (minutes) parsed with Jackson/Gson libraries?

Question

I am developping an Android application that dialogs with some Google App Engine web-services.

This application implements a Chat Activity which have very a very simple feature: send text.

During debugging I noticed that the messages I was listing from the server were not displayed in the same order I had sent them on my application. My first thought was that the problem was comming from the server.

At first I checked the raw Json I was receiving:

{
 "messages": [
  {
   "message": "test 3",
   "author": "daniel",
   "message_id": "5724160613416960",
   "sent_at": "2014-11-13T09:42:42.861950"
  },
  {
   "message": "test 2",
   "author": "daniel",
   "message_id": "5649050225344512",
   "sent_at": "2014-11-13T09:42:10.390960"
  },
  {
   "message": "test 1",
   "author": "daniel",
   "message_id": "5178081291534336",
   "sent_at": "2014-11-13T09:41:01.998830"
  }
 ],
 "kind": "company#chatsItem",
 "etag": "\"RUCkC9XynEQNZ2t5E0aa41edXro/xRNtgkWIUbq4zCgmv2iq2fy-UIg\""
}

As you can see, the raw data is correctly ordered. But here comes the funny part. When I add a JSON parser, such as JacksonFactory (or even GsonFactory):

Company.Builder builder = new Company.Builder(AndroidHttp.newCompatibleTransport(), new JacksonFactory(), null);
Company service = builder.build();
ChatsChatCollectionResponse response = service.chats().list(user_id, album_id, token).execute();
List<ChatsChatResponse> messagesResponse = response.getMessages();

Here are the ChatsChatResponse items ordered in the same way as above:

[0] = {com.appspot.com_pany.company.model.ChatsChatResponse@830029063096} size = 4
[0] = {com.google.api.client.util.DataMap$Entry@830029082528}"author" -> "daniel"
[1] = {com.google.api.client.util.DataMap$Entry@830029082552}"message" -> "test 3"
[2] = {com.google.api.client.util.DataMap$Entry@830029082576}"message_id" -> "5724160613416960"
[3] = {com.google.api.client.util.DataMap$Entry@830029082600}"sent_at" -> "2014-11-13T10:57:03.950+01:00"

[1] = {com.appspot.com_pany.company.model.ChatsChatResponse@830029066376} size = 4
[0] = {com.google.api.client.util.DataMap$Entry@830029083616}"author" -> "daniel"
[1] = {com.google.api.client.util.DataMap$Entry@830029083640}"message" -> "test 2"
[2] = {com.google.api.client.util.DataMap$Entry@830029083664}"message_id" -> "5649050225344512"
[3] = {com.google.api.client.util.DataMap$Entry@830029083688}"sent_at" -> "2014-11-13T10:48:40.960+01:00"

[2] = {com.appspot.com_pany.company.model.ChatsChatResponse@830029068008} size = 4
[0] = {com.google.api.client.util.DataMap$Entry@830029084760}"author" -> "daniel"
[1] = {com.google.api.client.util.DataMap$Entry@830029084784}"message" -> "test 1"
[2] = {com.google.api.client.util.DataMap$Entry@830029084808}"message_id" -> "5178081291534336"
[3] = {com.google.api.client.util.DataMap$Entry@830029084832}"sent_at" -> "2014-11-13T10:57:39.830+01:00"

Why is there such a random difference on the "sent_at" field ?

EDIT I forgot to mention that I am not talking about the 1 hour shift that corresponds to TimeZone, but rather on how random the minutes are.

Could you include POJO definition for `ChatsChatResponse`? This is important to know what datatypes are being used. It looks like you are using Jackson-based deserializer, so version of Jackson may also matter: you may want to verify you using latest patch version as well. — StaxMan, Nov 25 '14 at 00:12

score 3 · Answer 1 · answered Nov 19 '14 at 21:32

I'm not sure about the solution, but the explanation seems quite clear: the variation is not random at all, but rather caused by adding everything after the decimal point as milliseconds.

Take the third entry, for example:

2014-11-13T09:41:01.998830

Evidently, the parser reads this as:

hours: 09
minutes: 41
seconds: 01
milliseconds: 998830

Since milliseconds is greater than 1000, it ends up being transformed into 998s 830ms, which when accounting for full minutes is 16m 38s 830ms. Hence:

hours: 09
minutes: 41 + 16
seconds: 01 + 38
milliseconds: 830

Which produces exactly the result you're seeing (modulo timezone adjustment):

2014-11-13T10:57:39.830+01:00

If you have control over the service output (this point is not quite clear from the question wording), the easiest fix would be to output only three digits after the decimal points (i.e. rounding at milliseconds instead of millionths). For the example, this would be 2014-11-13T09:41:01.999.

It might also be possible to fix this on the parser itself, but it would require a bit more research (and ideally, a gist that reproduces the problem).

score 1 · Answer 2 · answered Nov 24 '14 at 13:32

1

try setting a custom date format like

Gson gson = new GsonBuilder().setDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS").create();

answered Nov 24 '14 at 13:32

Rares Barbantan

761
4
9

score 0 · Answer 3 · edited May 23 '17 at 10:33

As @matiash noted, you have a non-standard date format that contains microseconds. The problem is that SimpleDateFormat, used by Jackson, incorrectly parses microseconds.

To alleviate this you should create a custom deserializer that truncates the microseconds to milliseconds:

public class MicrosecondDateSerializer extends JsonDeserializer<Date> {

  private static SimpleDateFormat formatter = 
    new SimpleDateFormat("dd-MM-yyyyThh:mm:ss:SSS");

  @Override
  public Date deserialize(JsonParser jp, DeserializationContext ctxt)
        throws IOException, JsonProcessingException {

    ObjectCodec oc = jp.getCodec();
    JsonNode node = jp.getCodec().readTree(jp);
    String dateString = node.get("sent_at").asText();
    String dateStringTruncated = dateString.substring(0, dateString.length() - 3);

    return formatter.parse(dateStringTruncated);
  }
}

How to fix inaccurate DateTime (minutes) parsed with Jackson/Gson libraries?

3 Answers3