6

Requirement is to sync mails from Gmail for an user into our CRM. The system in place is based on Google Pub/Sub which watches inbox of the user for any change and fires the notification to our HTTPs endpoint. More on this at Gmail cloud pub/sub.

Based on the above procedure we git history of changes. And then i am interested in only new messages, so history.getMessagesAdded is preferred as per this guide. Issue we are facing now is the first mail of a thread is not captured under messagesAdded all the subsequent messages are passing through our system.

Note: For the first mail, we do get push from Google. But when we try to get Messages added it turns out empty. Is there anything special needs to be done for the first mail of the thread or am i missing out something.

Itachi
  • 1,383
  • 11
  • 22
  • 1
    I found here a [question](http://stackoverflow.com/questions/26305327/getting-history-events-for-just-new-messages) that used [messages.list](https://developers.google.com/gmail/api/v1/reference/users/messages/list) to check the history of a message. It uses the parameter q in the messages.list to supply the most recent time stamp. Just check this question if it can help you. For more info, check this [one](http://stackoverflow.com/questions/25571679) and [this](http://stackoverflow.com/questions/33467106). – KENdi Feb 08 '17 at 14:38
  • @KENdi thank you. Though it didn't answer why fetching history fails, nevertheless sync based on message list as suggested in those answers works fine. – Itachi Feb 10 '17 at 10:18

1 Answers1

21

I was experiencing a very similar problem, and my mistake was that I was using the historyId from the push notification, the solution was to store the last known historyId on my database, so, every time I get a notification, I get the history from the id I have stored, not the one from the notification.

In my case, the historyId from the notification doesn't even make part of the history, maybe because of my watch restrictions: labelIds=['INBOX']

This is the google pub/sub notification:

{
  message:
  {
    data: {"emailAddress": "user@example.com", "historyId": "9876543210"},
    message_id: "1234567890",
  }

  subscription: "projects/myproject/subscriptions/mysubscription"
}

I was using the message.data.historyId, wich was causing the confusion!

The message.data, comes as a base64 encoded string, in this example I just decoded it!

Step by step for watching new e-mails on the inbox:

  1. Do all the configuration in the google pub/sub.

  2. Start watching the user with the filters you want (docs.: https://developers.google.com/gmail/api/v1/reference/users/watch)

  3. Store the historyId obtained in the step 2

  4. When receive the notification, get all the events (history) using the stored id as the startHistoryId parameter (docs: https://developers.google.com/gmail/api/v1/reference/users/history/list)

  5. In the history list obtained on the step 4, look for the new messages: history.getMessagesAdded().

  6. Update the last known history id in your database, so you don't need to deal with the whole history every time!

I hope it helps.

Fábio Magagnin
  • 605
  • 7
  • 11
  • 1
    This is so useful. I couldn't understand why my historyID first appeared to be empty, but then when looking at it again it had a history. I understand now that I can't immediately use the historyID received in the push notification, but have to store it instead and use the previous one. In my case this is because I was using the `startHistoryId` filter, which looks for history records **after** the provided id. – gdvalderrama Nov 10 '17 at 16:03
  • 1
    Great! Thank you! – Facundo Fasciolo Feb 01 '18 at 23:12