Getting ALL Email headers with Microsoft Graph SDK in a unified list?

Question

I'm using Microsoft Graph to get information from a specific account's inbox. That account has multiple email aliases. However, even after I $select query the InternetMessageHeaders property, it does not include the alias the email was sent to.

I found out that if I also select and expand the query using singleValueExtendedProperties, I get what I need, but now I'm missing some headers that are present in InternetMessageHeaders. It also comes back in a giant blob of text that doesn't seem too straight forward to parse. If I could somehow parse this blob of text into a key:value map, I could just merge both these values and call it a day, but I don't know if that's possible. I was hoping it might be possible with multiValueExtendedProperties, but I don't think that's an intended use for it.

I'm using Microsoft Graph's SDK for .NET (currently using 4.0.0) and this is what I have so far to get all the emails:

public async Task<List<Message>> PollEmails() {
    var completeList = new List<Message>();

    var messagesRequest = await graphClient.
        Users[GraphOptions.AccountId]                                               // Email account ID
            .Messages                                                               // All Emails
            .Request()                                                              // Create request w/ authorization & headers needed
            .Select("internetMessageHeaders,singleValueExtendedProperties")         // Additionally, select the email's headers
            .Expand("singleValueExtendedProperties($filter=id eq 'String 0x007D')") // Without this filter, we cannot tell which email alias was used in the To: field
            .Top(100)                                                               // Get emails in batches of 100 (default is 10)
            .GetAsync();                                                            // Run request and await results

    completeList.AddRange(messagesRequest);

    // Results are paginated, so keep getting all the results until there are no more pages
    while (messagesRequest.NextPageRequest != null) {
        messagesRequest = await messagesRequest.NextPageRequest.GetAsync();
        completeList.AddRange(messagesRequest);
    }

    return completeList;
}

Message.InternetMessageHeaders has most of the headers I need, and the rest are in Message.SingleValueExtendedProperties, but there's no easy way to merge the two into one list without duplicate headers.

I could just blindly loop through InternetMessageHeaders and append $"{Name}: {Value}" to the SingleValueExtendedProperties value, but I'll end up with duplicate headers. I could just search the string for the key before I append it, but that just feels like a hack.

Is there an official way to get every single header using Microsoft Graph's SDK?

Thank you,

EDIT

Here's a list comparing the headers in case anyone is curious about what I meant with headers starting with "ARC-*":

This is an example (from the API) of what I get back from InternetMessageHeaders in JSON format: https://pastebin.com/Bdk35C5q

This is what I get back from SingleValueExtendedProperties in plain text: https://pastebin.com/N7c0JLB6

Related questions that don't answer my question:

Microsoft Graph: How to get the alias from an email message? - How to get the alias email used via singleValueExtendedProperties - doesn't include headers starting with ARC-*

Get message using Graph missing some InternetMessageHeaders - Get "complete" list of headers, again via singleValueExtendedProperties - again, doesn't include headers starting with ARC-*

https://learn.microsoft.com/en-us/answers/questions/673767/how-to-read-all-internet-message-headers-of-an-ite.html - Asking exactly what I'm asking but never got a response.

What exactly are you using the MIME headers for? Is there a particular header that you need? — Dmitry Streblechenko, Jan 27 '22 at 22:14
My company wants to be able to, basically, take emails from a specific account, shove them into a database, and then delete them from Outlook. But, before we start deleting emails, we want to make sure we have ALL potential information we might need, which includes EVERY single header. — Chauncey Hoover, Jan 27 '22 at 22:17
Oh, so you mean all the item properties? Graph is not the best API for that - MIME is not a high fidelity format, and you will lose all MAPI properties used by Outlook - you can see them in OutlookSpy (click IMessage button). You probably need something similar to the MSG file format (it stores all/most MAPI properties). In EWS, you can use `ExportItem` operation. Fast Transfer Stream format is not documented, but it is similar to MSG and Redemption can import back into either standalone MSG file or an Outlook item. — Dmitry Streblechenko, Jan 28 '22 at 03:49
Sorry if this is a stupid question - I don't work with emails very often and this is my first time using Graph. If I'm just trying to get headers, doesn't that include more than just MIME? I also need to be able to do all of this automatically as a service - there should be 0 manual work. Would any of the options you suggested be capable of that? And it seems like Graph is capable of what I want, it just doesn't seem to be very clean or intuitive. I can get all the headers, I just have to do it in 2 separate places and manually join it. I'm more so curious if there's a built in way to combine? — Chauncey Hoover, Jan 28 '22 at 14:21
What exactly do you mean by "headers"? "Headers" usually means the MIME headers of a message (or a MIME part, such as an attachment). Exchange and Outlook do not store messages in the MIME format, it is only used when a message is sent. There is a large number of MAPI properties that Outlook uses (e.g. replied/forwarded icon, reminder, etc.) that are not mapped to any MIME headers. But if you are not using these properties, there is no need to retrieve them, unless you are planning to reconstruct the message in Outlook at a later point. — Dmitry Streblechenko, Jan 28 '22 at 14:38
Take a look at Outlook messages with OutlookSpy (https://www.dimastr.com/outspy) to see which properties are available (click IMessage button). — Dmitry Streblechenko, Jan 28 '22 at 14:41

score 5 · Accepted Answer · answered Feb 01 '22 at 20:36

Turns out Outlook just isn't consistent with which headers each email has. So, in this case, the singleValueExtendedProperties was actually what I needed.

What I didn't realize I was asking for was I needed MAPI properties, as Dmitry pointed out (thank you for clarifying).

In case anyone else stumbles on this question before the other ones I found, here's what you need to do to access the MAPI properties:

API:

GET https://graph.microsoft.com/v1.0/users/{id | userPrincipalName}/messages/{id}?$expand=singleValueExtendedProperties($filter id eq 'String 0x007D')

SDK:

// Since `$select` only returns the properties you specify and the only way to get this property is with both `$select` and `$expand`, you have to manually select all the properties you want. Thanks, Microsoft.
var selectProperties =
    "id,subject,receivedDateTime,sentDateTime,from,toRecipients,ccRecipients,bccRecipients," +
    "replyTo,internetMessageHeaders,singleValueExtendedProperties,bodyPreview,body,hasAttachments," +
    "sender,categories,parentFolderId,conversationId,conversationIndex,isDeliveryReceiptRequested," +
    "isReadReceiptRequested,isRead,isDraft,webLink,inferenceClassification,flag";

// Get user's emails (Requires permission Mail.Read) - 
var messagesRequest = await graphClient.
    Users[AccountId]                                                            // Replace this with account ID
        .Messages                                                               // All Emails (or specify specific email via ["emailId"]
        .Request()                                                              // Create request w/ authorization & headers needed
        .Select(selectProperties)                                             // Select all the specified properties
        .Expand("singleValueExtendedProperties($filter=id eq 'String 0x007D')") // Select MAPI property Transport Message Headers to get the original `To:` email
        .Top(100)                                                               // Get emails in batches of 100 (default is 10)
        .GetAsync();

For a complete list of MAPI properties, see this page. For the specific MAPI property used in this example, see this page.

Can you get PR_TRANSPORT_MESSAGE_HEADERS 0x007D from the .Net Microsoft Graph API?

Getting ALL Email headers with Microsoft Graph SDK in a unified list?

1 Answers1