Authentication Failure for IMAP using Client Credential flow for OAuth2.0 | Java | Exchange Online

Question

I am facing an authentication failure issue while trying to connect for both IMAP protocols using the Client Credential Grant flow for OAuth2.0. Where, I have been following the steps suggested by Microsoft in its step-by-step guide i.e. "Authenticate an IMAP, POP or SMTP connection using OAuth"

I have been using this github project to fetch the Access Token using Client Credential Grant flow: MSAL Client Credential Grant using Java

Java Code for IMAP

public static void connectIMAP(String userEmail, String accessToken){
    String SSL_FACTORY = "javax.net.ssl.SSLSocketFactory";
    Properties props= new Properties();
    
    props.put("mail.imap.ssl.enable", "true");
    props.put("mail.imap.sasl.enable", "true");
    props.put("mail.imap.port", "993");
    
    props.put("mail.imap.auth.mechanisms", "XOAUTH2");
    props.put("mail.imap.sasl.mechanisms", "XOAUTH2");
    
    props.put("mail.imap.auth.login.disable", "true");
    props.put("mail.imap.auth.plain.disable", "true");
    
    props.setProperty("mail.imap.socketFactory.class", SSL_FACTORY);
    props.setProperty("mail.imap.socketFactory.fallback", "false");
    props.setProperty("mail.imap.socketFactory.port", "993");
    props.setProperty("mail.imap.starttls.enable", "true");
    
    props.put("mail.debug", "true");
    props.put("mail.debug.auth", "true");

    Session session = Session.getInstance(props);
    session.setDebug(true);

    try {
        final Store store = session.getStore("imap");                   
        store.connect("outlook.office365.com",userEmail, accessToken);  
        
    } catch (NoSuchProviderException e) {   // session.getStore()
        e.printStackTrace();
    } catch (MessagingException e) {        // store.connect()
        e.printStackTrace();
    }
}

Following are the credentials I have used while performing the Client Credential Grant flow using MSAL library

userEmail:- Email of the user which is used to login to Azure portal (eg, <registered_azure_email_id>)
authority=https://login.microsoftonline.com/<tenant - id - here>/
client_id=<client(application) - id - here> IMAGE for app overview screen containing Tenant and Client ID
secret=<client - secret - key - here>
scope=https://outlook.office.com/.default

[Note: I have been using the Default Active Directory, and the default user(Admin) for my Azure account. Is it fine this way ? or does it require a new custom Azure AD and a separate tenant for performing client credential flow]

Following image contains list of permissions I have applied in my app: IMAGE for list of permissions applied

Error Logs:

    *** IMAP *** 

DEBUG: JavaMail version 1.5.6
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.5.6
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle]
DEBUG IMAP: mail.imap.fetchsize: 16384
DEBUG IMAP: mail.imap.ignorebodystructuresize: false
DEBUG IMAP: mail.imap.statuscachetimeout: 1000
DEBUG IMAP: mail.imap.appendbuffersize: -1
DEBUG IMAP: mail.imap.minidletime: 10
DEBUG IMAP: enable STARTTLS
DEBUG IMAP: enable SASL
DEBUG IMAP: SASL mechanisms allowed: XOAUTH2
DEBUG IMAP: closeFoldersOnStoreFailure
DEBUG IMAP: trying to connect to host "outlook.office365.com", port 993, isSSL true
* OK The Microsoft Exchange IMAP4 service is ready. [UABO......]
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: AUTH: XOAUTH2
DEBUG IMAP: protocolConnect login, host=outlook.office365.com, user=ManishPrajapati@SampleOrg2022.onmicrosoft.com, password=<non-null>
DEBUG IMAP: SASL Mechanisms:
DEBUG IMAP:  XOAUTH2
DEBUG IMAP: 
DEBUG IMAP: SASL client XOAUTH2
DEBUG IMAP: SASL callback length: 2
DEBUG IMAP: SASL callback 0: javax.security.auth.callback.NameCallback@73f9ac
DEBUG IMAP: SASL callback 1: javax.security.auth.callback.PasswordCallback@1064425
A1 AUTHENTICATE XOAUTH2 dXNlcj.....
A1 NO AUTHENTICATE failed.
javax.mail.AuthenticationFailedException: AUTHENTICATE failed.
    at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:725)
    at javax.mail.Service.connect(Service.java:366)
    at javax.mail.Service.connect(Service.java:246)
    at test.ClientCredentialGrantAndConnect.connectIMAP(ClientCredentialGrantAndConnect.java:166)
    at test.ClientCredentialGrantAndConnect.main(ClientCredentialGrantAndConnect.java:45)

Any help in figuring out the issue will be highly appreciated.

Thank you.

Yeah I know... We are pretty much on the same boat. I'm finding a workaround to access mails though https://graph.microsoft.com/.default — HardcoreGamer, Jul 22 '22 at 06:58
I'm able to send mail using **Microsoft Graph** instead of outlook scope. With `client credentials flow` and grant `Mail.Read.All`, `Mail.Write.All` ...etc. The permission has to be added as dedicated daemon process instead of acting as certain user. — HardcoreGamer, Aug 02 '22 at 03:11

user3206771 · Answer 1 · 2022-08-19T07:15:27.733

I was able to connect to mailbox using IMAP OAuth2 access token generated by client credential grant. Here are the details:

Java code (Replace clientid, secret, authority and email id in the below code with the values from your azure app registration)

//this method returns the token
public String getAccessTokenByClientCredentialGrant()  {
        
    String accessToken = null;
    String clientId = "<client id from azure app registration>";
    String secret = "<client secret from azure app registration>";
    String authority = "https://login.microsoftonline.com/<tenant-id from azure>/oauth2/v2.0/token";
    String scope = "https://outlook.office365.com/.default";
    log.info("Client ID : "+clientId);
    log.info("Client Secret : "+secret);
    log.info("Auth Server: "+authority);
    log.info("Scope: "+scope);
    
    try {
        
    
        ConfidentialClientApplication app = ConfidentialClientApplication.builder(
                clientId,
                ClientCredentialFactory.createFromSecret(secret))
                .authority(authority)
                .build();   
        
        // With client credentials flows the scope is ALWAYS of the shape "resource/.default", as the
        // application permissions need to be set statically (in the portal), and then granted by a tenant administrator
        ClientCredentialParameters clientCredentialParam = ClientCredentialParameters.builder(
                Collections.singleton(scope))
                .build();
        
        CompletableFuture<IAuthenticationResult> future = app.acquireToken(clientCredentialParam);
        IAuthenticationResult result = future.get();
        accessToken = result.accessToken();
        
    } catch(Exception e) {
        log.error("Exception in acquiring token: "+e.getMessage());
        e.printStackTrace();
    }
    log.info("Access Token : "+accessToken);
    return accessToken;
}

//This method connects to store using the access token
public Store connect(String userEmailId, String oauth2AccessToken) throws Exception {

    String host = "outlook.office365.com";
    String port = "993";
    Store store = null;
    
    
    String SSL_FACTORY = "javax.net.ssl.SSLSocketFactory";
    Properties props= new Properties();

    props.put("mail.imaps.ssl.enable", "true");
    props.put("mail.imaps.sasl.enable", "true");
    props.put("mail.imaps.port", port);

    props.put("mail.imaps.auth.mechanisms", "XOAUTH2");
    props.put("mail.imaps.sasl.mechanisms", "XOAUTH2");
    
    props.put("mail.imaps.auth.login.disable", "true");
    props.put("mail.imaps.auth.plain.disable", "true");

    props.setProperty("mail.imaps.socketFactory.class", SSL_FACTORY);
    props.setProperty("mail.imaps.socketFactory.fallback", "false");
    props.setProperty("mail.imaps.socketFactory.port", port);
    props.setProperty("mail.imaps.starttls.enable", "true");

    props.put("mail.debug", "true");
    props.put("mail.debug.auth", "true");

    Session session = Session.getInstance(props);
    session.setDebug(true);
    
    store = session.getStore("imaps");
    
    log.info("OAUTH2 IMAP trying to connect with system properties to Host:" + host + ", Port: "+ port
            + ", userEmailId: " + userEmailId+ ", AccessToken: " + oauth2AccessToken);
    try {
    
        store.connect(host, userEmailId, oauth2AccessToken);
        log.info("IMAP connected with system properties to Host:" + host + ", Port: "+ port
            + ", userEmailId: " + userEmailId+ ", AccessToken: " + oauth2AccessToken);
        if(store.isConnected()){
            log.info("Connection Established using imap protocol successfully !");      
        }
    } catch (Exception e) {
        log.error("Store.Connect failed with the errror: "+e.getMessage());
        StringWriter sw = new StringWriter();
        e.printStackTrace(new PrintWriter(sw));
        String exceptionAsString = sw.toString();
        log.error(exceptionAsString);
        
    }

    return store;

}

//the main method which calls the above 2 methods.
public void getEmailContents() throws Exception {
    Store store = null;
    
    String accessToken = getAccessTokenByClientCredentialGrant();
    String emailId = "<email which needs to be read>";

    try {
        store = connect(emailId, accessToken );
    } catch (Exception ex) {
        log.error("Exception in connecting to email " + ex.getMessage());
        ex.printStackTrace();
        
    }

    //write code to read email using javax.mail code

}

Make sure your Application is registered under Azure App Registrations and the following API permissions (Application permissions) are granted in "API Permissions" for the app.

Office 365 Exchange Online

full_access_as_app
IMAP.AccessAsApp
Mail.Read
Mail.ReadWrite
Mail.Send

Also make sure that the mailbox is linked to Azure using the command in the below link: https://learn.microsoft.com/en-us/graph/auth-limit-mailbox-access

Test-ApplicationAccessPolicy -Identity <email> -AppId <app id>

You can also decode the access token generated from java code using https://jwt.io/ and verify if the permissions are correct.

Verify office365 email configuration and make sure that IMAP is enabled: Help can be found in the below link. https://www.limilabs.com/blog/office365-enable-imap-pop3-smtp

I spent several hours thrashing around this issue repeatedly hitting NO AUTHENTICATE and this one worked for me. I wish I had more upvotes to give it. — woddle, Nov 08 '22 at 11:23
even if I connect successfuly I can not get the IMAP folders with the error: A3 LIST "" "%" A3 BAD User is authenticated but not connected. javax.mail.MessagingExceptionwhen I try to get the folder names — benchpresser, Mar 23 '23 at 07:15

score 4 · Accepted Answer · answered Aug 13 '22 at 10:30

I am now able to perform the OAuth2.0 authentication for IMAP protocol on exchange-online server. I found the issue with my approach was that, I was using some parameters from wrong places due to lack of experience on working with Azure.

Following the instructions given in step by step guide and setting permissions of newly created application was OK. But the real problem was with the queries given at the end of this post where we need to run 3 commands in order to make it possible to perform OAuth2.0

As per my understanding, following is the list of parameters used while performing Service Principal related queries:

Parameters used (and where to find them):

appId: Application (client) ID [ found in Application Overview screen, from both Enterprise and App reg]
entObjId: Object ID(Enterprise app) [ found in Enterprise Application Overview screen only ]
orgId: Directory (tenant) ID [ found in Azure AD overview screen ]

Commands:

New-ServicePrincipal -AppId appId -ServiceId entObjId -Organization orgId
Get-ServicePrincipal -Organization entObjId | fl
Add-MailboxPermission -Identity "<email_id_here>" -User entObjId -AccessRights FullAccess

Confusions I faced:

In Add-MailboxPermission cmdlet, <SERVICE_PRINCIPAL_ID> creates confusion, because in order to apply permissions like "IMAP.AccessAsApp", the internet tells that "Service Principal ID" can be found at [ Azure AD -> Enterprise Application -> (chosen application) -> Permissions -> IMAP.AccessAsApp -> use the Service Principal ID from Flyout menu ]
It is okay to use the Enterprise Object ID in all 3 cmdlets

Can you post your complete App configuration and Java code? I've been following all the data on this post and the linked Microsoft documents but I am still getting an authorization error. Thank you! — Ben, Sep 21 '22 at 15:43
What caught me out was the tenant ID in the token request. Although you pass in your client secret the tenant ID is that of the email account you are connecting to. — Ben, Sep 26 '22 at 12:22

Authentication Failure for IMAP using Client Credential flow for OAuth2.0 | Java | Exchange Online

2 Answers2

Linked