Microsoft Graph 3.0 java.net.SocketTimeoutException while retrieving group members

Question

I have recently upgraded my application to :

Spring Boot: 2.4.4
microsoft-graph: 3.0.0

While upgrading the application i have followed the upgrade guide.

I'm retrieving the group members using below code:

public void getGroupMembersWithDevices(final IGroup group) {
  List<IUser> users = this.userService.getAllUsersWithDevices();
  List<IUser> groupUsers = new ArrayList<>();
  DirectoryObjectCollectionWithReferencesPage directoryObjectCollectionWithReferencesPage = this.graphClient
    .getGraphServiceClient()
    .groups(group.getGroupId().toString())
    .members()
    .buildRequest()
    .select("Id")
    .top(999)
    .get();

    while (directoryObjectCollectionWithReferencesPage != null) {
      final List<DirectoryObject> directoryObjects = directoryObjectCollectionWithReferencesPage.getCurrentPage();
      List<IUser> usersWithDevices = users.stream().filter(
          one -> directoryObjects.stream().anyMatch(two -> UUID.fromString(two.id).equals(one.getUserId())))
          .collect(Collectors.toList());

      if (usersWithDevices.size() > 0) {
        groupUsers.addAll(usersWithDevices);
        users.removeAll(usersWithDevices);
      }

      final DirectoryObjectCollectionWithReferencesRequestBuilder nextPage = 
        directoryObjectCollectionWithReferencesPage.getNextPage();
      if (nextPage == null) {
        break;
      } else {
        directoryObjectCollectionWithReferencesPage = nextPage.buildRequest().get();
      }
    }

    group.setUsers(groupUsers.stream().collect(Collectors.toSet()));
  }

IUser and IDevice are custom models created.

The end goal is to get all the users with devices who are not part of any specified Group. Hence we have first fetched all the users with devices (no errors) and then crosschecked the same against group members.

However, after upgrading the application to I have started getting errors. I have tried rerunning this in production and keep getting the same error at different intervals.

com.microsoft.graph.core.ClientException: Error executing the request
    at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(CoreHttpProvider.java:388)
    at com.microsoft.graph.http.CoreHttpProvider.send(CoreHttpProvider.java:214)
    at com.microsoft.graph.http.CoreHttpProvider.send(CoreHttpProvider.java:191)
    at com.microsoft.graph.http.BaseCollectionRequest.send(BaseCollectionRequest.java:102)
    at com.microsoft.graph.http.BaseEntityCollectionRequest.get(BaseEntityCollectionRequest.java:78)
    at com.app.intune.util.GroupUtil.getGroupMembersWithDevices(GroupUtil.java:128)
    at com.app.intune.ScheduleInventory.scheduleGetInventory(ScheduleInventory.java:85)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:95)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
  Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at sun.security.ssl.InputRecord.readFully(Unknown Source)
    at sun.security.ssl.InputRecord.read(Unknown Source)
    at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
    at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
    at sun.security.ssl.AppInputStream.read(Unknown Source)
    at okio.Okio$2.read(Okio.java:140)
    at okio.AsyncTimeout$2.read(AsyncTimeout.java:237)
    at okio.RealBufferedSource.indexOf(RealBufferedSource.java:358)
    at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:230)
    at okhttp3.internal.http1.Http1ExchangeCodec.readHeaderLine(Http1ExchangeCodec.java:242)
    at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.java:213)
    at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.java:115)
    at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:94)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at com.microsoft.graph.httpcore.RedirectHandler.intercept(RedirectHandler.java:137)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at com.microsoft.graph.httpcore.RetryHandler.intercept(RetryHandler.java:176)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at com.microsoft.graph.httpcore.AuthenticationHandler.intercept(AuthenticationHandler.java:59)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at com.microsoft.graph.httpcore.TelemetryHandler.intercept(TelemetryHandler.java:69)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
    at okhttp3.RealCall.execute(RealCall.java:81)
    at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(CoreHttpProvider.java:385)

   ... 20 more

Update 1:
Class created below for creating graph client and setting up proxy:

import java.net.InetSocketAddress;
import java.util.Arrays;
import java.util.List;

import javax.annotation.PostConstruct;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import com.azure.core.http.ProxyOptions;
import com.azure.core.http.netty.NettyAsyncHttpClientBuilder;
import com.azure.identity.ClientSecretCredential;
import com.azure.identity.ClientSecretCredentialBuilder;
import com.microsoft.graph.authentication.TokenCredentialAuthProvider;
import com.microsoft.graph.requests.GraphServiceClient;

@Component
public class GraphClient {
    private String clientId;
    private String clientSecret;
    private List<String> scopes;
    private String tenantId;

    @Autowired
    BasicConfiguration configuration;

    @Autowired
    IntuneConfig intuneConfig;

    @PostConstruct
    public void init() {
        this.clientId = this.configuration.getClientId();
        this.clientSecret = this.configuration.getSecretKey();
        this.scopes = Arrays.asList(this.configuration.getScope());
        this.tenantId = this.configuration.getTenant();
    }

    @SuppressWarnings({ "rawtypes" })
    public GraphServiceClient getGraphServiceClient() {
        final ClientSecretCredential clientSecretCredential = new ClientSecretCredentialBuilder()
                    .clientId(this.clientId).clientSecret(this.clientSecret).tenantId(this.tenantId)
                    .httpClient(
                            new NettyAsyncHttpClientBuilder()
                                    .proxy(new ProxyOptions(ProxyOptions.Type.HTTP, new InetSocketAddress(
                                            this.intuneConfig.getProxyHost(), this.intuneConfig.getProxyPort())))
                                    .build())
                    .build();

            final TokenCredentialAuthProvider tokenCredentialAuthProvider = new TokenCredentialAuthProvider(this.scopes,
                    clientSecretCredential);

            final GraphServiceClient graphClient = GraphServiceClient.builder()
                    .authenticationProvider(tokenCredentialAuthProvider).buildClient();

            return graphClient;
    }
}

Update 2: Updated GraphClient class to retrieve instance of GraphServiceClient. The instance is created just once.

import java.net.InetSocketAddress;
import java.util.Arrays;
import java.util.List;

import javax.annotation.PostConstruct;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import com.azure.core.http.ProxyOptions;
import com.azure.core.http.netty.NettyAsyncHttpClientBuilder;
import com.azure.identity.ClientSecretCredential;
import com.azure.identity.ClientSecretCredentialBuilder;
import com.microsoft.graph.authentication.TokenCredentialAuthProvider;
import com.microsoft.graph.requests.GraphServiceClient;

@Component
public class GraphClient {
    private String clientId;
    private String clientSecret;
    private List<String> scopes;
    private String tenantId;
    
    @SuppressWarnings("rawtypes")
    private static GraphServiceClient graphClient;

    @Autowired
    BasicConfiguration configuration;

    @Autowired
    IntuneConfig intuneConfig;

    @PostConstruct
    public void init() {
        this.clientId = this.configuration.getClientId();
        this.clientSecret = this.configuration.getSecretKey();
        this.scopes = Arrays.asList(this.configuration.getScope());
        this.tenantId = this.configuration.getTenant();
    }

    @SuppressWarnings({ "rawtypes" })
    public GraphServiceClient getGraphServiceClient() {
    if (GraphClient.graphClient == null) {
        final ClientSecretCredential clientSecretCredential = new ClientSecretCredentialBuilder()
                    .clientId(this.clientId).clientSecret(this.clientSecret).tenantId(this.tenantId)
                    .httpClient(
                            new NettyAsyncHttpClientBuilder()
                                    .proxy(new ProxyOptions(ProxyOptions.Type.HTTP, new InetSocketAddress(
                                            this.intuneConfig.getProxyHost(), this.intuneConfig.getProxyPort())))
                                    .build())
                    .build();

            final TokenCredentialAuthProvider tokenCredentialAuthProvider = new TokenCredentialAuthProvider(this.scopes,
                    clientSecretCredential);

            final GraphServiceClient client = GraphServiceClient.builder()
                    .authenticationProvider(tokenCredentialAuthProvider).buildClient();

           GraphClient.graphClient = client;
    }
    return GraphClient.graphClient; 
    
    }
}

can you share the core of the `getGraphServiceClient` method? I believe you're instantiating the client too often leading to a port exhaustion which in turns leads to time outs. — baywet, Mar 29 '21 at 12:42

score 1 · Accepted Answer · answered Mar 31 '21 at 12:46

1

The GraphServiceClient is being instantiated for each call. This in turns instantiates one OkHttpClient per GraphServiceClient, which in turns creates a connection pool. Because of the way connections are managed by OSes (they are kept open for a period because closing and opening a connection is a costly operation), this will lead to a port exhaustion on the machine.
The next requests that come in after no more ports are available, block waiting for a port to free and for a connection to be available, and eventually timeout with the exception you are seeing.

To fix that issue:

Make sure your GraphClient class is instantiated once throughout the lifecycle of your application
Implement some lazy loading for the getGraphServiceClient method, so it "caches" a client in a field and returns that if the value is not null instead of creating a new one for each call.

answered Mar 31 '21 at 12:46

baywet

4,377
4
20
49

thanks, i will give it a try. A small Q: supposing the graphclient class is instantiated once will it refresh the token on expiration of token considering the application might take more time than the expiration of first token? – Sujit J Mar 31 '21 at 13:09
the authentication provider automatically evaluates whether the token from the cache is still valid, and gets a new one if it's expired before sending the request – baywet Mar 31 '21 at 13:36
I have made sure the GraphClient class is instantiated only once and now not receiving the SocketTimeOutException. Ill keep monitoring for few days to see if the issue comes up again. – Sujit J Apr 05 '21 at 03:54
Im still getting the SocketTimeOutException after updating the GraphClient as is Update 2. This is intermittent though and does not happen everyday. – Sujit J Apr 09 '21 at 09:41
It looks like the proxy is not set for the http client the graph client is using, you can do so by [customizing the http client](https://learn.microsoft.com/en-us/graph/sdks/customize-client?tabs=java) to [set the proxy](https://stackoverflow.com/questions/37866902/okhttp-proxy-settings) – baywet Apr 09 '21 at 11:38
Setting the proxy using the above links does not work in spring boot due to managed dependency conflict. Check the post https://stackoverflow.com/questions/66744821/ms-graph-3-0-java-clientcredentials-error-java-lang-bootstrapmethoderror-jav For the same reason the proxy was setup using NettyAsyncHttpClientBuilder() as shown in Update2 – Sujit J Apr 10 '21 at 02:49
i have updated microsoft-graph to 3.2.0 however it still has dependency on azure-identity 1.2.3. I could see that azure-identity has a newer version 1.2.5 which updates the reactor-netty to 1.0.4. Are there any timelines to update the azure-identity to 1.2.5. – Sujit J Apr 12 '21 at 13:06
you should be able to force it to use azure identity 1.2.5 across the whole project just by adding a reference to azure identity 1.2.5 in your own project – baywet Apr 12 '21 at 14:04
Having read both https://learn.microsoft.com/en-us/graph/sdks/customize-client?tabs=java and https://stackoverflow.com/questions/37866902/okhttp-proxy-settings. Does HttpClients.createDefault() need to be given the same authentication provider object? i.e. TokenCredentialAuthProvider tokenCredAuthProvider. Im unable to get it working by providing the above. – Sujit J Apr 14 '21 at 11:02
no, you technically don't need to pass the authentication provider to the graph client builder anymore if you've already passed the http client. This is because the http client will contain the authentication handler interceptor, which itself already has a reference to the authentication provider. That was a limitation from the previous version that probably needs to be updated in the documentation. – baywet Apr 14 '21 at 19:01
just put a pull request together to fix that documentation page https://github.com/microsoftgraph/microsoft-graph-docs/pull/12311 – baywet Apr 14 '21 at 19:03
have followed the updated documentation. Pls check Update 3, im using only microsoft-graph 3.2 in a simple maven java app but im unable to set up the proxy and end up getting java.net.UnknownHostException: login.microsoftonline.com. Sry for going off-topic but i want to get everything set first as per recommendations and then see if i still get the socketexception – Sujit J Apr 15 '21 at 05:27
Please post a new question, link it here and mark the answer here. Stack overflow doesn't like threads changing topics – baywet Apr 17 '21 at 20:31
Here is the new question for proxy error https://stackoverflow.com/questions/67156456/using-proxy-with-microsoft-graph-3-x – Sujit J Apr 19 '21 at 05:38
Hey baywet from where to get/configure this proxy details. @SujitJ is your **SocketTimeOutException** resolved? – Shantaram Tupe Jun 07 '22 at 06:08

Microsoft Graph 3.0 java.net.SocketTimeoutException while retrieving group members

1 Answers1

Linked