1

Problem Description: MongoDB version is 3.4
In fact, did not do anything on the normal query, write,
because it is in the testing phase, QPS is small.

Question:
1: How is this anomaly produced.
2: what configuration or adjustment needs to be done? help me

02-01 15:11:47 WARN  - Got socket exception on connection [connectionId{localValue:43}] to 172.16.199.96:22001. All connections to 172.16.199.96:22001 will be closed.
02-01 15:11:47 INFO  - Closed connection [connectionId{localValue:43}] to 172.16.199.96:22001 because there was a socket exception raised by this connection.
        org.springframework.data.mongodb.UncategorizedMongoDbException: Exception receiving message; nested exception is com.mongodb.MongoSocketReadException: Exception receiving message
            at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:107)
            at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2135)
            at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:1978)
            at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:1784)
            at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:1767)
            at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:641)
            at org.springframework.data.mongodb.core.MongoTemplate.findOne(MongoTemplate.java:606)
            at org.springframework.data.mongodb.core.MongoTemplate.findOne(MongoTemplate.java:598)
            at com.xxx.xxx.xxx.xxx(xxxService.java:46)
            at com.xxx.xxx.xxx.xxx(xxxService.java:157)
            at com.xxx.xxx.xxx.xxx(xxxService.java:142)
            at com.xxx.xxx.xxx.xxx(xxxService.java:87)
            at com.alibaba.dubbo.common.bytecode.Wrapper2.invokeMethod(Wrapper2.java)
            at com.alibaba.dubbo.rpc.proxy.javassist.JavassistProxyFactory$1.doInvoke(JavassistProxyFactory.java:46)
            at com.alibaba.dubbo.rpc.proxy.AbstractProxyInvoker.invoke(AbstractProxyInvoker.java:72)
            at com.alibaba.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:53)
            at com.alibaba.dubbo.rpc.filter.ExceptionFilter.invoke(ExceptionFilter.java:64)
            at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:69)
            at com.alibaba.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:75)
            at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:69)
            at com.alibaba.dubbo.rpc.filter.TimeoutFilter.invoke(TimeoutFilter.java:42)
            at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:69)
            at com.alibaba.dubbo.rpc.protocol.dubbo.filter.TraceFilter.invoke(TraceFilter.java:78)
            at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:69)
            at com.alibaba.dubbo.rpc.filter.ContextFilter.invoke(ContextFilter.java:61)
            at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:69)
            at com.alibaba.dubbo.rpc.filter.GenericFilter.invoke(GenericFilter.java:132)
            at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:69)
            at com.alibaba.dubbo.rpc.filter.ClassLoaderFilter.invoke(ClassLoaderFilter.java:38)
            at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:69)
            at com.alibaba.dubbo.rpc.filter.EchoFilter.invoke(EchoFilter.java:38)
            at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:69)
            at com.alibaba.dubbo.rpc.protocol.dubbo.DubboProtocol$1.reply(DubboProtocol.java:100)
            at com.alibaba.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleRequest(HeaderExchangeHandler.java:98)
            at com.alibaba.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:170)
            at com.alibaba.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:52)
            at com.alibaba.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:81)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
        Caused by: com.mongodb.MongoSocketReadException: Exception receiving message
            at com.mongodb.connection.InternalStreamConnection.translateReadException(InternalStreamConnection.java:483)
            at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:228)
            at com.mongodb.connection.UsageTrackingInternalConnection.receiveMessage(UsageTrackingInternalConnection.java:96)
            at com.mongodb.connection.DefaultConnectionPool$PooledConnection.receiveMessage(DefaultConnectionPool.java:440)
            at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:112)
            at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168)
            at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289)
            at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:176)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:216)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:207)
            at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:113)
            at com.mongodb.operation.FindOperation$1.call(FindOperation.java:516)
            at com.mongodb.operation.FindOperation$1.call(FindOperation.java:510)
            at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:431)
            at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:404)
            at com.mongodb.operation.FindOperation.execute(FindOperation.java:510)
            at com.mongodb.operation.FindOperation.execute(FindOperation.java:81)
            at com.mongodb.Mongo.execute(Mongo.java:836)
            at com.mongodb.Mongo$2.execute(Mongo.java:823)
            at com.mongodb.DBCursor.initializeCursor(DBCursor.java:870)
            at com.mongodb.DBCursor.hasNext(DBCursor.java:142)
            at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:1964)
            ... 37 common frames omitted
        Caused by: java.net.SocketException: Connection reset
            at java.net.SocketInputStream.read(SocketInputStream.java:210)
            at java.net.SocketInputStream.read(SocketInputStream.java:141)
            at com.mongodb.connection.SocketStream.read(SocketStream.java:85)
            at com.mongodb.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:494)
            at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:224)
            ... 57 common frames omitted

java version 1.8.
spring boot version 1.5.3.
deployed with docker.

mongo.hosts=ip:port,ip:port,ip:port
mongo.database.name=dbname
mongo.username=username
mongo.password=pwd
mongo.connections.per.host=32
mongo.max.wait.time=2000
mongo.connect.timeout=2000
Nilesh Singh
  • 1,750
  • 1
  • 18
  • 30
william
  • 79
  • 1
  • 9

2 Answers2

1

You can try, autoConnectRetry simply means the driver will automatically attempt to reconnect to the server(s) after unexpected disconnects. In production environments you usually want this set to true.

This is from another post, How to configure MongoDB Java driver MongoOptions for production use?

0

for everybody who is experiencing the same random MongoSocketReadException, you may need the socketTimeoutMS or maxIdleTimeMS parameters instead. The parameter autoConnectRetry is not exposed any more in the mongodb connection string.

Our situation: we switched to mongodb atlas serverless solution for our development and testing environments, ever since then we got this MongoSocketReadException like every 15 min. or randomly. We are also behind a enterprise firewall.

According to https://www.mongodb.com/docs/v6.0/tutorial/connection-pool-performance-tuning/:

a misconfigured firewall closes a socket connection incorrectly and the driver cannot detect that the connection closed improperly. you need => Use socketTimeoutMS to ensure that sockets are always closed. Set socketTimeoutMS to two or three times the length of the slowest operation that the driver runs.

because the socketTimeoutMS is by default 0, which will never timeout.

And another parameter maxIdleTimeMS may also affect the connection because if the socket is closed and on the client side it's not detected, the connection will be still waiting in idle time and not cloesd. And by default it's 0 meaning it waits forever with no upper boundaries.

So configure this to a small amount may help the driver to close the the problematic connection with its closed socket, before it tries to connect to the db using the same connection and presumes the connection is still there.

So our solution: ...mongodbUri...?socketTimeoutMS=150000&maxIdleTimeMS=150000

we changed the socketTimeoutMS from 0 to 15s and same for the maxIdleTimeMS.

peisun
  • 11
  • 3