EF6 Memory leak without any reason

Question

It's been a while for me to find this issue but I am 100% sure that it's happening and at least on my machine.

I was able to found out that just this small piece of code is causing of memory leaks, but still I have no idea why.

for (int i = 0; i < 1000000; i++)
{
    using (var db = new TestEntity())
    {
        db.Configuration.AutoDetectChangesEnabled = false;
        db.Configuration.ValidateOnSaveEnabled = false;

        // If I just create new Context everything is normal.
        // I need to request anything 
        {
            var any = (bool)db.Test.AsNoTracking().Any();
        }

        // Or more simply just in that way
        // var any = db.Test.AsNoTracking().Any();

    }
    if (i % 1000 == 0)
    {
        Console.WriteLine(i.ToString());
    }
}

This code is so simply and still I am unable to see what is causing the issues.

I am creating new Context in single iteration and then just Dispose() it. My bool variable is local and is never used. I am also using Any() which is returning bool value and not reference to object.

I will not agree to that Garbage Collector have no time to collect because wherever I will force him to Collect() it's still leaking. It's also recommended by EF Team to have Context for as short time as possible.

And more important is that this leak is going/happening into unmanaged memory. The result of using ANTS Memory Profiler is that I can see that there is leak but still I have no idea why.

This code is resulting in StackOverflowException in EntityFramework.dll after 1 minute ( ~200k iterations):

An unhandled exception of type 'System.StackOverflowException' occurred in EntityFramework.dll

In unmanaged memory I can see ~200k objects of size 8192 bytes.

It's not unrealistic case. It's the real problem which is stopping me.

This is very basic model of what is happening in my real software. I need to parse millions of lines in different format at first and then check if record exists in database.

To reproduce the issue you need: Main program:

using System;
using System.Linq;

namespace ConsoleApplication31
{
    class Program
    {
        static void Main(string[] args)
        {
            var test = new TestClass();

            test.Test();
            Console.ReadLine();
        }
    }

    public class TestClass
    {
        public void Test()
        {
            try
            {
                for (int i = 0; i < 1000000; i++)
                {
                    using (var db = new TestEntity())
                    {
                        db.Configuration.AutoDetectChangesEnabled = false;
                        db.Configuration.ValidateOnSaveEnabled = false;
                        {
                            var any = (bool)db.Test.AsNoTracking().Any();
                        }
                    }
                    if (i % 1000 == 0)
                    {
                        Console.WriteLine(i.ToString());
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }
}

And DB Model Entity :

namespace ConsoleApplication31
{
    using System.Data.Entity;

    public partial class TestEntity : DbContext
    {
        public TestEntity()
                : base("name=TestEntityConnectionString")
        {
        }

        public virtual DbSet<Test> Test { get; set; }

        protected override void OnModelCreating(DbModelBuilder modelBuilder)
        {
            Database.SetInitializer<TestEntity>(null);
        }
    }
}

With Test Table:

using System.ComponentModel.DataAnnotations;

namespace ConsoleApplication31
{
    public partial class Test
    {
            [Key]
            [Required]
            public long TestId { get; set; }

            [Required]
            public string TestName { get; set; }

            [Required]
            public string TestDescription { get; set; }
        }
}

My configurations:

a) 4C/8T 16GB RAM, Windows 10 14393 (RTM), Visual Studio 2015, Sql Server 2016

b) 2C/4T 8GB RAM, exactly the same system (the same disk, connected to other machine )

Tested on .NET Framework versions: 4.5, 4.5.2, 4.6.1, 4.6.2

Tested on Entity Framework versions: 6.0.0, 6.1.3

I will appreciate any help.

Call stack made when there was ~700MB of memory usage: [Managed to Native Transition]

    System.Data.dll!SNINativeMethodWrapper.SNIReadSyncOverAsync(System.Runtime.InteropServices.SafeHandle pConn = {System.Data.SqlClient.SNIHandle}, ref System.IntPtr packet = {System.IntPtr}, int timeout)   Unknown
    System.Data.dll!System.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync()   Unknown
    System.Data.dll!System.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket()   Unknown
    System.Data.dll!System.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer()   Unknown
    System.Data.dll!System.Data.SqlClient.TdsParserStateObject.TryReadByte(out byte value = 0)  Unknown
    System.Data.dll!System.Data.SqlClient.TdsParser.TryRun(System.Data.SqlClient.RunBehavior runBehavior = ReturnImmediately, System.Data.SqlClient.SqlCommand cmdHandler = {System.Data.SqlClient.SqlCommand}, System.Data.SqlClient.SqlDataReader dataStream = {System.Data.SqlClient.SqlDataReader}, System.Data.SqlClient.BulkCopySimpleResultSet bulkCopyHandler = null, System.Data.SqlClient.TdsParserStateObject stateObj, out bool dataReady = false)  Unknown
    System.Data.dll!System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()    Unknown
    System.Data.dll!System.Data.SqlClient.SqlDataReader.MetaData.get()  Unknown
    System.Data.dll!System.Data.SqlClient.SqlCommand.FinishExecuteReader(System.Data.SqlClient.SqlDataReader ds = {System.Data.SqlClient.SqlDataReader}, System.Data.SqlClient.RunBehavior runBehavior, string resetOptionsString, bool isInternal, bool forDescribeParameterEncryption)    Unknown
    System.Data.dll!System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(System.Data.CommandBehavior cmdBehavior, System.Data.SqlClient.RunBehavior runBehavior, bool returnStream, bool async, int timeout, out System.Threading.Tasks.Task task, bool asyncWrite, bool inRetry, System.Data.SqlClient.SqlDataReader ds, bool describeParameterEncryptionRequest)  Unknown
    System.Data.dll!System.Data.SqlClient.SqlCommand.RunExecuteReader(System.Data.CommandBehavior cmdBehavior, System.Data.SqlClient.RunBehavior runBehavior, bool returnStream, string method, System.Threading.Tasks.TaskCompletionSource<object> completion, int timeout, out System.Threading.Tasks.Task task, out bool usedCache, bool asyncWrite, bool inRetry)   Unknown
    System.Data.dll!System.Data.SqlClient.SqlCommand.RunExecuteReader(System.Data.CommandBehavior cmdBehavior, System.Data.SqlClient.RunBehavior runBehavior, bool returnStream, string method) Unknown
    System.Data.dll!System.Data.SqlClient.SqlCommand.ExecuteReader(System.Data.CommandBehavior behavior, string method) Unknown
    System.Data.dll!System.Data.SqlClient.SqlCommand.ExecuteDbDataReader(System.Data.CommandBehavior behavior)  Unknown
    System.Data.dll!System.Data.Common.DbCommand.ExecuteReader(System.Data.CommandBehavior behavior)    Unknown
    EntityFramework.dll!System.Data.Entity.Infrastructure.Interception.DbCommandDispatcher.Reader.AnonymousMethod__c(System.Data.Common.DbCommand t, System.Data.Entity.Infrastructure.Interception.DbCommandInterceptionContext<System.Data.Common.DbDataReader> c)    Unknown
    EntityFramework.dll!System.Data.Entity.Infrastructure.Interception.InternalDispatcher<System.Data.Entity.Infrastructure.Interception.IDbCommandInterceptor>.Dispatch<System.Data.Common.DbCommand, System.Data.Entity.Infrastructure.Interception.DbCommandInterceptionContext<System.Data.Common.DbDataReader>, System.Data.Common.DbDataReader>(System.Data.Common.DbCommand target, System.Func<System.Data.Common.DbCommand, System.Data.Entity.Infrastructure.Interception.DbCommandInterceptionContext<System.Data.Common.DbDataReader>, System.Data.Common.DbDataReader> operation, System.Data.Entity.Infrastructure.Interception.DbCommandInterceptionContext<System.Data.Common.DbDataReader> interceptionContext, System.Action<System.Data.Entity.Infrastructure.Interception.IDbCommandInterceptor, System.Data.Common.DbCommand, System.Data.Entity.Infrastructure.Interception.DbCommandInterceptionContext<System.Data.Common.DbDataReader>> executing, System.Action<System.Data.Entity.Infrastructure.Interception.IDbCommandInterceptor, System.Data.Common.DbCommand, System.Data.Entity.Infrastructure.Interception.DbCommandInterceptionContext<System.Data.Common.DbDataReader>> executed)   Unknown
    EntityFramework.dll!System.Data.Entity.Infrastructure.Interception.DbCommandDispatcher.Reader(System.Data.Common.DbCommand command, System.Data.Entity.Infrastructure.Interception.DbCommandInterceptionContext interceptionContext)    Unknown
    EntityFramework.dll!System.Data.Entity.Internal.InterceptableDbCommand.ExecuteDbDataReader(System.Data.CommandBehavior behavior)    Unknown
    System.Data.dll!System.Data.Common.DbCommand.ExecuteReader(System.Data.CommandBehavior behavior)    Unknown
    EntityFramework.dll!System.Data.Entity.Core.EntityClient.Internal.EntityCommandDefinition.ExecuteStoreCommands(System.Data.Entity.Core.EntityClient.EntityCommand entityCommand, System.Data.CommandBehavior behavior)  Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.Internal.ObjectQueryExecutionPlan.Execute<bool>(System.Data.Entity.Core.Objects.ObjectContext context = {System.Data.Entity.Core.Objects.ObjectContext}, System.Data.Entity.Core.Objects.ObjectParameterCollection parameterValues) Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.ObjectQuery<bool>.GetResults.AnonymousMethod__6()   Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction<System.__Canon>(System.Func<System.__Canon> func, System.Data.Entity.Infrastructure.IDbExecutionStrategy executionStrategy, bool startLocalTransaction, bool releaseConnectionOnSuccess = false) Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.ObjectQuery<bool>.GetResults.AnonymousMethod__5()   Unknown
    EntityFramework.SqlServer.dll!System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute<System.Data.Entity.Core.Objects.ObjectResult<bool>>(System.Func<System.Data.Entity.Core.Objects.ObjectResult<bool>> operation)   Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.ObjectQuery<bool>.GetResults(System.Data.Entity.Core.Objects.MergeOption? forMergeOption)   Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.ObjectQuery<bool>.System.Collections.Generic.IEnumerable<T>.GetEnumerator.AnonymousMethod__0()  Unknown
    EntityFramework.dll!System.Data.Entity.Internal.LazyEnumerator<bool>.MoveNext() Unknown
    System.Core.dll!System.Linq.Enumerable.Single<bool>(System.Collections.Generic.IEnumerable<bool> source)    Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.ELinq.ObjectQueryProvider.GetElementFunction.AnonymousMethod__3<bool>(System.Collections.Generic.IEnumerable<bool> sequence)    Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.ELinq.ObjectQueryProvider.ExecuteSingle<bool>(System.Collections.Generic.IEnumerable<bool> query, System.Linq.Expressions.Expression queryRoot) Unknown
    EntityFramework.dll!System.Data.Entity.Core.Objects.ELinq.ObjectQueryProvider.Execute<bool>(System.Linq.Expressions.Expression expression)  Unknown
    EntityFramework.dll!System.Data.Entity.Internal.Linq.DbQueryProvider.Execute<bool>(System.Linq.Expressions.Expression expression)   Unknown
    System.Core.dll!System.Linq.Queryable.Any<ConsoleApplication31.Test>(System.Linq.IQueryable<ConsoleApplication31.Test> source)  Unknown
>   ConsoleApplication31.exe!ConsoleApplication31.TestClass.Test() Line 31  C#
    ConsoleApplication31.exe!ConsoleApplication31.Program.Main(string[] args = {string[0]}) Line 12 C#
    [Native to Managed Transition]  
    [Managed to Native Transition]  
    mscorlib.dll!System.AppDomain.ExecuteAssembly(string assemblyFile, System.Security.Policy.Evidence assemblySecurity, string[] args) Unknown
    Microsoft.VisualStudio.HostingProcess.Utilities.dll!Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()   Unknown
    mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context(object state)    Unknown
    mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)   Unknown
    mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)   Unknown
    mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Unknown
    mscorlib.dll!System.Threading.ThreadHelper.ThreadStart()    Unknown

Try instantiating your instance of your `TestEntity` outside of the loop. You're creating a lot of instances of it very very fast. — Dispersia, Feb 06 '17 at 18:02
This really doesn't make much sense. A `StackOverflowException` usually only occurs when there are [too many nested method calls](https://msdn.microsoft.com/en-us/library/system.stackoverflowexception.aspx). Can you interrupt the app half way through and check the call stack? — DavidG, Feb 06 '17 at 18:20
@Wiktor The example code you provided does NOT cause a memory leak or throw an exception. There must be something else going on here. What version of EF are you using? What data provider is in use? How many rows do you have in the `Tests` table? — DavidG, Feb 06 '17 at 18:41
I also just tried to reproduce and this does not throw for me. Are you sure the connection is stable / configured properly? — Dispersia, Feb 06 '17 at 18:43
@DavidG: My EF version is 6.1.3. Version of Sql Server is 2016 Developer edition. I've also tested that on 6.0.0. Test table is containing 1 record. — Wiktor, Feb 06 '17 at 18:44
@Dispersia: I think so that should be. Test database is on my PC. — Wiktor, Feb 06 '17 at 18:45
There is no issue with the code you provided, it works fine. There must be something else you are not showing us. — DavidG, Feb 06 '17 at 18:46
@DavidG I have no idea what that could be. I've created that test also on my machine to be sure where is problem. I've also downloaded latest updates for Sql Server, Windows... I don't know what else could cause this... — Wiktor, Feb 06 '17 at 18:49
@DavidG: Yes, of course. I've started looking the issue on my machine. I don't know, maybe some Windows update or VS. I am not sure thought. — Wiktor, Feb 06 '17 at 20:28
@DavidG: Around 200 000 to crash, but after just 20 000 i have more than 200MB of memory usage. Which is quite a lot. — Wiktor, Feb 06 '17 at 20:56
@DavidG : It's weird but it seems to be my Windows installation related issue. I don't know how yet but on my other Windows 10 everything seems fine... — Wiktor, Feb 06 '17 at 22:26

score 1 · Accepted Answer · answered Feb 13 '17 at 04:54

1

I still have no idea why this issue is occuring but now I know that it's related to my Sql Server installation.

When I am using Sql Server installed in Hyper-V then everything seems to be fine.

As fast as I change connection string to point to my localhost Sql server instance then I've got this weird leak.

For now this is some kind of sollution for me, at least I can move on.

answered Feb 13 '17 at 04:54

Wiktor

754
1
7
24

Could it be related to how your client library connects to the SQL server? There are more options for a local SQL server than for a remote, such as named pipes. You said that the leaks happens in unmanaged memory, so I'm thinking this could be caused by a connection related bug. Try modifying the ConnectionString to turn off or modify connection pooling and see if that changes anything. Try specifying that the connection should happen with tcp. [msdn doc](https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.connectionstring(v=vs.110).aspx) – Jørn Jensen Nov 09 '17 at 14:14

EF6 Memory leak without any reason

1 Answers1