Large dataset in memory - .NET Framework and C#

Question

I have a DLL project that makes a singleton of all the display strings used in my application. The problem is I need a reference of this DLL in many independently running applications.

I understand that if I reference the DLL project in all the independently running applications they will all load the DLL into memory and make individual singletons to which they can refer, but this is redundancy I want to avoid.

Is there a way I can keep one instance of this singleton in the memory and reference it from all the independent projects?

I understand I may be sounding stupid and that this loading may not be happening at all, but I've been searching since about an hour now :)

I am using the .NET framework with C# as the target language.

Answer 1

Generally, different processes have different address spaces. Since a DLL file is usually loaded into a process address space it is necessary to write some special code to implement data sharing between processes. Usually, this is done using the Windows API, for example, with the help of memory mapped files. You can learn more about this at:

Sharing memory between two processes (C, Windows)

Using the Windows API is not bad, but it somehow contradicts the .NET Framework philosophy. I think it is better to create a client-server architecture when a single server can work with different clients and pass the required data to them. This can be done using, for example, Remoting or WCF.

Answer 2

I would not trouble myself with the memory consumption until it is really a problem. We can do some simple math: 2^13 (8192) strings in 2^3 (8) applications is 2^16 strings and if one string has average size of 2^8 (256) chars, the total number of bytes for all applications is 2 * 2^24 (32 MiB). That should not pose any problems even if the number of strings will be multiplied by 10.

The amount of work required by other solutions will make them infeasible at current stage. Also there will be no benefits of common string service, if most of the shared strings are used by all applications. It provides benefits only if a small portion of the strings are really shared by them all.

With remoting you will have to make sure that the singleton instance is initialized only in one place and you have to access it through interfaces. Most likely you want to create a local singleton instance that queries the strings from the remote instance. This solution can also be implemented afterwards if you keep using the old singleton implementation until other solutions are necessary.

If you use taskmanager to track memory usage, you should have a very big string in the singleton class.

public static Strings {
    private readonly string MemoryUsageTest;

    static Strings() {
#if DEBUG
        int size = 104000000;
        var text = "abcdefghijklmnopqrstuvwxyz";
        var sb = new StringBuilder(size);

        for (int i = 0; i < size / text.Length; ++i) {
            sb.Append(text);
        }

        MemoryUsageTest = sb.ToString();
#endif
    }
}

I am not so sure that remoting would be a good alternative at any point. Anyway, if we assume that it would be a good alternative, I would suggest that the current implementation should use a singleton class that implements an interface. This will make the use of remote instance much easier.

public interface IStrings {
    string HelloWorld { get; }
}

public sealed class LocalStringService : IStrings {
    internal LocalStringService() { }

    string IStrings.HelloWorld {
        get { return "Hello World!"; }
    }
}

public static class StringService {
    private static readonly IStrings SingletonInstance = new LocalStringService();

    // If empty static constructor does not make any sense, read this:
    // http://csharpindepth.com/Articles/General/Beforefieldinit.aspx
    static StringService() { }

    public static IStrings Instance {
        get { return SingletonInstance; }
    }
}

Now, if you want to make this remote, you will only need to rename LocalStringService to RemoteStringService, make it inherit MarshalByRefObj and change StringService to create a remote instance of it. In addition, the RemoteStringService should be in another assembly so that it is not loaded into memory with other classes.

Still, I doubt that this would give you any benefits in this case.

Answer 3

The string constants used are most likely interned. Call the string.IsInterned static function to see for a particular string. If that's the case, they are not taking excess memory, since every string is in CLR intern pool.

IsInterned

Answer 4

Make a service wrapper on this DLL file, and call the service from other applications instead.