How to speed up Cross-AppDomain calls?

Question

We have developed some system which works with DB queues and which components reside in a separate Windows Services. We have an idea to execute all the components within a single executable but in separated AppDomains. The main purpose is to prevent the whole system crashing when a single components crash.

After performing few tests we have faced the problem of Cross-AppDomain calls being too slow. I tried to sign the assemblies with the strong name and put them to GAC and also applied LoaderOptimization.MultiDomain(Host), but the performance is still not acceptable: 14 000 000 calls/sec with simple single AppDomain calls and 90 000 calls/sec in Cross-Domain calls mode. In the application two calls are performed: first one from Default AppDomain to the Non-Default AppDomain, the second one - in the opposite direction. 90 000 calls/sec is very good result, but considering the simplicity of the test application and after projecting the complexity of our system to the multiple AppDomain approach we figured out that it is way too slow.

The main question is: Is there a way to significantly speed up the cross-domain calls? Maybe some sort of checks can be eliminated?

The code:

class Program
{
    public class TestMessage : MarshalByRefObject { }

    public class TestClass : MarshalByRefObject
    {
        private TestClass() { }


        public void TestMethod(TestMessage message) { }


        public static TestClass Create(bool defaultDomain)
        {
            if (defaultDomain)
            {
                return TestClass.createDefaultDomain();
            }

            return TestClass.createInNewDomain();
        }


        private static TestClass createDefaultDomain()
        {
            return new TestClass();
        }

        private static TestClass createInNewDomain()
        {
            AppDomain appDomain = AppDomain.CreateDomain("NewDomain");

            return appDomain.CreateInstance(
                typeof(TestClass).Assembly.FullName,
                typeof(TestClass).FullName,
                true, BindingFlags.NonPublic | BindingFlags.Instance,
                null, null, null, null)
                .Unwrap() as TestClass;
        }
    }

    [STAThread]
    [LoaderOptimization(LoaderOptimization.MultiDomain)]
    static void Main(string[] args)
    {
        TestClass testClass = TestClass.Create(true);
        TestMessage message = new TestMessage();

        int calls = 0;

        Stopwatch sw = Stopwatch.StartNew();

        while (sw.ElapsedMilliseconds < 10000)
        {
            testClass.TestMethod(message);

            calls++;
        }

        sw.Start();

        Console.WriteLine("{0:N3}", calls / sw.Elapsed.TotalSeconds);
    }
}

A main point of the AppDomain concept is to provide isolation. Removing security checks would negate that and is, I assume, not possible. At least some of the slowness comes from marshaling your data across the AppDomain boundary. So one approach would be to work to minimize that. Passing only value types or strings, for example. Do your own parameter serialization if you need to. If the remaining AppDomain overhead is still unacceptable, perhaps use a different mechanism to communicate between AppDomains, such as named pipes, sockets, shared memory, etc. — Peter Duniho, Dec 12 '14 at 20:10
@PeterDuniho, when passing pure serializable objects across AppDomains the performance is pretty high, but the problem is that we need to pass MarshalByRefObj derived objects through the AppDomain-s which indeed slows the performance down. — Rauf, Dec 12 '14 at 20:17
However they are passed, even your `MarshalByRefObject` types will be serialized/deserialized. The only question is whether you leave it to .NET, or do it yourself. My point is that if you can take over that process, such that the actual cross-domain call is simply a string (i.e. the serialized object as a string), then you have more control over the serialization and so can optimize it (e.g. if the same object is passed multiple times, serialize it once and reuse the serialized string for each call). — Peter Duniho, Dec 12 '14 at 20:32
@PeterDuniho, we need to pass .NET objects, which are MarshalByRegObj-s and were not marked as Serializable, I'm afraid serializing them will be impossible. — Rauf, Dec 12 '14 at 21:41
I understand. And if you _must_ use `MarshalByRefObject`, then you seem to be stuck. But if the only reason these objects are `MarshalByRefObject` is so that you can use them cross-domain, then you should be able to accomplish similar results with a lighter-weight implementation (i.e. not using `MarshalByRefObject` and instead handing the cross-domain communication yourself) — Peter Duniho, Dec 12 '14 at 21:48

score 0 · Answer 1 · edited May 23 '17 at 12:22

It seems with multiple AppDomains you are out of luck:

A direct invocation is just a few opcodes, but with remoting there are multiple classes involved, real/transparent proxies, security checks, serialization

If your only concern is that component crashes don't spread (rather than e.g. preventing the memory leak associated with dynamically loading into a single AppDomain, or preventing direct access), you might be able to address the issue by try/catching all invocations of code in these components. If you are considering/using multiple AppDomains already, there can't be that many different points of interaction anyway, so try/catching that, plus perhaps try/catching at the outermost levels of your (self-created) threads - which is a good idea in general anyway - could be good enough.

If you need to however prevent random third-party code from accessing other code/data, a single domain is out of the question.

How to speed up Cross-AppDomain calls?

1 Answers1