9

So I was thinking of writing online c# compiler and execution environment. And of course problem #1 is security. I ended up creating a little-privileged appdomain for user code and starting it in a new process which is tightly monitored for cpu and memory consumption. Standard console application namespaces are available. So my question is this: can you think of ways of breaking something in some way? You can try your ideas on the spot rundotnet.

Edit2 If anyone cares about the code, there is now open source fork of this project: rextester at github

Edit1 As a response to one of the comments here are some code samples.

So basically you create a console application. I'll just post a big chunk of it:

class Sandboxer : MarshalByRefObject
{
    private static object[] parameters = { new string[] { "parameter for the curious" } };

    static void Main(string[] args)
    {
        Console.OutputEncoding = Encoding.UTF8;
        string pathToUntrusted = args[0].Replace("|_|", " ");
        string untrustedAssembly = args[1];
        string entryPointString = args[2];
        string[] parts = entryPointString.Split(new string[] { "|" }, StringSplitOptions.RemoveEmptyEntries);
        string name_space = parts[0];
        string class_name =  parts[1];
        string method_name = parts[2];

        //Setting the AppDomainSetup. It is very important to set the ApplicationBase to a folder 
        //other than the one in which the sandboxer resides.
        AppDomainSetup adSetup = new AppDomainSetup();
        adSetup.ApplicationBase = Path.GetFullPath(pathToUntrusted);

        //Setting the permissions for the AppDomain. We give the permission to execute and to 
        //read/discover the location where the untrusted code is loaded.
        PermissionSet permSet = new PermissionSet(PermissionState.None);
        permSet.AddPermission(new SecurityPermission(SecurityPermissionFlag.Execution));


        //Now we have everything we need to create the AppDomain, so let's create it.
        AppDomain newDomain = AppDomain.CreateDomain("Sandbox", null, adSetup, permSet, null);


        //Use CreateInstanceFrom to load an instance of the Sandboxer class into the
        //new AppDomain. 
        ObjectHandle handle = Activator.CreateInstanceFrom(
            newDomain, typeof(Sandboxer).Assembly.ManifestModule.FullyQualifiedName,
            typeof(Sandboxer).FullName
            );
        //Unwrap the new domain instance into a reference in this domain and use it to execute the 
        //untrusted code.
        Sandboxer newDomainInstance = (Sandboxer)handle.Unwrap();

        Job job = new Job(newDomainInstance, untrustedAssembly, name_space, class_name, method_name, parameters);
        Thread thread = new Thread(new ThreadStart(job.DoJob));
        thread.Start();
        thread.Join(10000);
        if (thread.ThreadState != ThreadState.Stopped)
        {
            thread.Abort();
            Console.Error.WriteLine("Job taking too long. Aborted.");
        }
        AppDomain.Unload(newDomain);
    }

    public void ExecuteUntrustedCode(string assemblyName, string name_space, string class_name, string method_name, object[] parameters)
    {
        MethodInfo target = null;
        try
        {
            target = Assembly.Load(assemblyName).GetType(name_space+"."+class_name).GetMethod(method_name);
            if (target == null)
                throw new Exception();
        }
        catch (Exception)
        {
            Console.Error.WriteLine("Entry method '{0}' in class '{1}' in namespace '{2}' not found.", method_name, class_name, name_space);
            return;
        }

        ...            

        //Now invoke the method.
        try
        {
            target.Invoke(null, parameters);
        }
        catch (Exception e)
        {
            ...
        }
    }
}

class Job
{
    Sandboxer sandboxer = null;
    string assemblyName;
    string name_space;
    string class_name;
    string method_name;
    object[] parameters;

    public Job(Sandboxer sandboxer, string assemblyName, string name_space, string class_name, string method_name, object[] parameters)
    {
        this.sandboxer = sandboxer;
        this.assemblyName = assemblyName;
        this.name_space = name_space;
        this.class_name = class_name;
        this.method_name = method_name;
        this.parameters = parameters;
    }

    public void DoJob()
    {
        try
        {
            sandboxer.ExecuteUntrustedCode(assemblyName, name_space, class_name, method_name, parameters);
        }
        catch (Exception e)
        {
            Console.Error.WriteLine(e.Message);
        }
    }
}

You compile the above and have executable which you start and monitor in a new process:

using (Process process = new Process())
{
    try
    {
        double TotalMemoryInBytes = 0;
        double TotalThreadCount = 0;
        int samplesCount = 0;

        process.StartInfo.FileName = /*path to sandboxer*/;
        process.StartInfo.Arguments = folder.Replace(" ", "|_|") + " " + assemblyName + " Rextester|Program|Main"; //assemblyName - assembly that contains compiled user code
        process.StartInfo.UseShellExecute = false;
        process.StartInfo.CreateNoWindow = true;
        process.StartInfo.RedirectStandardOutput = true;
        process.StartInfo.RedirectStandardError = true;

        DateTime start = DateTime.Now;
        process.Start();

        OutputReader output = new OutputReader(process.StandardOutput);
        Thread outputReader = new Thread(new ThreadStart(output.ReadOutput));
        outputReader.Start();
        OutputReader error = new OutputReader(process.StandardError);
        Thread errorReader = new Thread(new ThreadStart(error.ReadOutput));
        errorReader.Start();


        do
        {
            // Refresh the current process property values.
            process.Refresh();
            if (!process.HasExited)
            {
                try
                {
                    var proc = process.TotalProcessorTime;
                    // Update the values for the overall peak memory statistics.
                    var mem1 = process.PagedMemorySize64;
                    var mem2 = process.PrivateMemorySize64;

                    //update stats
                    TotalMemoryInBytes += (mem1 + mem2);
                    TotalThreadCount += (process.Threads.Count);
                    samplesCount++;

                    if (proc.TotalSeconds > 5 || mem1 + mem2 > 100000000 || process.Threads.Count > 100 || start + TimeSpan.FromSeconds(10) < DateTime.Now)
                    {
                        var time = proc.TotalSeconds;
                        var mem = mem1 + mem2;
                        process.Kill();

                        ...
                    }
                }
                catch (InvalidOperationException)
                {
                    break;
                }
            }
        }
        while (!process.WaitForExit(10)); //check process every 10 milliseconds
        process.WaitForExit();
        ...
}

...

class OutputReader
{
    StreamReader reader;
    public string Output
    {
        get;
        set;
    }
    StringBuilder sb = new StringBuilder();
    public StringBuilder Builder
    {
        get
        {
            return sb;
        }
    }
    public OutputReader(StreamReader reader)
    {
        this.reader = reader;
    }

    public void ReadOutput()
    {
        try
        {                
            int bufferSize = 40000;
            byte[] buffer = new byte[bufferSize];
            int outputLimit = 200000;
            int count;
            bool addMore = true;
            while (true)
            {
                Thread.Sleep(10);
                count = reader.BaseStream.Read(buffer, 0, bufferSize);
                if (count != 0)
                {
                    if (addMore)
                    {
                        sb.Append(Encoding.UTF8.GetString(buffer, 0, count));
                        if (sb.Length > outputLimit)
                        {
                            sb.Append("\n\n...");
                            addMore = false;
                        }
                    }
                }
                else
                    break;
            }
            Output = sb.ToString();
        }
        catch (Exception e)
        {
           ...
        }
    }
}

Assemblies that user code can use are added at compile time:

CompilerParameters cp = new CompilerParameters();
cp.GenerateExecutable = false;
cp.OutputAssembly = ...
cp.GenerateInMemory = false;
cp.TreatWarningsAsErrors = false;
cp.WarningLevel = 4;
cp.IncludeDebugInformation = false;

cp.ReferencedAssemblies.Add("System.dll");
cp.ReferencedAssemblies.Add("System.Core.dll");
cp.ReferencedAssemblies.Add("System.Data.dll");
cp.ReferencedAssemblies.Add("System.Data.DataSetExtensions.dll");
cp.ReferencedAssemblies.Add("System.Xml.dll");
cp.ReferencedAssemblies.Add("System.Xml.Linq.dll");

using (CodeDomProvider provider = CodeDomProvider.CreateProvider(/*language*/))
{
    cr = provider.CompileAssemblyFromSource(cp, new string[] { data.Program });
}
ren
  • 3,843
  • 9
  • 50
  • 95
  • 1
    What if I created thousands of backgroundworker threads? – Anirudh Ramanathan Jun 10 '11 at 21:54
  • 1
    I cant answer your question but i really do support the idea! its very usefull! – Polity Jun 10 '11 at 21:57
  • 1
    @Anirudwha... Tried that and it got nicely killed – Polity Jun 10 '11 at 22:00
  • 1
    Nice little project. Tried mucking around with the System.IO namespace and with AppDomain, trying to get at the file system on the server. Kept getting exceptions from the CodeAccessSecurityEngine (which is good!). Without knowing how this is set up, one thing to check would be if you could enumerate the files in the web application directory. – rsbarro Jun 10 '11 at 22:05
  • 1
    I noticed unsafe code is disabled, which is good too. Keep it that way – JBSnorro Jun 10 '11 at 22:56
  • 1
    @ren - pretty cool project. what do you use it for? also how do you do non .NET languages? Does it run on a *nix environment? – DotnetDude Nov 08 '11 at 03:58
  • 1
    @DotnetDude Thanks. Personally, I use it for quick testing if I am not sure whether something will work as expected. The basic idea is that it's (hopefully) faster than doing this in a separate visual studio project. Non .net languages run on ubuntu 11.04. There is a short explanation of how this is done on home page. And based on my experience it is way harder to set up secure environment using ubuntu's security mechanisms than it is with .net which gives sandboxing functionality almost out of the box. – ren Nov 08 '11 at 16:46
  • 1
    @ren - LOL, infact I wanted to create something like this for exactly the same reason - to be able to test something fast. I got as far as executing the .net code using `CodeDom` although people have suggested using the `Mono` compiler as a service. I run it as a separate Process, but I need to some how isolate it (create an appdomain maybe?). Also, you mention a python wrapper for non .net languages. What's the intent of this wrapper? – DotnetDude Nov 08 '11 at 21:52
  • 1
    @DotnetDude Right, as described [here](http://stackoverflow.com/questions/5161708/online-c-sharp-interpreter-security-issues/6319291#6319291) appdomain is used. Basically you compile user code in an assembly, then invoke it in an appdomain with execution permisson only. In the link above there is info (at msdn) how to switch from trusted appdomain to unprivileged one. This takes care of so many security problems, which in linux give a headache. All this stuff is started in a separate process, because it's easy to monitor for things like cpu and memory consumption and is easy to kill. (Cont.) – ren Nov 08 '11 at 23:36
  • 1
    @DotnetDude As for python wrapper - well, what it does is makes call to `setrlimt` system call, which sets limits on cpu, memory, etc consumption on child processes. Then it starts user code as a subprocess and kills it if doesn't terminate in time. This was a humble attempt to enforce security. There are other measures like iptables and stuff. But overall it's not as secure as in .net: user code can make system calls, can execute different utilities, read files. That's why I didn't ask a question about whether anyone here on SO could break it - I'm pretty sure it could be done:) Well... – ren Nov 08 '11 at 23:46
  • 1
    @ren Thanks for the appdomain link. Will review it soon. Did you consider running non .NET languages from a Windows machine (ex: Java can run on windows - http://download.oracle.com/javase/tutorial/getStarted/cupojava/win32.html) so that you can still use the appdomain approach to make it secure for these languages as well? It appears that you host that site on windows and send requests for compiling/executing java,c,c++ code to a ubuntu box (via a webservice?) – DotnetDude Nov 09 '11 at 04:29
  • 1
    @DotnetDude Appdomains are .net concepts, not operating system's (I think). So you can create appdomains and execute .net compiled code in them, but if you'd want to execute c++ compiled binary then you'd have to create new process that would have nothing to do with .net. You are right about hosting and service, I guess other languages could be hosted on windows too, but in linux this seems to be an easier task. – ren Nov 09 '11 at 14:20
  • 1
    @ren - In your SO answer you linked you mention - "b) create new process and start your appdomain in it." How exactly did you do this. The msdn link doesn't create a separate process. Also, how do you monitor cpu usage on the process? Any code sample will be really helpful. Thanks! – DotnetDude Nov 10 '11 at 05:24
  • 1
    @DotnetDude Ok, edited my question to give some code. Hope this is helpful. – ren Nov 10 '11 at 22:51
  • 1
    @ren - Yes, this is definitely helpful. Thanks! By the way, is the user's code in C# limited by the assemblies that have been referenced? For example, if user's code includes linq to xml, how do they add a reference to `System.Xml.Linq.dll`. Have you already added reference to the most commonly used dlls? – DotnetDude Nov 12 '11 at 03:44
  • 1
    Also, `OutputReader output = new OutputReader(process.StandardOutput); ` - What namespace is the `OutputReader` in or is this a custom type ? – DotnetDude Nov 12 '11 at 03:51
  • 1
    @DotnetDude Edited question to add more code. – ren Nov 13 '11 at 11:32
  • What about console input? How do add support of it? – Dmitry Dec 07 '11 at 21:53
  • @Altaveron I guess [ProcessStartInfo.RedirectStandardInput](http://msdn.microsoft.com/en-us/library/system.diagnostics.processstartinfo.redirectstandardinput.aspx) would have helped here – ren Dec 08 '11 at 21:45

2 Answers2

3

Have you looked at Mono's Compiler as a service? I think that is pretty cool what they are doing, perhaps something there could be useful to you for this project.

Auðunn
  • 31
  • 2
2

For a good example of something similar to this already in existence, there is a place at http://www.topcoder.com, which has an "Algorithm Arena", where code is submitted and automatically scored. There are restrictions against using certain types of classes, such as Exception, but it may be a good idea to examine their application for a proof of concept.

marknuzz
  • 2,847
  • 1
  • 26
  • 29