6

Background (unnecessary, confusing, only for the curious)

I'm using the free version of Unity3D for Mobile and it doesn't allow me to use the System.Net.Sockets namespace on mobile devices. The problem is that I'm using a compiled .dll library (namely, IKVM) that references the System.Net.Sockets. I'm not actually using the classes in IKVM that references that references System.Net.Sockets, so instead of buying the $3000 Unity Pro mobile licenses, I made a stub library of the Sockets namespace called dudeprgm.Net.Sockets that just replaces all the classes and methods with stubs (I did this using the Mono source code).


My problem

I need to replace all System.Net.Sockets.* references in my dlls to dudeprgm.Net.Sockets.*. I know that something like this is possible and done by other people(See EDIT below, at the bottom of the page). I would like to know how to do it myself.

I was able to come up with the following code using Mono.Cecil.
It goes through all the IL instructions, checks if the operand is an InlineType, then checks if the inline type is part of System.Net.Sockets, then renames it to dudeprgm.Net.Sockets and writes it. **I'm not sure if this is the right way to go about "finding-and-replacing" in Mono.Cecil. Problem is, this doesn't catch all Sockets usages (see below).

private static AssemblyDefinition stubsAssembly;

static void Main(string[] args) {
    AssemblyDefinition asm = AssemblyDefinition.ReadAssembly(args[0]);
    stubsAssembly = AssemblyDefinition.ReadAssembly("Socket Stubs.dll");
    // ...
    // Call ProcessSockets on everything
    // ...
    asm.Write(args[1]);
}

/*
 * This will be run on every property, constructor and method in the entire dll given
 */
private static void ProcessSockets(MethodDefinition method) {
    if (method.HasBody) {
        Mono.Collections.Generic.Collection<Instruction> instructions = method.Body.Instructions;
        for (int i = 0; i < instructions.Count; i++) {
            Instruction instruction = instructions[i];
            if (instruction.OpCode.OperandType == OperandType.InlineType) {
                string operand = instruction.Operand.ToString();
                if (operand.StartsWith("System.Net.Sockets")) {
                    Console.WriteLine(method.DeclaringType + "." + method.Name + "(...) uses type " + operand);
                    Console.WriteLine("\t(Instruction: " + instruction.OpCode.ToString() + " " + instruction.Operand.ToString() + ")");
                    instruction.Operand = method.Module.Import(stubsAssembly.MainModule.GetType("dudeprgm.Net.Sockets", operand.Substring(19)));
                    Console.WriteLine("\tReplaced with type " + "dudeprgm.Net.Sockets" + operand.Substring(18));
                }
            }
        }
    }
}

It works fine, but only catches "simple" instructions. Decompiled with ildasm, I can see where it replaced the types like here:

box        ['Socket Stubs'/*23000009*/]dudeprgm.Net.Sockets.SocketOptionLevel/*01000058*/

But it didn't catch these "complex" instructions:

callvirt   instance void [System/*23000003*/]System.Net.Sockets.Socket/*0100003F*/::SetSocketOption(valuetype [System/*23000003*/]System.Net.Sockets.SocketOptionLevel/*01000055*/,
                                                                                                                                                valuetype [System/*23000003*/]System.Net.Sockets.SocketOptionName/*01000056*/,
                                                                                                                                                int32) /* 0A000094 */

Now the .dlls are a jumble of dudeprgm.Net.Sockets and System.Net.Sockets references.

I'm pretty sure that this is happening because I'm only changing OperandType.InlineTypes, but I'm not sure on how else to do this. I've tried looking around everywhere, but it seems to me like Mono.Cecil has no way to set operands to a string, everything seems to have to be done using the Cecil API only (https://stackoverflow.com/a/7215711/837703).

(Sorry if I'm using incorrect terms, I'm pretty new to IL in general.)

Question

How can I replace all places where System.Net.Sockets appear in Mono.Cecil, rather than just where the operand is an InlineType? I don't really want to go through every OperandType there is in Cecil, I was just looking for some find-and-replace method in Cecil where I wouldn't have to work with plain IL myself.

EDIT: (also unnecessary, confusing, and only for the curious)
This person was able to do something similar for $25: http://www.reddit.com/r/Unity3D/comments/1xq516/good_ol_sockets_net_sockets_for_mobile_without/.

Automatic patcher tool that detects and fixes socket usage in scripts and .dll.

    ...

"DLL's are patched using Mono.Cecil. ...

You can go look at the second screenshot at https://www.assetstore.unity3d.com/en/#!/content/13166 and see that it says that it can replace namespaces.

That library doesn't fit my needs, because 1) it doesn't rename to the namespace I want (dudeprgm.Net.Sockets), 2) the library that it is renaming to does not support all the System.Net.Sockets classes that IKVM needs, because IKVM uses pretty much every Sockets class and 3) it costs $25 and I don't really want to buy something that I'm not going to use. I just wanted to show that replacing namespace/type references in Mono.Cecil is possible.

Community
  • 1
  • 1
  • ...anybody? Please help. –  Aug 03 '14 at 08:28
  • If you can get ahold of the unity dlls you can disassemble them to see what's going on – Philip Pittle Aug 15 '14 at 23:56
  • @PhilipPittle Okay, but I'm not sure that would help. I don't think I would be able to edit them to allow Unity Free to compile code that references `System.Net.Sockets` (plus, think that that part of unity is just in a big .exe). I was looking more for a way of editing my IKVM dlls to not include `System.Net.Sockets`: basically, just replacing one namespace with another in a dll. :) –  Aug 16 '14 at 00:01

3 Answers3

11

[01] Similar problem

Your problem with replacing references to a dll (and types within) with another dll (and types within) is technically similar to problem known as

Google: "c# add strong name to 3rd party assembly"

In this problem you want to have your application signed by strong name and possibly installed into GAC or Ngen-ed, but your application depends on a legacy 3rd party library which does not have a strong name added at compile time, which breaks the requirement saying that a strong named assembly can use only strong named assemblies. You don't have source code for the 3rd party library, only binaries, so you can't recompile it (== "simplified description")

There are several solutions possible, 3 most typical being:

[02] Similar problem's solution #1

You can use ildasm/ilasm round trip, convert all binaries into text form, change all references into their strong name equivalents (recursively) and turn text back into code. Examples: http://buffered.io/posts/net-fu-signing-an-unsigned-assembly-without-delay-signing/ and https://stackoverflow.com/a/6546134/2626313

[03] Similar problem's solution #2

You can use tools already written to solve exactly this problem, example: http://brutaldev.com/post/2013/10/18/NET-Assembly-Strong-Name-Signer

[04] Similar problem's solution #3

You can create a tool crafted to match your exact needs. It is possible, I have done it, it took several weeks and the code weighs several thousand lines of code. For the most dirty work I have reused (with some slight modifications) mainly source code from (unordered):

  • ApiChange.Api.Introspection.CorFlagsReader.cs
  • GACManagerApi.Fusion
  • brutaldev/StrongNameSigner
  • icsharpcode/ILSpy
  • Mono.Cecil.Binary
  • Mono.Cecil.Metadata
  • Mono.ResGen
  • Ricciolo.StylesExplorer.MarkupReflection
  • and reading http://referencesource.microsoft.com/

[05] Your problem

Although the problem you have described looks like just a subset of what I'm describing above, it may well turn out to be the same problem if you want to use GAC installation which in turn requires strong name signing.

My recommendation for you is

[06] Your problem's solution #1

Give the easiest solution [02] a tryand to get into least trouble use ilasm/ildasm tools from the Mono package not the ones provided by Microsoft's .NET Framework (Microsoft's Resgen in .NET Framework 4.5 is broken and cannot round-trip resx format, Ildasm output does not handle non-ASCII characters correctly etc. While you can't fix Microsoft's broken closed source, you can fix Mono's open source but I did not have to.)

[07] Your problem's solution #2

If [06] does not work for you then study (debug) → ILSpy ← and study Mono documentation for various command line tools doing what you need and their sources - you'll see how exactly they use the Mono.Cecil library

If you face the need to validate strong named or even signed assemblies (tampering them will invalidate the signatures) or remove the signatures etc. You are going to dive into code longer than a simple Stack Overflow answer can describe.

[08] Your problem's solution #3

Lurking around what ILMerge does and how can point you to an easier solution

[09] Your problem's solution #4

Another easier solution might be (if IKVM supports it) hooking the AssemblyResolve event where you can remap dll name into physical dll, loaded e.g. From totally different file or from a resource stream etc. As shown in several answers of older Stack Overflow question Embedding DLLs in a compiled executable

(EDIT #1: after comments)

[10] Your problem's solution #5

If your more or less general question actually boils down into "How can I make IKVM.dll to use my socket classes instead of those from namespace System.Net.Sockets" then quite straightforward solution might be:

Compile and deploy your own customized version of IKVM.dll using source code available at http://www.ikvm.net/download.html - no binary Mono.Cecil magic needed.

As all code is open it should be possible to find and redirect all references pointing to namespace System.Net into dudeprgm.Net by

  • [10.1] get IKVM source code and all other prerequisites and make sure you can compile working IKVM.dll
  • [10.2] add dudeprgm.Net.cs project to the solution
  • [10.3] in all source files find and remove everything looking like using System.Net
  • [10.4] in all source files full text find and replace everything that looks like System.Net with dudeprgm.Net
  • [10.5] compile. When compiler complains about a missing symbol (that was before in the System.Net namespace) then add it to your stub file. goto [10.5]
  • [10.6] if the above step does not settle down as "build ok" after 2 hours then think about another solution (or get some sleep)
  • [10.7] check IKVM license (http://sourceforge.net/p/ikvm/wiki/License/) if there is something you must change/claim/acknowledge as the original source code was modified

(EDIT #2: after comments)

[11] Your problem's solution #6

If you choose track [04] and working with text files and ilasm/ildasm tools (style [02]) would not seem productive then below is the key relevant part of my automatic strong name signer which changes assembly references into other references using Mono.Cecil. The code is pasted as is (without lines of code before, after and all around) in a form that works for me. Reading keys: a is Mono.Cecil.AssemblyDefinition, b implements Mono.Cecil.IAssemblyResolver, key method in b instance is the method AssemblyDefinition Resolve(AssemblyNameReference name) which translates required DLL name into call to AssemblyDefinition.ReadAssembly(..). I did not need to parse the instruction stream, remapping assembly references was enough (I can paste here few other pieces from my code if needed)

/// <summary>
/// Fixes references in assembly pointing to other assemblies to make their PublicKeyToken-s compatible. Returns true if some changes were made.
/// <para>Inspiration comes from https://github.com/brutaldev/StrongNameSigner/blob/master/src/Brutal.Dev.StrongNameSigner.UI/MainForm.cs
/// see call to SigningHelper.FixAssemblyReference
/// </para>
/// </summary>
public static bool FixStrongNameReferences(IEngine engine, string assemblyFile, string keyFile, string password)
{
    var modified = false;

    assemblyFile = Path.GetFullPath(assemblyFile);

    var assemblyHasStrongName = GetAssemblyInfo(assemblyFile, AssemblyInfoFlags.Read_StrongNameStatus)
        .StrongNameStatus == StrongNameStatus.Present;

    using (var handle = new AssemblyHandle(engine, assemblyFile))
    {
        AssemblyDefinition a;

        var resolver = handle.GetAssemblyResolver();

        a = handle.AssemblyDefinition;

        foreach (var reference in a.MainModule.AssemblyReferences)
        {
            var b = resolver.Resolve(reference);

            if (b != null)
            {
                // Found a matching reference, let's set the public key token.
                if (BitConverter.ToString(reference.PublicKeyToken) != BitConverter.ToString(b.Name.PublicKeyToken))
                {
                    reference.PublicKeyToken = b.Name.PublicKeyToken ?? new byte[0];
                    modified = true;
                }
            }
        }

        foreach (var resource in a.MainModule.Resources.ToList())
        {
            var er = resource as EmbeddedResource;
            if (er != null && er.Name.EndsWith(".resources", StringComparison.OrdinalIgnoreCase))
            {
                using (var targetStream = new MemoryStream())
                {
                    bool resourceModified = false;

                    using (var sourceStream = er.GetResourceStream())
                    {
                        using (System.Resources.IResourceReader reader = new System.Resources.ResourceReader(sourceStream))
                        {
                            using (var writer = new System.Resources.ResourceWriter(targetStream))
                            {
                                foreach (DictionaryEntry entry in reader)
                                {
                                    var key = (string)entry.Key;
                                    if (entry.Value is string)
                                    {
                                        writer.AddResource(key, (string)entry.Value);
                                    }
                                    else
                                    {
                                        if (key.EndsWith(".baml", StringComparison.OrdinalIgnoreCase) && entry.Value is Stream)
                                        {
                                            Stream newBamlStream = null;
                                            if (FixStrongNameReferences(handle, (Stream)entry.Value, ref newBamlStream))
                                            {
                                                writer.AddResource(key, newBamlStream, closeAfterWrite: true);
                                                resourceModified = true;
                                            }
                                            else
                                            {
                                                writer.AddResource(key, entry.Value);
                                            }
                                        }
                                        else
                                        {
                                            writer.AddResource(key, entry.Value);
                                        }
                                    }
                                }
                            }
                        }

                        if (resourceModified)
                        {
                            targetStream.Flush();
                            // I'll swap new resource instead of the old one
                            a.MainModule.Resources.Remove(resource);
                            a.MainModule.Resources.Add(new EmbeddedResource(er.Name, resource.Attributes, targetStream.ToArray()));
                            modified = true;
                        }
                    }
                }
            }
        }

        if (modified)
        {
            string backupFile = SigningHelper.GetTemporaryFile(assemblyFile, 1);

            // Make a backup before overwriting.
            File.Copy(assemblyFile, backupFile, true);
            try
            {
                try
                {
                    AssemblyResolver.RunDefaultAssemblyResolver(Path.GetDirectoryName(assemblyFile), () => {
                        // remove previous strong name https://groups.google.com/forum/#!topic/mono-cecil/5If6OnZCpWo
                        a.Name.HasPublicKey = false;
                        a.Name.PublicKey = new byte[0];
                        a.MainModule.Attributes &= ~ModuleAttributes.StrongNameSigned;

                        a.Write(assemblyFile);
                    });

                    if (assemblyHasStrongName)
                    {
                        SigningHelper.SignAssembly(assemblyFile, keyFile, null, password);
                    }
                }
                catch (Exception)
                {
                    // Restore the backup if something goes wrong.
                    File.Copy(backupFile, assemblyFile, true);

                    throw;
                }
            }
            finally
            {
                File.Delete(backupFile);
            }
        }
    }

    return modified;
}

[12] Your turn

Community
  • 1
  • 1
xmojmr
  • 8,073
  • 5
  • 31
  • 54
  • +1 Thanks for the detailed list of options! I see that this is harder than I expected. Pity. I'm pretty new to IL, so I wanting a relatively quick way to do this without disassembling/reassembling code. –  Aug 19 '14 at 19:14
  • Okay, I'm trying to do **[02]**, using Mono's `monodis` and `ilasm`, but I keep getting this cryptic error: `[ERROR] FATAL UNHANDLED EXCEPTION: System.ArgumentException: Key duplication when adding: System.Collections.DictionaryEntry` –  Aug 19 '14 at 23:20
  • Any idea of what it could mean? I'm just doing `monodis`, then immediately doing `ilasm` on the generate IL, to see if it will compile without any modifications , but even that won't work. –  Aug 19 '14 at 23:26
  • @dudeprgm without seeing callstack and without knowing exactly which file did you try and the exact command lines - it can mean quite anything. I've added perhaps more feasible option **[10]** to my answer – xmojmr Aug 20 '14 at 04:04
  • 1
    I fixed the error by using Microsoft's `ilasm` and `ildasm` from the VS Command Prompt. I'm trying to write a script that will be able to do this for all my dlls. (Also, **[10]** won't exactly work for me because I have some closed source dlls that reference `System.Data`, which in turn references `System.Net.Sockets` and I'll have to do the same kind of thing with them too). –  Aug 20 '14 at 13:01
  • @dudeprgm I'm super happy that you have find a way that seems to work :relieved: I've Google out an old "[Mono-bugs][Bug 76980]..ilasm fails to assembly classes containing fields with the same name" dated 2005 which had similarly cryptic error message as you have seen. Maybe the equilibrium between Mono framework and Microsoft's framework had changed since I've hit the obstacles, sorry for misguiding you. Just in case that you settle back on Mono.Cecil, I have added a code snippet as **[11]** in my (again) edited answer – xmojmr Aug 20 '14 at 16:17
  • Thank you so much for your help. I didn't test out **[11]** yet, but [02] works fine. I made a script to do it for me, because I'm going to have to do it for about 25 dlls, see my "answer" (IL turned out to be actually quite readable). –  Aug 20 '14 at 17:57
  • @dudeprgm thanks for the bounty points. I can't see your "answer" yet. Actually publishing the script that works for you here as a **→ → [self-answer](http://stackoverflow.com/help/self-answer) ← ←** might be very valuable result of your 19 days long journey - for future readers targeting Unity3D for Mobile (it looks like overestimated but very popular platform recently) – xmojmr Aug 20 '14 at 18:13
  • Oh, sorry, when I wrote the comment, I hadn't posted the answer yet. The answer is up now. –  Aug 20 '14 at 18:18
  • 3
    What an amazing journey, stackoverflow should have more of these type of answers... This particular answer is great for anyone considering "any of the above" just to understand what they're getting into. – Jonno Sep 28 '15 at 15:19
2

This is actually meant to be an extension to @xmojmer's answer.

I wrote a small bash script to automate xmojmer's option [02]:

# vsilscript.sh

# Usage:
# Open Cygwin
# . vsilscript.sh <original .dll to patch>
# output in "Sockets_Patched" subdirectory
nodosfilewarning=true # just in case cygwin complains
ILASM_PATH="/cygdrive/c/Windows/Microsoft.NET/Framework64/" # modify for your needs
ILASM_PATH="$ILASM_PATH"$(ls "$ILASM_PATH" -v | tail -n 1)
ILDASM_PATH="/cygdrive/c/Program Files (x86)/Microsoft SDKs/Windows/v8.1A/bin/NETFX 4.5.1 Tools" # modify for your needs
PATH="$ILDASM_PATH:$ILASM_PATH:$PATH"
base=$(echo $1 | sed "s/\.dll//g")
ildasm /out=$base".il" /all /typelist $base".dll"
cp $base".il" "tempdisassembled.il"
cat "tempdisassembled.il" | awk '
BEGIN { print ".assembly extern socketstubs { }"; }
{ gsub(/\[System[^\]]*\]System\.Net\.Sockets/, "[socketstubs]dudeprgm.Net.Sockets", $0); print $0; }
END { print "\n"; }
' 1> $base".il" #change the awk program to swap out different types
rm "tempdisassembled.il"
mkdir "Sockets_Patched"
# read -p "Press Enter to assemble..."
ilasm /output:"Sockets_Patched/"$base".dll" /dll /resource:$base".res" $base".il"

Modify the ILASM_PATH and ILDASM_PATHvariables if your computer is 32bit or have ilasm and ildasm on different locations. This script assumes you are using Cygwin on Windows.

The modification happens in the awk command. It adds a reference to my library of stubs (called socketstubs.dll, and is stored in the same directory.) by adding

.assembly extern sockectstubs { }

at the beginning of the disassembled IL code. Then, for each line, it looks for [System]System.Net.Sockets and replaces them with [socketstubs]dudeprgm.Net.Sockets. It adds a newline at the end of the IL code, or else ilasm won't re-assemble it, as per the docs:

Compilation might fail if the last line of code in the .il source file does not have either trailing white space or an end-of-line character.

The final patched .dll will be placed in a new directory called "Sockets_Patched".

You can modify this script to swap any two namespaces in any dll:

  1. Replace the System in [System[^\]] to whatever assembly contains your old namespace
  2. Replace System\.Net\.Sockets with your old namespace and put a backslash in front of every period
  3. Replace all socketstubs in the script with your library containing the new namespace
  4. Replace dudeprgm.Net.Sockets with your new namespace
  • 1
    After applying your script... does your application run? I mean if you did not meet the assembly strong name problem. When a DLL binary gets modified, the DLL's checksum changes which can invalidate the (possible) digital signature and the strong name hash. As a result, .NET Framework may reject loading such DLL. So does it really work? – xmojmr Aug 20 '14 at 18:35
  • 1
    I don't think Unity does any such checking (or maybe it suppresses it), and I'm not putting it in the GAC, so it worked for me, at least. Maybe it's because unity Uses Mono. I'm not sure how much it might help other people though. –  Aug 20 '14 at 19:06
0

You could wrap the sockets objects in an interface and then dependency inject the implementation you really want. The code might be a little tedious, but you can swap out the sockets to anything you want after that without directly referencing System.Net.Sockets

Owen Johnson
  • 2,416
  • 2
  • 19
  • 23