Compile Brotli into a DLL .NET can reference

Question

So I'd like to take advantage of Brotli but I am not familiar with Python and C++..

I know someone had compiled it into a Windows .exe. But how do I wrap it into a DLL or something that a .NET app can reference? I know there's IronPython, do I just bring in all the source files into an IronPython project and write a .NET adapter that calls into the Brotli API and exposes them? But actually, I'm not even sure if the Brotli API is Python or C++..

Looking at tools/bro.cc, it looks like the "entry" methods are defined in encode.c and decode.c as BrotliCompress(), BrotliDecompressBuffer(), BrotliDecompressStream() methods. So I suppose a DLL can be compiled from the C++ classes.

Answer 1

To avoid the need for Python, I have forked the original brotli source here https://github.com/smourier/brotli and created a Windows DLL version of it that you can use with .NET. I've added a directory that contains a "WinBrotli" Visual Studio 2015 solution with two projects:

• WinBrotli: a Windows DLL (x86 and x64) that contains original unchanged C/C++ brotli code.
• Brotli: a Windows Console Application (Any Cpu) written in C# that contains P/Invoke interop code for WinBrotli.

To reuse the Winbrotli DLL, just copy WinBrotli.x64.dll and WinBrotli.x86.dll (you can find already built release versions in the WinBrotli/binaries folder) aside your .NET application, and incorporate the BrotliCompression.cs file in your C# project (or port it to VB or another language if C# is not your favorite language). The interop code will automatically pick the right DLL that correspond to the current process' bitness (X86 or X64).

Once you've done that, using it is fairly simple (input and output can be file paths or standard .NET Streams):

        // compress
        BrotliCompression.Compress(input, output);

        // decompress
        BrotliCompression.Decompress(input, output);

To create WinBrotli, here's what I've done (for others that would want to use other Visual Studio versions)

• Created a standard DLL project, removed the precompiled header
• Included all encoder and decoder original brotli C/C++ files (never changed anything in there, so we can update the original files when needed)
• Configured the project to remove dependencies on MSVCRT (so we don't need to deploy other DLL)
• Disabled the 4146 warning (otherwise we just can't compile)
• Added a very standard dllmain.cpp file that does nothing special
• Added a WinBrotli.cpp file that exposes brotli compression and decompression code to the outside Windows world (with a very thin adaptation layer, so it's easier to interop in .NET)
• Added a WinBrotli.def file that exports 4 functions

Answer 2

I'll show one way to do that via calling python native library from .NET code. What you need:

1. You need to intall python 2.7 (hope that is obvious)

2. You need to compile brotli from source. Hopefully that is easy. First install Microsoft Visual C++ compiler for Python 2.7. Then clone brotli repository via git clone https://github.com/google/brotli.git and compile using python setup.py build_ext. When it's done, in build\lib.win32-2.7 directory you will find brotli.pyd file. This is python c++ module - we will need it later.

3. You need to either download pythonnet binaries or compile it from source. The reason we use pythonnet here, and not for example Iron Python is because Iron Python does not support native (C\C++) python modules, and that is what we need here. So, to compile from source, clone via git clone https://github.com/pythonnet/pythonnet.git then compile via python setup.py build. In result you will get Python.Runtime.dll (in build\lib.win32-2.7 directory), which is what we need.

When you have all that in place, create console project, reference Python.Runtime.dll and then:

public static void Main()
{            
    PythonEngine.Initialize();            
    var gs = PythonEngine.AcquireLock();
    try {                
        // import brotli module
        dynamic brotli = PythonEngine.ImportModule(@"brotli");
        // this is a string we will compress
        string original = "XXXXXXXXXXYYYYYYYYYY";
        // compress and interpret as byte array. This array you can save to file for example
        var compressed = (byte[]) brotli.compress(original);                
        // little trick to pass byte array as python string
        dynamic base64Encoded = new PyString(Convert.ToBase64String(compressed));
        // decompress and interpret as string
        var decompressed = (string) brotli.decompress(base64Encoded.decode("base64"));
        // works
        Debug.Assert(decompressed == original);
    }
    finally {
        PythonEngine.ReleaseLock(gs);
        PythonEngine.Shutdown();
    }            
    Console.ReadKey();
}

Then build that and put brotli.pyc you get above in the same directory with your .exe file. After all that manipulations you will be able to compress and decompress from .NET code, as you see above.

Answer 3

You may use Brotli.NET which provides full stream support.

1. github: https://github.com/XieJJ99/brotli.net/.
2. Nuget: https://www.nuget.org/packages/Brotli.NET/.

To compress a stream to brotli data:

   public Byte[] Encode(Byte[] input)
   {
       Byte[] output = null;
       using (System.IO.MemoryStream msInput = new System.IO.MemoryStream(input))
       using (System.IO.MemoryStream msOutput = new System.IO.MemoryStream())
       using (BrotliStream bs = new BrotliStream(msOutput, System.IO.Compression.CompressionMode.Compress))
       {
           bs.SetQuality(11);
           bs.SetWindow(22);
           msInput.CopyTo(bs);
           bs.Close();
           output = msOutput.ToArray();
           return output;
       }
   }

To decompress a brotli stream:

   public Byte[] Decode(Byte[] input)
   {
       using (System.IO.MemoryStream msInput = new System.IO.MemoryStream(input))
       using (BrotliStream bs = new BrotliStream(msInput, System.IO.Compression.CompressionMode.Decompress))
       using (System.IO.MemoryStream msOutput = new System.IO.MemoryStream())
       {
           bs.CopyTo(msOutput);
           msOutput.Seek(0, System.IO.SeekOrigin.Begin);
           output = msOutput.ToArray();
           return output;
       }

   }

To support dynamic compress in web applications,add the code like this in the Global.asax.cs:

    protected void Application_PostAcquireRequestState(object sender, EventArgs e)
    {
                       var app = Context.ApplicationInstance;
            String acceptEncodings = app.Request.Headers.Get("Accept-Encoding");

            if (!String.IsNullOrEmpty(acceptEncodings))
            {
                System.IO.Stream baseStream = app.Response.Filter;
                acceptEncodings = acceptEncodings.ToLower();

                if (acceptEncodings.Contains("br") || acceptEncodings.Contains("brotli"))
                {
                    app.Response.Filter = new Brotli.BrotliStream(baseStream, System.IO.Compression.CompressionMode.Compress);
                    app.Response.AppendHeader("Content-Encoding", "br");
                }
                else
                if (acceptEncodings.Contains("deflate"))
                {
                    app.Response.Filter = new System.IO.Compression.DeflateStream(baseStream, System.IO.Compression.CompressionMode.Compress);
                    app.Response.AppendHeader("Content-Encoding", "deflate");
                }
                else if (acceptEncodings.Contains("gzip"))
                {
                    app.Response.Filter = new System.IO.Compression.GZipStream(baseStream, System.IO.Compression.CompressionMode.Compress);
                    app.Response.AppendHeader("Content-Encoding", "gzip");
                }

            }
       }