Currupted file in non-english locale (encoding problem?)

Question

In my MSI Windows Installer I have a custom VBScript action which extracts some files from the 'Binary' table to the filesystem. This is the code I'm using:

Inspired by: https://www.itninja.com/question/how-to-call-an-exe-which-is-stored-in-a-binary-table-through-a-vbscript-custom-action-in-the-msi

Function ExtractFromBinary(ByVal binaryName, ByVal binaryOutputFile)

 Dim oFSO : Set oFSO = CreateObject("Scripting.FileSystemObject")

 Const msiReadStreamInteger = 0
 Const msiReadStreamBytes = 1
 Const msiReadStreamAnsi = 2 
 Const msiReadStreamDirect = 3

 Dim binaryView : Set binaryView = Session.Database.OpenView("SELECT Data FROM Binary WHERE Name = '" & binaryName & "'") 
 binaryView.Execute

 Dim binaryRecord : Set binaryRecord = binaryView.Fetch 
 Dim binaryData : binaryData = binaryRecord.ReadStream(1, binaryRecord.DataSize(1), msiReadStreamAnsi) 
 Set binaryRecord = Nothing

 Dim binaryStream : Set binaryStream = oFSO.CreateTextFile(binaryOutputFile, True, False) 
 binaryStream.Write binaryData
 binaryStream.Close
 Set binaryStream = Nothing 

End Function

This has been used without any issues in production for 2-3 years now. However now we have a case on a Japanese Windows installation where the extracted binary files are corrupted:

As you can see, the problem typically after a '?' where the script either inserts an 'E', or overwrites the following character.

Both the ReadStream method and the CreateTextFile method have a parameter which affect encoding. The combination shown above seems to be the only one which works on my English Windows 10.

What do I need to change in the code above to make it work also on a Japanese system?

Answer 1

@Robert-Hegner I'll propose this as an answer, even though it is subject to your testing (I have no way of testing where I am)!

I've included an updated approach here (you will need to scroll down to the second example)

It uses msiReadStreamDirect (not msiReadStreamAnsi) to extract a string of Byte pairs, converts these into binary and creates the output file using the ADODB.Stream (not the FSO).

Dim oFSO : Set oFSO = CreateObject("Scripting.FileSystemObject")

Dim tempFolder : tempFolder = oFSO.GetSpecialFolder(2) 
Dim outputFile : outputFile = tempFolder & "\notepad.exe"

extractFromBinary "notepad", outputFile

Function MultiByteToBinary(MultiByte)
  'obtained from http://www.motobit.com
  'MultiByteToBinary converts multibyte string To real binary data (VT_UI1 | VT_ARRAY)
  'Using recordset
  Dim RS, LMultiByte, Binary
  Const adLongVarBinary = 205
  Set RS = CreateObject("ADODB.Recordset")
  LMultiByte = LenB(MultiByte)
  If LMultiByte>0 Then
    RS.Fields.Append "mBinary", adLongVarBinary, LMultiByte
    RS.Open
    RS.AddNew
      RS("mBinary").AppendChunk MultiByte & ChrB(0)
    RS.Update
    Binary = RS("mBinary").GetChunk(LMultiByte)
  End If
  Set RS = Nothing
  MultiByteToBinary = Binary
End Function

Function SaveBinaryData(FileName, ByteArray)
  Const adTypeBinary = 1
  Const adSaveCreateOverWrite = 2

  'Create Stream object
  Dim BinaryStream
  Set BinaryStream = CreateObject("ADODB.Stream")

  'Specify stream type - we want To save binary data.
  BinaryStream.Type = adTypeBinary

  'Open the stream And write binary data To the object
  BinaryStream.Open
  BinaryStream.Write ByteArray

  'Save binary data To disk
  BinaryStream.SaveToFile FileName, adSaveCreateOverWrite

  Set BinaryStream = Nothing
End Function

Function extractFromBinary(ByVal binaryName, ByVal binaryOutputFile)

    Const msiReadStreamInteger = 0 
    Const msiReadStreamBytes = 1 
    Const msiReadStreamAnsi = 2  
    Const msiReadStreamDirect = 3

    Dim binaryView : Set binaryView = Session.Database.OpenView("SELECT * FROM Binary WHERE Name = '" & binaryName & "'")  
    binaryView.Execute

    Dim binaryRecord : Set binaryRecord = binaryView.Fetch  
    Dim binaryData : binaryData = binaryRecord.ReadStream(2, binaryRecord.DataSize(2), msiReadStreamDirect)  
    Set binaryRecord = Nothing  

    'convert to string of byte pairs to binary
    binaryData = MultiByteToBinary(binaryData)

    'save binary data
    SaveBinaryData binaryOutputFile, binaryData

End Function

Set oFSO = Nothing

Answer 2

Japanese Code Page: From this blog entry: "Binary Files and the File System Object Do Not Mix": "In the Japanese code page, just-plain-chr(E0) is not even a legal character, so Chr will turn it into a zero... Do not use the FSO to read/write binary files, you're just asking for a world of hurt as soon as someone in DBCS-land runs your code."

---

Alternatives? How about .NET? I realized too late that you are in a custom action, I made the samples as standalone .NET console applications. The WiX framework has mechanisms to create a DTF custom action. Found this on github.com.

Rehashing?: Can we ask what you are actually doing? Why do you need to extract files this way? There could be other approaches that are more reliable if you explain the scenario?

---

DTF / .NET: Though I am not a huge .NET fan for deployment use (too many layers of dependencies), I think you would do better using .NET / DTF for this. What is DTF?

Sample DTF C# Application: Below is a simple, C# sample application showing one way to extract a binary stream from the Binary table (there are several other ways, I am not a .NET expert).

1. Create a new C# Console App (.NET Framework).
2. Paste the below code in and adjust parameters.
3. Add reference to Microsoft.Deployment.WindowsInstaller.dll (DTF framework).

using Microsoft.Deployment.WindowsInstaller;

namespace MSIExtractBinaryTableEntry
{
    class Program
    {
        static void Main(string[] args)
        {
            // ADJUST 1: Name of Binary Table Entry
            var binarytableentry = "ImageBmp"; 

            // ADJUST 2: Source MSI path
            var msifullpath = @"C:\MySetup.msi";

            // ADJUST 3: Output target path for binary stream
            var binaryfileoutputpath = @"C:\Output.XXX";

            using (var db = new Database(msifullpath, DatabaseOpenMode.ReadOnly))
            {
                using (var binaryView = db.OpenView("SELECT Name, Data FROM Binary WHERE Name='" + binarytableentry + "'"))
                {
                    binaryView.Execute();
                    binaryView.Fetch().GetStream(2, binaryfileoutputpath); // force overwrites output path
                }
            }
        }
    }
}

---

Alternative: Here is a tweak that exports the whole Binary Table to a folder called "Output" on the user's desktop.

Same procedure to create a test project as above. Only one parameter to specify: the full path to the input MSI.

using System;
using System.IO;
using Microsoft.Deployment.WindowsInstaller;

namespace MSIExtractBinaryTableEntry
{
    class Program
    {
        static void Main(string[] args)
        {
            // ADJUST 1: Specify MSI file path
            var msifullpath = @"C:\MySetup.msi";

            var outputpath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), @"Output\");
            Directory.CreateDirectory(outputpath);

            using (var db = new Database(msifullpath, DatabaseOpenMode.ReadOnly))
            {
                using (var binaryView = db.OpenView("SELECT Name, Data FROM Binary"))
                {
                    binaryView.Execute();

                    foreach (var rec in binaryView)
                    {
                        rec.GetStream("Data", outputpath + rec.GetString("Name"));
                    }
                }
            }
        }
    }
}

Answer 3

Here is what I ended up with.

As suggested by Stein Åsmul I rewrote the custom action using C# (.NET / DTF). Initially I was hesitant to writing custom actions in C# as it introduces additional prerequisites to the installer. But it turns out that if the custom action targets .NET Framework 2.0, it should be supported on most machines without the need to manually install the framework (see here).

So here is my code:

public static class TemporaryFilesExtractor
{

    [CustomAction]
    public static ActionResult ExtractTemporaryFiles(Session session)
    {
        ExtractFromBinary(session, "binaryname1", "<filePath1>");
        ExtractFromBinary(session, "binaryname2", "<filePath2>");
        return ActionResult.Success;
    }

    private static void ExtractFromBinary(Session session, string binaryName, string binaryOutputFile)
    {
        session.Log($"Extracting {binaryName} to {binaryOutputFile}");
        byte[] buffer = new byte[4096];

        using (var view = session.Database.OpenView("SELECT Data FROM Binary WHERE Name = '{0}'", binaryName))
        {
            view.Execute();
            using (var record = view.Fetch())
            using (var dbStream = record.GetStream(1))
            using (var fileStream = File.OpenWrite(binaryOutputFile))
            {
                int count;
                while ((count = dbStream.Read(buffer, 0, buffer.Length)) != 0)
                    fileStream.Write(buffer, 0, count);
            }
        }
    }

}