JavaScript: Convert binary string to Blob

Question

As entitled, how can I accomplish the following assuming the binary string is well-formed (regardless escape sequence) and that it is inline string (not requested by GET/XHR), where I want to use the converted string to BLOB in createObjectURL to avoid using Base64 encoding:

var FileBinaryString = '�PNG


IHDR��csRGB���IDATHKݖMJ�@��r=
�d�nA��9�����z��B��n���3q�CI�v1vW�`0����z��[��h�-��veb)7���c��0�|�� ��Wn���e���Ξ��{Ӝྋ�mZ�Jr��*
�I\ut"yN��O(%�/('%VHQ�P��Xv%r�Y����S���X����_J\�'UR�kir�p��c�(ɱ&���)�+��-J�^�^I�Pǚ�_�G�k%#F�_��v���wa��^��"���Tg���sz�ڼiIEND�B`�';

var BinaryToBlob = new Blob([FileBinaryString], {type:'image/png'});

UPDATE

I managed to return the binary file as hexadecimal string, where I have issue now to escape hex characters inside the generated string, see the following code:

var FileHex = "89504e470d0a1a0a0000000d494844520000009f00000021080200000048b9eeb2000000017352474200aece1ce90000000467414d410000b18f0bfc6105000000097048597300000ec300000ec301c76fa864000002f9494441546843ed986b76ab300c84b32e16c47a580d9b6131b97e4846b6474a741b4aeae3ef4f8b6d46230de929793c27e332d31d9999eec8cc744766a63b3233dd91f992748f6d790496eda085085c1c1867bffb1a4f3fd69dae0133ddefe1be745f55a67db396c16fa5cb3635cd57fb57782a38b567ba1db64fde558bfeb04d1b67bf33dd1ed3280d2c82abbe31d1ffc7d9ef4c17c00902a7225cdc091df8b02366a6fb01d48872bd655db5b15d1baeb7df2f49f7d8b775c9db916559b7bd9181f256cd4633882eeb7eb8c6d2ea52b920a3f4f2733fa544bc61e5e32c68e827f9743890a4bf225d5e8f848ec545a504e5b59a87e8b421f44d872c70bc79354d1e16d6ec38fc90c4bae59f0c1dd20a9ccf81848769757c6dba34c6fab6f0d4e6557116cae39a7c779c5cd908cf76ae6f774b40b35937174395b11b9f1f2e1c29c70ffe88dbfd9ef2d507d9ecd799ee4b402fdd3c8ae3f3303c8a16d9071035b65af8a8702bc32d06a554e739e1f3c34b3813daada454fd228594980bd34566997654f02c58641bb825908942a7430be5bacbd20ef74d3fa02301d8b5f4b1a38a0bff3253f570530fed94d3501e2c76f3aab066d1d068d365ef9e57fa0309a79fa66a03d8b5f41b8788df485765d9ccb6f56e958e8009957a6ef9aabaaf2a45ca9db0d30f6cb3a0f70b6fb06b27ae4f57bd4502e59ddd42133a522affae5787f6224e3faa4e02ec5afab47753ba604905ca83455bd39e75cb399dacdade26cae7a340d7e907742400bb96febde99ad61aa03c5aa49690112ea77a6ce1f1d0db6767b3d4dfb470bd7e609b05abdfce5c7909be2b5dd9fbf92a9838e25735f2fd10c95b8b695dbe5f72b3aa4300db4b802955fb8ab0cb0feca860f75bbdef72cdc07de98a472c52fdbf5c89598d75352bcd8a30013af316657201d8998857eddce3c79ea2b22b5d9e04e9fc27e5ce7403e173da7f071b9e73a904e58d9ae9b32f1f14c7d7cc8273704a6325def3df7bc89b7eec29eabbf5b753fc3d7d36f78974277f9199eec8cc744766a63b3233dd9199e98ecc4c775c9ecf7f306e83d5acd1d8d50000000049454e44ae426082";

var result = '';

for( i = 0 ; i < FileHex.length ; i += 2 ) {

    hex = FileHex[i] + FileHex[i + 1];

    ascii = parseInt(hex, 16);

    if(ascii >= 32 && ascii <= 126) {

        if([34,39,92].includes(ascii))  result += '\\';
    
        result += String.fromCharCode(ascii);

    }
    else {
    
        result+= '\\x' + hex;

    }
}

When I log the result variable in browser console it escape the hex prefix '\x' to be '\\x' which causes the file to be invalid, see the following screenshot where the first part is the result variable output, while the second part the valid output that I should get:

Is the string a valid representation of the binary data? For example, the data could contain a byte that translates to a backslash and then, when pasting that into code, would be interpreted as an escape sequence, corrupting the data. One specific instance is `\�` which would be seen by javascript as just `�` since the backslash is treated as a character with special behavior. Generally, you can't just paste binary into text and expect it to work. You need to do some preprocessing before the text is inserted (and before the code is run). — Ouroborus, Nov 11 '22 at 12:24
Notice that [strings passed to the `Blob` constructor](https://developer.mozilla.org/en-US/docs/Web/API/Blob/Blob) "*are encoded as UTF-8, unlike the usual JavaScript UTF-16 strings.*" Is your "binary string" actually well-formed with respect to that? — Bergi, Nov 11 '22 at 12:26
If you really do mean for us to ignore the topic of improper escapes, fix the binary string in your example. — Ouroborus, Nov 11 '22 at 12:29
Other than the problem with `�` (U+FFFD, usually obtained from trying invalid utf8 sequences), another problem is that a string declared with single quotes `'` cannot contain a newline character or an unescaped `'`. — qrsngky, Nov 11 '22 at 16:42

qrsngky · Accepted Answer · 2022-11-19T16:45:09.913

The binary data in your example is not valid for a PNG file, so I created my own.

let FileBinaryString = "\x89PNG\x0d\x0a\x1a\x0a\x00\x00\x00\x0dIHDR\x00\x00\x00\x9f\x00\x00\x00!\x08\x02\x00\x00\x00H\xb9\xee\xb2\x00\x00\x00\x01sRGB\x00\xae\xce\x1c\xe9\x00\x00\x00\x04gAMA\x00\x00\xb1\x8f\x0b\xfca\x05\x00\x00\x00\x09pHYs\x00\x00\x0e\xc3\x00\x00\x0e\xc3\x01\xc7o\xa8d\x00\x00\x02\xf9IDAThC\xed\x98kv\xab0\x0c\x84\xb3.\x16\xc4zX\x0d\x9ba1\xb9~HF\xb6GJt\x1bJ\xea\xe3\xefO\x8bmF#\x0d\xe9)y<\'\xe32\xd3\x1d\x99\x99\xee\xc8\xcctGf\xa6;23\xdd\x91\xf9\x92t\x8fmy\x04\x96\xed\xa0\x85\x08\\\x1c\x18g\xbf\xfb\x1aO?\xd6\x9d\xae\x013\xdd\xef\xe1\xbet_U\xa6}\xb3\x96\xc1o\xa5\xcb65\xcdW\xfbWx*8\xb5g\xba\x1d\xb6O\xdeU\x8b\xfe\xb0M\x1bg\xbf3\xdd\x1e\xd3(\x0d,\x82\xab\xbe1\xd1\xff\xc7\xd9\xefL\x17\xc0\x09\x02\xa7\"\\\xdc\x09\x1d\xf8\xb0#f\xa6\xfb\x01\xd4\x88r\xbde]\xb5\xb1]\x1b\xae\xb7\xdf/I\xf7\xd8\xb7u\xc9\xdb\x91eY\xb7\xbd\x91\x81\xf2V\xcdF3\x88.\xeb~\xb8\xc6\xd2\xeaR\xb9 \xa3\xf4\xf2s?\xa5D\xbca\xe5\xe3,h\xe8\'\xf9t8\x90\xa4\xbf\"]^\x8f\x84\x8e\xc5E\xa5\x04\xe5\xb5\x9a\x87\xe8\xb4!\xf4M\x87,p\xbcy5M\x1e\x16\xd6\xec8\xfc\x90\xc4\xba\xe5\x9f\x0c\x1d\xd2\x0a\x9c\xcf\x81\x84\x87iu|m\xba4\xc6\xfa\xb6\xf0\xd4\xe6Uq\x16\xca\xe3\x9a|w\x9c\\\xd9\x08\xcfv\xaeowK@\xb3Y7\x17C\x95\xb1\x1b\x9f\x1f.\x1c)\xc7\x0f\xfe\x88\xdb\xfd\x9e\xf2\xd5\x07\xd9\xec\xd7\x99\xeeK@/\xdd<\x8a\xe3\xf30<\x8a\x16\xd9\x07\x105\xb6Z\xf8\xa8p+\xc3-\x06\xa5T\xe79\xe1\xf3\xc3K8\x13\xda\xad\xa4T\xfd\"\x85\x94\x98\x0b\xd3Ef\x99vT\xf0,Xd\x1b\xb8%\x90\x89B\xa7C\x0b\xe5\xba\xcb\xd2\x0e\xf7M?\xa0#\x01\xd8\xb5\xf4\xb1\xa3\x8a\x0b\xff2S\xf5pS\x0f\xed\x94\xd3P\x1e,v\xf3\xaa\xb0f\xd1\xd0h\xd3e\xef\x9eW\xfa\x03\x09\xa7\x9f\xa6j\x03\xd8\xb5\xf4\x1b\x87\x88\xdfHWe\xd9\xcc\xb6\xf5n\x95\x8e\x80\x09\x95zn\xf9\xaa\xba\xaf*E\xca\x9d\xb0\xd3\x0fl\xb3\xa0\xf7\x0bo\xb0k\'\xaeOW\xbdE\x02\xe5\x9d\xddB\x13:R*\xff\xaeW\x87\xf6\"N?\xaaN\x02\xecZ\xfa\xb4wS\xba`I\x05\xca\x83E[\xd3\x9eu\xcb9\x9d\xac\xda\xde&\xca\xe7\xa3@\xd7\xe9\x07t$\x00\xbb\x96\xfe\xbd\xe9\x9a\xd6\x1a\xa0<Z\xa4\x96\x90\x11.\xa7zl\xe1\xf1\xd0\xdbgg\xb3\xd4\xdf\xb4p\xbd~`\x9b\x05\xab\xdf\xce\\y\x09\xbe+]\xd9\xfb\xf9*\x988\xe2W5\xf2\xfd\x10\xc9[\x8bi]\xbe_r\xb3\xaaC\x00\xdbK\x80)U\xfb\x8a\xb0\xcb\x0f\xec\xa8`\xf7[\xbd\xefr\xcd\xc0}\xe9\x8aG,R\xfd\xbf\\\x89Y\x8du5+\xcd\x8a0\x01:\xf3\x16er\x01\xd8\x99\x88W\xed\xdc\xe3\xc7\x9e\xa2\xb2+]\x9e\x04\xe9\xfc\'\xe5\xcet\x03\xe1s\xda\x7f\x07\x1b\x9es\xa9\x04\xe5\x8d\x9a\xe9\xb3/\x1f\x14\xc7\xd7\xcc\x82spJc%\xde\xf3\xdf{\xc8\x9b~\xec)\xea\xbb\xf5\xb7S\xfc=}6\xf7\x89t\'\x7f\x91\x99\xee\xc8\xcctGf\xa6;23\xdd\x91\x99\xe9\x8e\xccLw\\\x9e\xcf\x7f0n\x83\xd5\xac\xd1\xd8\xd5\x00\x00\x00\x00IEND\xaeB`\x82";

//needed to call charCodeAt so e.g. '3' in FileBinaryString[150] is converted to 51 (ASCII value) instead of 3
let my_uint8_array = Uint8Array.from(FileBinaryString, c => c.charCodeAt(0)); 
//note that this uses Uint8Array.from; Array.from won't give a correct file in the end.

let blob = new Blob([my_uint8_array], { type: 'image/png' });
console.log(blob);
//the [] above is necessary, otherwise it produces a wrong Blob with a wrong size (2182) 
// for the above string, the correct Blob has a size of 868

let myUrl = window.URL.createObjectURL(blob)

//the part below is just so that you can copy the code to the console and see that it results in a valid PNG file
const link = document.createElement("a");
link.href = myUrl;
link.setAttribute("download", "hello_world.png");
document.body.appendChild(link);
link.click();
link.remove();

And here is a way to get the string from an actual hello_world.png file, using node JS:

let fs = require('fs');

let buffer = fs.readFileSync("./hello_world.png");
let string = Array.from(buffer, num => {
    //most of printable characters in ASCII don't need to be escaped
    //0x7f should use hex escape, otherwise your command line output may miss that character.
    if (num >= 0x20 && num < 0x7f) {
        let chr = String.fromCharCode(num);
        if (chr === '\\' || chr === '"' || chr === "'") return '\\' + chr;
        else return chr;
    }
    let hexString = num.toString(16);
    if (hexString.length < 2) {
        hexString = "0" + hexString;
    }
    return '\\x' + hexString
}).join('');

console.log('"' + string + '"')

Edit: for processing data string saved as hex string

var FileHex = "89504e470d0a1a0a0000000d494844520000009f00000021080200000048b9eeb2000000017352474200aece1ce90000000467414d410000b18f0bfc6105000000097048597300000ec300000ec301c76fa864000002f9494441546843ed986b76ab300c84b32e16c47a580d9b6131b97e4846b6474a741b4aeae3ef4f8b6d46230de929793c27e332d31d9999eec8cc744766a63b3233dd91f992748f6d790496eda085085c1c1867bffb1a4f3fd69dae0133ddefe1be745f55a67db396c16fa5cb3635cd57fb57782a38b567ba1db64fde558bfeb04d1b67bf33dd1ed3280d2c82abbe31d1ffc7d9ef4c17c00902a7225cdc091df8b02366a6fb01d48872bd655db5b15d1baeb7df2f49f7d8b775c9db916559b7bd9181f256cd4633882eeb7eb8c6d2ea52b920a3f4f2733fa544bc61e5e32c68e827f9743890a4bf225d5e8f848ec545a504e5b59a87e8b421f44d872c70bc79354d1e16d6ec38fc90c4bae59f0c1dd20a9ccf81848769757c6dba34c6fab6f0d4e6557116cae39a7c779c5cd908cf76ae6f774b40b35937174395b11b9f1f2e1c29c70ffe88dbfd9ef2d507d9ecd799ee4b402fdd3c8ae3f3303c8a16d9071035b65af8a8702bc32d06a554e739e1f3c34b3813daada454fd228594980bd34566997654f02c58641bb825908942a7430be5bacbd20ef74d3fa02301d8b5f4b1a38a0bff3253f570530fed94d3501e2c76f3aab066d1d068d365ef9e57fa0309a79fa66a03d8b5f41b8788df485765d9ccb6f56e958e8009957a6ef9aabaaf2a45ca9db0d30f6cb3a0f70b6fb06b27ae4f57bd4502e59ddd42133a522affae5787f6224e3faa4e02ec5afab47753ba604905ca83455bd39e75cb399dacdade26cae7a340d7e907742400bb96febde99ad61aa03c5aa49690112ea77a6ce1f1d0db6767b3d4dfb470bd7e609b05abdfce5c7909be2b5dd9fbf92a9838e25735f2fd10c95b8b695dbe5f72b3aa4300db4b802955fb8ab0cb0feca860f75bbdef72cdc07de98a472c52fdbf5c89598d75352bcd8a30013af316657201d8998857eddce3c79ea2b22b5d9e04e9fc27e5ce7403e173da7f071b9e73a904e58d9ae9b32f1f14c7d7cc8273704a6325def3df7bc89b7eec29eabbf5b753fc3d7d36f78974277f9199eec8cc744766a63b3233dd9199e98ecc4c775c9ecf7f306e83d5acd1d8d50000000049454e44ae426082";

var binaryString = '';

for(let i = 0 ; i < FileHex.length ; i += 2 ) {

    let hex = FileHex[i] + FileHex[i + 1];

    let charCode = parseInt(hex, 16);
    binaryString += String.fromCharCode(charCode);

}

let my_uint8_array = Uint8Array.from(binaryString, c => c.charCodeAt(0)); 

let blob = new Blob([my_uint8_array], { type: 'image/png' });
console.log(blob);
let myUrl = window.URL.createObjectURL(blob)

const link = document.createElement("a");
link.href = myUrl;
link.setAttribute("download", "hello_world.png");
document.body.appendChild(link);
link.click();
link.remove();

(n.b. just copy the above to a browser console to test the download.)

EDIT 2: More experimentation with node JS led to this:

let fs = require('fs');

let buffer = fs.readFileSync("./hello_world.png");

let specialMap = new Map([
    //[0x08, '\\b'],
    //[0x0c, '\\f'],
    [0x0a, '\\n'],
    [0x0d, '\\r'],
   // [0x09, '\\t'],
    //[0x11, '\\v'],
   // [0x00, '\\x00'],
])

let string = Array.from(buffer, num => {
    if (specialMap.has(num)) {
        return specialMap.get(num);
    }
    let chr = String.fromCharCode(num);
    if (chr === '\\' || chr === '"') return '\\' + chr;
    else return chr;

}).join('');

const FileBinaryString = '"' + string + '"'
console.log(FileBinaryString.length) //1013, original: 868


let after = `
let my_uint8_array = Uint8Array.from(FileBinaryString, c => c.charCodeAt(0)); 

let blob = new Blob([my_uint8_array], { type: 'image/png' });

let myUrl = window.URL.createObjectURL(blob)

const link = document.createElement("a");
link.href = myUrl;
link.setAttribute("download", "hello_world.png");
document.body.appendChild(link);
link.click();
link.remove();`;

fs.writeFileSync("./hello_world2.js", "let FileBinaryString = " + FileBinaryString + ';' + after)

and HTML file:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
</head>
<body>
    <script defer src="./hello_world2.js"></script>
</body>
</html>

After running the nodeJS script of Edit 2, it downloads the hello_world png just fine in the Chrome. But I can't copy it here since there are characters like "\x00".

thanks for your answer and I believe it will help, I updated my question and I really appreciate if you look at it. — Akkad, Nov 18 '22 at 13:04
@user2155873 In this case you don't need to manually escape it. You could simply use `result += String.fromCharCode(ascii);` without the if-else, see also my edited answer. — qrsngky, Nov 18 '22 at 16:09
thanks a lot, it worked and I marked your reply as answer, and by the way I spent a lot of time online and I end up with your solution. one last question here, I avoided to use Base64 to shrink the string size, and now I converted the binary to hex (which also increase the string length) because I understood from your code that I have to use hex, so isn't there a way to set a string inside JS with binary (freak chars) itself? — Akkad, Nov 19 '22 at 08:16
The thing is you need to have a valid JS string. There are plenty of characters that cannot be copied directly from console output, for example, `\x00` is problematic as it normally "terminates" the string, and then there are variations for newline characters, like `\n`, `\r` and `\r\n` which the console may 'auto-correct' for you. One missing newline or one more, it's corrupted. I can improve a little bit noting that characters from `0x80` to `0xff` don't need to be escaped, and that `\\r` `\\r` `\\v` are shorter than their hex escapes, but I'm not sure how much more it can be improved. — qrsngky, Nov 19 '22 at 15:43
Also, assuming your JS file is saved in UTF-8, taking `«` (\xab) as an example, the `«` character by iself takes 2 bytes (even if it appears to be one only one character in your text editor/IDE). So it's not better than saving 'ab' in terms of disk space. You can test it here: https://mothereff.in/byte-counter ; Another character as an example, it seems to be ok to copy the output of `console.log("\x13")` from a browser console, but not ok if you copy from PowerShell or cmd, where it gives you character 8252(\u203c) instead (when pasted, it becomes `‼` rather than a non-printable character). — qrsngky, Nov 19 '22 at 16:00
\x00 problem: If you try to copy the output of `console.log("A\x00something_here")` from Chrome, when you paste you get just "A". \x01 problem: If you try `console.log("A\x01B")` from a browser console and paste it into VSCode it works fine, but if you try nodeJS console (through cmd or PowerShell) it's replaced by 9786 (`263a`, "☺"). I'm not sure how many other characters are problematic, but I think most of those from cmd and PowerShell are. Another thing that came to mind was to open the PNG file in a text editor, but then you'll get a problem when it mixes up `\r\n` and `\n`. — qrsngky, Nov 19 '22 at 16:16
I managed to make it much more efficient (888 bytes to save a 868 bytes PNG file). Looks like if you don't rely on console.log, but use `fs.writeFileSync` instead, you can remove most of the limitations from non-printable characters (and only `\`, `\r`, `\n` and `"` need to have special treatment). I'll edit my answer. But I can't copy the output here, as the browser will cut off things like \x00. — qrsngky, Nov 19 '22 at 16:39
So it seems the direction which am taking to avoid Base64 will cause me a problem to handle special chars specially in my case am using Linux command to open files and returns them as string to be used in JS, where I am avoiding Base64 as it became a hassle when it comes to large size images (Base64 is 3 times larger than original binary size) thats why I was looking to use binary instead, so is there any way to display a large image but to be loaded faster? with whatever is the format binary/hex/Base64? but in my case I have no option but to use JS string as I cant use GET/Ajax. — Akkad, Nov 19 '22 at 16:44
For the "mostly binary but stored in JS" approach there is also a question of whether your text editor/IDE can handle the resulting "JS with special characters". Like, if you add more JS code after it, can the editor re-save the file without corrupting the data? What if you have a colleague who uses another editor? However, even very basic text editors can handle "ab" and "\xab". For "ab" but it is 2 bytes for each byte of your original image, and for "\xab" it's ~4 bytes each (most are not normal letters). I'm not sure which one will run faster considering that the for loops are different. — qrsngky, Nov 19 '22 at 17:03
Usually the time for the script is much less (can be ignored) than the time for file transfer, so it may seem that a smaller file size may be important. However, people may use gzip compression. The "mostly normal text" file may be bigger, but it is more compressible than the mostly binary version, so what is actually transferred through the network may not differ that much (I don't know how to benchmark this, though). Probably only slightly bigger? You may try curl with `Accept-encoding: gzip` without decompressing it — qrsngky, Nov 19 '22 at 17:22

JavaScript: Convert binary string to Blob

1 Answers1