6

I have faced a problem, unfortunately, I have not found a correct solution: I need to decode url-slice that is encoded with windows-1251 (cp1251).

I know there are theese methods - decodeURI() and decodeURIComponent(), but they work for UTF-8 only (as I have understood). A solution that I found uses deprecated methods escape() and unescape().

For example, there is sequence:

%EF%F0%EE%E3%F0%E0%EC%EC%E8%F0%EE%E2%E0%ED%E8%E5 (программирование)

The methods decodeURI() and decodeURIComponent() will cause an exception.

Will be grateful for the help.

Nickolay
  • 31,095
  • 13
  • 107
  • 185
Romanzhivo
  • 161
  • 1
  • 3
  • 11

2 Answers2

5

There's no built-in support for the percent-encoding scheme with legacy charsets in the browser, as far as I can see. You'll have to:

  1. find the %-escapes representing the win-1251 octets,
  2. decode the win-1251 octets to the corresponding characters (JS String)

Below is one way to do it. For the #1 I assume that only 3-character upper-case escapes need decoding, and the rest of the string is already ASCII, so I just use inputStr.replace(/%([0-9A-Z]{2})/g, replacerFunction) for this.

For the actual decoding you need a mapping from the win-1251 octets to JS characters. In the example below I build the mapping using TextDecoder.decode() API, just for fun (and in case someone finds this answer while trying to convert between different charsets in JS). (Note: it isn't universally supported as of this time -- only Gecko/Blink support it).

There's also https://github.com/mathiasbynens/windows-1251 , which I initially wanted to use for this answer, but it turned out to be easier to just build the decoding map by hand.

var decodeMap = {};
var win1251 = new TextDecoder("windows-1251");
for (var i = 0x00; i <= 0xFF; i++) {
  var hex = (i <= 0x0F ? "0" : "") +      // zero-padded
            i.toString(16).toUpperCase();
  decodeMap[hex] = win1251.decode(Uint8Array.from([i]));
}
// console.log(decodeMap);
// {"10":"\u0010", ... "40":"@","41":"A","42":"B", ... "C0":"А","C1":"Б", ...


// Decodes a windows-1251 encoded string, additionally
// encoded as an ASCII string where each non-ASCII character of the original
// windows-1251 string is encoded as %XY where XY (uppercase!) is a
// hexadecimal representation of that character's code in windows-1251.
function percentEncodedWin1251ToDOMString(str) {
  return str.replace(/%([0-9A-F]{2})/g,
    (match, hex) => decodeMap[hex]);
}

console.log(percentEncodedWin1251ToDOMString("%EF%F0%EE%E3%F0%E0%EC%EC%!%E8%F0%EE%E2%E0%ED%E8%FFa"))
Nickolay
  • 31,095
  • 13
  • 107
  • 185
  • Nickolay, thank you for the answer! This decision works, but it seems I have found else one decision. I don't know, can I publish a script, as it is from site http://www.codenet.ru That decision contains a map and have no experimental technologies like TextDecoder.decode() API – Romanzhivo Jan 03 '17 at 20:34
  • @Romanzhivo can you post a link to the solution you found? (Note that the 'experimental' part of my example can be easily replaced with the hardcoded map.) – Nickolay Jan 03 '17 at 20:40
  • Nickolay, I think I could, the solution from this site http://codenet.ru, I found it in a site's scripts and I added some changes and placed in my directory temporary http://code.romanzhivo.com/test/decode-windows-1251.js P.S. I cannot link to your name in comments :) – Romanzhivo Jan 03 '17 at 20:48
  • Thanks for sharing. I still feel my solution is clearer, but it was interesting to see the alternative approach of replacing the encoded win1251 escapes with their UTF8 equivalents. – Nickolay Jan 03 '17 at 21:09
  • 1
    In loop conditions you need to include the last `FF` also. `for (var i = 0x00; i <= 0xFF; i++) {`. So `FF: 'я'`. – Alexey Vol Feb 12 '23 at 10:02
2
  1. find strings: "%EF%F0%EE%E3%F0%E0%EC%EC", "%E8%F0%EE%E2%E0%ED%E8%E5"
  2. replace "%" with ",0x": ",0xEF,0xF0,0xEE,0xE3,0xF0,0xE0,0xEC,0xEC", ",0xE8,0xF0,0xEE,0xE2,0xE0,0xED,0xE8,0xE5"
  3. slice without first comma: "0xEF,0xF0,0xEE,0xE3,0xF0,0xE0,0xEC,0xEC", "0xE8,0xF0,0xEE,0xE2,0xE0,0xED,0xE8,0xE5"
  4. split into an array of strings: ["0xEF","0xF0","0xEE","0xE3","0xF0","0xE0","0xEC","0xEC"], ["0xE8","0xF0","0xEE","0xE2","0xE0","0xED","0xE8","0xE5"]
  5. create an array of bytes: [239,240,238,227,240,224,236,236], [232,240,238,226,224,237,232,229]
  6. decode bytes to strings win-1251: "программ", "ирование" and replace with them
var win1251 = new TextDecoder("windows-1251"),
s = "%EF%F0%EE%E3%F0%E0%EC%EC%!%E8%F0%EE%E2%E0%ED%E8%E5a"
s = s.replace(/(?:%[0-9A-F]{2})+/g,
s => win1251.decode(new Uint8Array(
s.replace(/%/g, ",0x").slice(1).split(",")
)))
alert(s)
Дмитрий Кондратьев
  • 21
  • 2