3

Is in possible to read web page in non utf8 encoding? For example windows-1251. I tried to convert result using node-iconv:

var convertedBody = new Iconv('windows-1251','utf-8').convert(responseBody));

But I get exception:

Error: EILSEQ, Illegal character sequence.
    at IncomingMessage.<anonymous> (/root/nodejstest/test2.js:22:19)
    at IncomingMessage.emit (events.js:59:20)
    at HTTPParser.onMessageComplete (http.js:111:23)
    at Socket.ondata (http.js:1183:22)
    at Socket._onReadable (net.js:654:27)
    at IOWatcher.onReadable [as callback] (net.js:156:10)

Thanks!

generalhenry
  • 17,227
  • 4
  • 48
  • 63
chardex
  • 323
  • 1
  • 5
  • 17
  • 1
    Did you already check out [this thread](http://groups.google.com/group/nodejs/browse_thread/thread/b2603afa31aada9c) on the nodejs google group? Seems to target your issue... – schaermu Feb 28 '11 at 12:45

4 Answers4

7

Here is working solution to your problem. You have to use Buffer and convert your string to binary first.

request({ 
uri: website_url,
method: 'GET',
encoding: 'binary'
}, function (error, response, body) {
    body = new Buffer(body, 'binary');
    conv = new iconv.Iconv('windows-1251', 'utf8');
    body = conv.convert(body).toString();
     }
});
Alex Kolarski
  • 3,255
  • 1
  • 25
  • 35
4

Take a look at the iconv-lite library. Using it your code may look like this:

var iconv = require('iconv-lite');
request(
    { 
        uri: website_url,
        method: 'GET',
        encoding: 'binary'
    },
    function(err, resp, body){
        body = iconv.decode(body, 'win1251');
    }
);
dchertousov
  • 69
  • 5
3

Iconv doesn't has windows-1251.

You can verify the list of encodings from bnoordhuis/node-iconv.

BTW, from wikipedia:

Windows-1251 and KOI8-R (or its Ukrainian variant KOI8-U) are much more commonly used than ISO 8859-5.

ricardopereira
  • 11,118
  • 5
  • 63
  • 81
0
const request = require('request');
const iconv = require('iconv-lite');

request({
    url: 'http://meta.ua',
    encoding: 'binary',
}, (err,res,body) => {
    if (err) throw err;

    var decoded = iconv.decode(res.body, 'win1251');

    console.log(decoded);
});
asdflh
  • 1