Umlauts broken when doing get request

Question

I'm trying to query a webservice which answers with plain text. The text often has german umlauts in it. In the received stream the umlauts are broken. Any ideas what am I doing wrong?

Regards, Torsten

Here is the sample code:

var request = require('request');        
var uri = <anUriWithUserId>;        
request(uri, {encoding: 'utf8','content-type': 'text/plain; charset=UTF-8'}, 
    function (error, response, body) 
    {
        console.log("encoding: " + response.headers['content-encoding']);
        console.log("type: " + response.headers['content-type']);
        console.log(body);
    });

And the response:

encoding: undefined
type: text/plain

error=0
---
asin=
name=Eistee
detailname=Pfanner Der Gr�ne Tee, Zitrone - Kaktusfeige, 2,0 l
vendor=Hermann Pfanner Getr�nke GmbH, Lauterach, �sterreich
maincat=Getr�nke, Alkohol

It is not the terminal, because `console.log('üäö');` works great — Torsten, Apr 16 '14 at 20:24
Possible answer: (http://stackoverflow.com/questions/16267274/how-to-get-utf-8-in-node-js) — PeterVC, Apr 16 '14 at 20:28
maybe it's just a typo but `content-type` is not supposed to be in quotes... — Dunken, Apr 16 '14 at 20:37

score 0 · Answer 1 · answered Apr 17 '14 at 10:35

When you set the encoding option in your request call, you advise the request module to decode the response body with this encoding. In this way you ignore the encoding used by the webservice, wich may or may not be utf-8. You need to find out wich encoding was used be the webservice and use that.

Depending on how complient the webservice you could also try to set the Accept-Charset: utf-8 header.

As your output shows, the webservice doesn't provide the used encoding in the Content-Type header, which is a bad habbit imho.

Sidenote: Content-Encoding isn't for charset, but for compression, gzip migh be a valid value for it.

Umlauts broken when doing get request

1 Answers1

Linked