Why do my German Umlaut characters not get displayed correctly when content is dynamically loaded?

Question

Possible Duplicate:
UTF8 problem with MySQL 5

I'm having trouble displaying "German Umlaute" characters on a HTML page.

The actual page is saved UTF-8 without BOM encoded, has it's UTF-8 metatag, it is requested via AJAX and I'm manually setting the response header to UTF-8, too. Still, all German characters are broken.

I also place another AJAX call for dynamic content, which is returned correctly (in Firebug), but when displayed in the browser, the Umlaute are broken again.

I can fix everything by setting iso-8859-1 in all Ajax response headers, but I thought UTF-8 can handle special characters and I wouldn't have to mix character encoding.

Question:
Shouldn't UTF-8 handle characters correctly? Or do I need another charset? Am I missing something obvious?

Thanks!

EDIT:
Here is a screenshot: enter image description here

Code next.

EDIT:
THis is what I'm getting back from the server: enter image description here

And the response header also shows the content being delivered to be UTF-8 enter image description here

So from my unknowing point of view, the server response seems to be correct in UTF-8 and foreign characters showing, is it so?

The actual page is requested by Jquery Mobile, so I can't say what's happening there. The dynamic content, I'm doing myself. This is the call from my application controller:

var form = "",
    service = "../services/views.cfc",
    method = "byPass",
    returnformat = "json",
    targetUrl = "",
    formdata = "form_submitted=getUsers&method="+method+"&returnformat="+returnformat,
    successHandler = function(objResponse, cbk) {
        cbk( objResponse );
    };

ajaxFormSubmit( form, service, formdata, targetUrl, successHandler, "yes", "", returnformat, cbk );

which triggers:

var ajaxFormSubmit = 
    function ( form, service, formdata, targetUrl, successHandler, dataHandler, errorHandler, returnformat, type ){
    var override = null;

    if ( type !== "" && type !== "post" ){
        override = type;
        type = "get";
    }

    $.ajax({
        async: false,
        type: type == "" ? "get" : type,
        url: service,
        data: formdata,
        dataType: returnformat,
        success: function( objResponse ){
            if (objResponse.SUCCESS == true || typeof objResponse === "string" ){
                dataHandler == "yes" ? successHandler( objResponse, override ) : successHandler( override );
            } else {},  
        error: function (jqXHR, XMLHttpRequest, textStatus, errorThrown) {}
        });
    }

On the server (I'm using Coldfusion8 and MySQL 5.0.88. I'm ending up in this view:

<cffunction name="getUsers_abc" access="public" returntype="any" output="false" hint="JSON vcard library">
    <cfargument name="local" type="struct" required="true" hint="Local Object" />
    <cfscript>
        var THAT = local;
        THAT.displayStart = 0;
        THAT.displayLength = 10;
        THAT.count = 0;
        THAT.loginid = Session.id;
    </cfscript>
    <cftry>
    <!--- Database call --->
    <cfquery datasource="#Session.datasource#" name="getUsers">
        SELECT 
            tn.iln, 
            tn.typ, 
            ...
        FROM table AS Tn
        WHERE tn.freigeschaltet != "5"
        AND tn.typ = "abc"
        LIMIT #THAT.displayStart#,#THAT.displayLength# 
    </cfquery>

    <!--- CREATE JSON --->
    <cfsavecontent variable="jsonRetailers">
        <cfoutput>{"data":[</cfoutput>
            <cfloop query="getUsers">
                <cfset THAT.count = THAT.count + 1>
                    <cfoutput>
                        <cfoutput>{</cfoutput>
                            <cfoutput>"type":"#getUsers.typ#",</cfoutput>
                            ...
                            <cfoutput>}]</cfoutput>
                        <cfoutput>}</cfoutput>
                    </cfoutput>
                    <cfif getUsers.recordcount LT THAT.displayStart + THAT.displayLength>
                        <cfif THAT.count is not getUsers.recordcount><cfoutput>, </cfoutput></cfif>
                    <cfelse>
                        <cfif THAT.count LT THAT.displayLength><cfoutput>, </cfoutput></cfif>
                    </cfif>
                </cfloop>
                <cfoutput>]</cfoutput>
                <cfoutput>,"SUCCESS":true,"Count":#getUsers.recordcount#}</cfoutput>
            </cfsavecontent>
            <cfset variables.alredayBinary = "false">

            <!--- GZIP if possible --->
            <cfif cgi.HTTP_ACCEPT_ENCODING contains "gzip">
                <cfinvoke method="gzip" stringToZip="#jsonRetailers#" returnvariable="passBackObject"></cfinvoke>     
                <cfheader name="Content-Encoding" value="gzip">
                <cfset variables.alredayBinary = "true">
            </cfif>
            <!--- setting UTF-8 --->
            <cfheader name="Content-Type" value="text/json; charset=UTF-8">
            <cfheader name="Content-Length" value="#len(passBackObject)#" >
            <cfif variables.alredayBinary EQ "false">
                <!--- send to browser --->
                <cfcontent reset="no" variable="#CharsetDecode(passBackObject, "UTF-8")#" />
            <cfelse>
                <cfcontent reset="no" variable="#passBackObject#" />    
            </cfif>
            <cfreturn  />
</cffunction>

So, no PHP unfortunately. THe code is not telling me where the problem is, but maybe you see something, I don't .

Thanks!

broken = they are showing as "?" question marks. Code coming up. — frequent, Dec 15 '12 at 11:56
hey man are the German Umlaut characters in unicode hex code? — Muhammad Talha Akbar, Dec 15 '12 at 11:57
@AspiringAqib: if I knew how to output the wrong character here. 1 sec — frequent, Dec 15 '12 at 12:00

score 4 · Accepted Answer · edited May 23 '17 at 11:58

4

broken = they are showing as "?" question marks

This means most likely that you are fetching the characters as latin1 in your remote script, and displaying them in a UTF-8 context. (The default encoding for Ajax requests is UTF-8.)

Check out UTF-8 all the way through and make sure you are using UTF-8 everywhere. If the Ajax script fetches data from a database, make sure you explicitly set the encoding.

edited May 23 '17 at 11:58

Community

1
1

answered Dec 15 '12 at 11:58

Pekka

442,112
142
972
1,088

screenshot above. Is that what you mean? – frequent Dec 15 '12 at 12:02
@frequent yup. It's how latin1 (single-byte) umlaut characters show up in UTF-8. The problem is in the source script – Pekka Dec 15 '12 at 12:03
Ok. Checking the database language tables `MySQL`. Tables are all set to `COLLATE='latin1_swedish_ci'`, so that should be ok, shouldn't it? – frequent Dec 15 '12 at 12:05
@frequent it's how you *output* the data. You can use latin1 no problem, but you have to make sure your script emits UTF-8 because that's what Ajax expects by default. See the "data access" section in the answer I linked to. It works when you set headers to ISO-8859-1 because then the browser does the conversion for you – Pekka Dec 15 '12 at 12:06
two more screenshots. I think the server response is ok. So it must be something in either the browser or my handling of dynamic content...? – frequent Dec 15 '12 at 12:15
@frequent hmm, strange. The browser may be automatically switching to latin1 though. Can you show the PHP DB code? – Pekka Dec 15 '12 at 12:20
no PHP, but the code is above. – frequent Dec 15 '12 at 12:36
You should have closed the question and give your link as duplicate. – markus Dec 15 '12 at 12:50

Why do my German Umlaut characters not get displayed correctly when content is dynamically loaded?

1 Answers1