Google app engine datastore string encoding problem

Question

Hello I'm using Google App Engine for a project I'm doing and I need to store some Strings. I'm using Java and JDOHelper.getPersistenceManagerFactory("transactions-optional")

While debugging it on my computer everything works fine and the Strings are saved correctly. But when i upload it to google app engine, all the Strings i save will have their unicode characters replaced by question marks (?). If I go to the DataViewer on the project page, I can see that the Strings are actually saved with question marks.

Like I said, when running it on my computer it works fine. Is there anyone who knows what I should do?

Are you talking about using "encoded" Strings or "unencoded" Strings — Travis Webb, Mar 27 '11 at 04:53
What character encoding have you specified in your pages? UTF-8 is working fine for me. — Eelke, Mar 27 '11 at 07:00
I don't know really. I'm pretty new to GAE and encoding-stuff. But I did a test, tried to write a string with the char "å" directly to the persistancemanager from my servlet, and that worked fine. So I guess it has something to do with the decoding of the string coming in via the POST. But as I said, works fine when debugging locally — Joel, Mar 27 '11 at 07:33
Thanks for the effort, but it was just an encoding nightmare, not sure what I've done different but it works now... — Joel, Mar 27 '11 at 15:21
@Joel - I'm having the same issue. Maybe you can investigate and tell us how you solved this. Thanks. — Luca Matteis, Apr 17 '11 at 17:52
@Luca: Do you have any details you can add that might help someone figure out a solution? @Joel: If you can provide any details (or an answer) that might be helpful to others who have the same issue. Thanks. — Bill the Lizard, Apr 22 '11 at 12:29
Hello,Can you check if both version of servlets are identical on the app servers ? Maybe one is 2.4 and the other is 2.5 and they changed stuff and I already faced a strange encoding problem. Maybe the encoding is correct but on one case it is in header and the other, no. Try to reproduce the problem in local having the exact same environment. — Alexis, May 16 '11 at 10:17

score 3 · Accepted Answer · edited May 23 '17 at 12:30

3

It sounds like you were not specifying the encoding for your HTTP POST content. Have a look at this question for details.

edited May 23 '17 at 12:30

Community

1
1

answered May 29 '11 at 13:57

jackrabbit

5,525
1
27
38

score 1 · Answer 2 · edited Jun 28 '12 at 18:07

Like Jackrabbit said, you should specify charset. I still had some troubles on Google App Engine. After setting charset to UTF-8 AND using Spring's CharacterEncodingFilter nothing has bothered me encoding wise.

See How To Get Character Encoding Correct, which includes information on this code to add to your web.xml file:

<filter>
    <filter-name>SetCharacterEncoding</filter-name>
    <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
    <init-param>
        <param-name>encoding</param-name>
        <param-value>UTF-8</param-value>
    </init-param>
    <init-param>
       <param-name>forceEncoding</param-name>
       <param-value>true</param-value>
    </init-param>
</filter>
 <filter-mapping>
    <filter-name>SetCharacterEncoding</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

Be sure to add this as the first step in your filter chain!

Also, the author of the blog suggests setting the charsets in your JSP pages to utf-8 as well:

<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>

Google app engine datastore string encoding problem

2 Answers2

Linked