Spring Encoding with CharacterEncodingFilter in web.xml

Question

Encoding on stackoverflow.com

{|}~€‚ƒ„…†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎ ±ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö±øùúûüýþŸ▲►▼◄■□▣▤▥▦▧▨▩▪▫◊○●☺☻☼€¢£¥¤♀♂♂♠♤♣♧♥""♡♦★☆⌂№☎☏♨☜☞♩♪♫♬♭†‡←↑→↓↔↕↖↗↘↙×÷+-Ω√¼½¾⅓⅔⅛⅜⅜⅝%‰¹²³

Endcoding on my site:

{|}~???????????????????????????¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎ ±ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö±øùúûüýþ?????¦???????????????¤?¢£¥¤????????""????¦???????????????????????×÷+-Ov¼½¾??????%?¹²³

Solution:

<#ftl attributes={"content_type":"text/html"} encoding="UTF-8"/>

and put this into my HttpsCoookieFilter:

        request.setCharacterEncoding("UTF-8");
        response.setCharacterEncoding("UTF-8");
        response.setContentType("text/html; charset=UTF-8");

Aparently somehow I (ab)use a HttpServlet instead of a Freemarker to generate HTML content using out.write() so I added the above.

Here is the servlet source now. Any tips on how to change it are more than welcome:

public class HttpsCookieFilter implements Filter {
private static Logger log = Logger.getLogger(HttpsCookieFilter.class);

@Override
public void destroy() {
}

@Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {

    final HttpServletRequest req = (HttpServletRequest) request;
    final HttpServletResponse res = (HttpServletResponse) response;

        req.setCharacterEncoding("UTF-8");
        res.setCharacterEncoding("UTF-8");
        res.setContentType("text/html; charset=UTF-8");

    final HttpSession session = req.getSession(false);

    if (session != null) {
        setCookie(req, res);
    }
    try {
        chain.doFilter(req, res);
    } catch (IllegalStateException e) {
        log.warn("HttpsCookieFilter redirect problem! ", e);
    }
}

@Override
public void init(FilterConfig arg0) throws ServletException {
}

private void setCookie(HttpServletRequest request, HttpServletResponse response) {
    Cookie cookie = new Cookie("JSESSIONID", request.getSession(false).getId());
    cookie.setMaxAge(-1);
    cookie.setPath(getCookiePath(request));
    cookie.setSecure(false);
    response.addCookie(cookie);
}

private String getCookiePath(HttpServletRequest request) {
    String contextPath = request.getContextPath();
    return contextPath.length() > 0 ? contextPath : "/";
}
}

Now the UTF-8 is working everywhere ;) Thank you BalusC !!!

Answer 1

Question marks are typically used when the character-to-byte converter/writer is by itself aware of the charset the characters are actually encoded in and of the charset the characters have to be decoded to. If the decoding charset doesn't support the particular character in the original encoding, then it's converted to a question mark.

In the average webapplication with a database backend, there are two places where this can happen:

1. When the user submitted data is about to be inserted/updated in the DB.
2. When the HTTP response body is about to be written and sent to the client.

In both cases a TCP/IP network is been used which only understand bytes and both the server and the client are usually aware of the charset used on the both sides. In all other cases you would have seen Mojibake instead.

To cover the first case, you need to ensure that the DB and the table is been configured to use UTF-8. You usually specify that during CREATE. Here's an example in MySQL dialect.

CREATE DATABASE db_name CHARACTER SET utf8;
CREATE TABLE tbl_name (...) CHARACTER SET utf8;

With some JDBC drivers, such as the MySQL one, you also need to instruct the driver itself to use UTF-8.

jdbc:mysql://localhost:3306/db_name?useUnicode=yes&characterEncoding=UTF-8

To cover the second case, you need to ensure that the response writer is been instructed to use UTF-8 to decode the characters to bytes. When using JSPs as view, just adding the following to top of every JSP page (also the includes) ought to be sufficient (it not only sets the response encoding, but also implicitly sets the right response header).

<%@ page pageEncoding="UTF-8" %>

See also:

• Unicode - How to get the characters right?

---

As to the Spring character encoding filter which you're currently using, it only sets the request encoding so that you can ensure that the submitted data is interpreted as UTF-8. All it basically does is the following:

request.setCharacterEncoding("UTF-8");

and nothing more. Note that this only covers POST requests, for GET requests you would still need to configure the webserver to interpret URLs as UTF-8.

Answer 2

in your dispatcher servlet context xml, you have to add a propertie "<property name="contentType" value="text/html;charset=UTF-8" />" on your viewResolver bean. we are using freemarker for views.

it looks something like this:

<bean id="viewResolver" class="org.springframework.web.servlet.view.freemarker.FreeMarkerViewResolver">
       ...
       <property name="contentType" value="text/html;charset=UTF-8" />
       ...
</bean>