How to resolve URI encoding problem in spring-boot?

Question

I am using spring-boot to host a http request service.

@RequestMapping("/extract")
    @SuppressWarnings("unchecked")
    @ResponseBody
    public ExtractionResponse extract(@RequestParam(value = "extractionInput") String input) {

        // LOGGER.info("input: " + input);
        JSONObject inputObject = JSON.parseObject(input);


        InputInfo inputInfo = new InputInfo();

        //Object object = inputObject.get(InputInfo.INPUT_INFO);
        JSONObject object = (JSONObject) inputObject.get(InputInfo.INPUT_INFO);

        String inputText = object.getString(InputInfo.INPUT_TEXT);
        inputInfo.setInputText(inputText);

        return jnService.getExtraction(inputInfo);
    }

When there is a % sign, as follows, it got an errror:

 http://localhost:8090/extract?extractionInput={"inputInfo":{"inputText":"5.00%"}}

The error message is below:

2018-10-09 at 19:12:53.340 [http-nio-8090-exec-1] INFO  org.apache.juli.logging.DirectJDKLog [180] [log] - Character decoding failed. Parameter [extractionInput] with value [{"inputInfo":{"inputText":"5.0022:%225.00%%22}}] has been ignored. Note that the name and value quoted here may be corrupted due to the failed decoding. Use debug level logging to see the original, non-corrupted values.
 Note: further occurrences of Parameter errors will be logged at DEBUG level.
2018-10-09 at 19:12:53.343 [http-nio-8090-exec-1] WARN  org.springframework.web.servlet.handler.AbstractHandlerExceptionResolver [140] [resolveException] - Resolved [org.springframework.web.bind.MissingServletRequestParameterException: Required String parameter 'extractionInput' is not present]

How to configure the URI encoding to fix this issue in my spring-boot configurations?

EDIT: Possible Java client code to make the request:

public String process(String question) {

        QueryInfo queryInfo = getQueryInfo(question);

        ObjectMapper mapper = new ObjectMapper();
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        String jsonResult = null;
        try {
            String jsonStr = mapper.writeValueAsString(queryInfo);
            String urlStr = Parameters.getQeWebserviceUrl() + URLEncoder.encode(jsonStr, "UTF-8");
            URL url = new URL(urlStr);
            BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
            jsonResult = in.readLine();
            in.close();
        } catch (Exception jpe) {
            jpe.printStackTrace();
        } 
     return jsonResult
}

possible duplicate with this question https://stackoverflow.com/questions/35992342/how-to-send-special-characters-in-request-parameters-in-spring — Khirye Zhou, Oct 10 '18 at 02:26
use preprocessors [related question](https://stackoverflow.com/questions/34462155/how-to-preprocess-all-requests-in-spring-web-application) — Downhillski, Oct 10 '18 at 02:49

Downhillski · Accepted Answer · 2018-10-11T11:00:52.373

Without encoding from your client side - you could still achieve this if you follow any of the following strategies by encoding before the request is processed in the servlet:

use Spring preprocessor bean to preprocess the controller endpoint request
use Spring AspectJ to preprocess the controller endpoint request
use Spring servlet filter to preprocess the controller endpoint request

With any of the above cross-cutting strategies, you could encode the request URL and pass back to the endpoint.

For example below is one implmentation using Filter. You could possibly do some caching there if you need better performance.

@Component
public class SomeFilter implements Filter {
    private static final Logger LOGGER = LoggerFactory.getLogger(SomeFilter.class);

    @Override
    public void init(final FilterConfig filterConfig) throws ServletException {

    }

    @Override
    public void doFilter(final ServletRequest servletRequest, final ServletResponse servletResponse, final FilterChain filterChain) throws IOException, ServletException {
        HttpServletRequest request = (HttpServletRequest) servletRequest;
        HttpServletRequest modifiedRequest = new SomeHttpServletRequest(request);
        filterChain.doFilter(modifiedRequest, servletResponse);
    }

    @Override
    public void destroy() {

    }

    class SomeHttpServletRequest extends HttpServletRequestWrapper {
        HttpServletRequest request;

        SomeHttpServletRequest(final HttpServletRequest request) {
            super(request);
            this.request = request;
        }

        @Override
        public String getQueryString() {
            String queryString = request.getQueryString();
            LOGGER.info("Original query string: " + queryString);

            try {
                // You need to escape all your non encoded special characters here
                String specialChar = URLEncoder.encode("%", "UTF-8");
                queryString = queryString.replaceAll("\\%\\%", specialChar + "%");

                String decoded = URLDecoder.decode(queryString, "UTF-8");
                LOGGER.info("Modified query string: "  + decoded);
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
            }

            return queryString;
        }

        @Override
        public String getParameter(final String name) {
            String[] params = getParameterMap().get(name);
            return params.length > 0 ? params[0] : null;
        }

        @Override
        public Map<String, String[]> getParameterMap() {
            String queryString = getQueryString();
            return getParamsFromQueryString(queryString);
        }

        @Override
        public Enumeration<String> getParameterNames() {
            return Collections.enumeration(getParameterMap().keySet());
        }

        @Override
        public String[] getParameterValues(final String name) {
            return getParameterMap().get(name);
        }

        private Map<String, String[]> getParamsFromQueryString(final String queryString) {
            String decoded = "";
            try {
                decoded = URLDecoder.decode(queryString, "UTF-8");
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
            }
            String[] params = decoded.split("&");
            Map<String, List<String>> collect = Stream.of(params)
                .map(x -> x.split("="))
                .collect(Collectors.groupingBy(
                    x -> x[0],
                    Collectors.mapping(
                        x -> x.length > 1 ? x[1] : null,
                        Collectors.toList())));

            Map<String, String[]> result = collect.entrySet().stream()
                .collect(Collectors.toMap(
                    x -> x.getKey(),
                    x -> x.getValue()
                        .stream()
                        .toArray(String[]::new)));

            return result;
        }
    }
}

could you give one example to achieve this? Not very farmiliar with spring. — user697911, Oct 10 '18 at 02:50
Given my case, how to write such an interceptor to fix this? I thought this should be a very very common need. — user697911, Oct 10 '18 at 02:53
can you please take a look at https://stackoverflow.com/questions/52746082/whats-the-problem-of-my-request-parameter-interceptor-in-spring-boot-restful-se? I was following with an Interceptor. — user697911, Oct 10 '18 at 18:06

score 0 · Answer 2 · answered Oct 10 '18 at 02:28

0

You probably need to URLEncode the query parameter, e.g.

http://localhost:8090/extract?extractionInput=%7B%22inputInfo%22%3A%7B%22inputText%22%3A%225.00%25%22%7D%7D

The generally easier way to pass a parameter like this is to use an HTTP POST instead of a GET, and pass your JSON object in the body.

answered Oct 10 '18 at 02:28

GreyBeardedGeek

29,460
2
47
67

Is there a way to do a configuration in spring-boot configuration to fix this? I did some searched and tried a few configuration options, but they didn't resolve it. – user697911 Oct 10 '18 at 02:29
No, if you stay with a GET and the query parameter, you would need to create the full request url with something like: `"http://localhost:8090/extract?extractionInput=" + URLEncoder.encode("{\"inputInfo\":{\"inputText\":\"5.00%\"}}", "UTF-8");"` – GreyBeardedGeek Oct 10 '18 at 02:35
@GreyBearedGeek, can you please give a version of the POST request, based on the code I posted? I have never used POST request before. – user697911 Oct 10 '18 at 02:41
You didn't post any code that makes the request - I have no idea what language you are using to make the request, let alone what library, etc. If you post the code that makes the GET request, I can probably help. – GreyBeardedGeek Oct 10 '18 at 03:19
please see my updates. That's way to make the request in Java, and it could also be used in Python code. But the first thing I want it to work is to just get the right result in a browser, not in any code. Currently, it crashes the service. – user697911 Oct 10 '18 at 17:11
It would be very difficult and error prone to do this with the Java's URL class - it's just too low-level. Java 11 has a new HttpClient class, but assuming that you're using an earlier version of java (most people are at the moment), I'd use a third-party HTTP Client class, perhaps the Apache HttpClient. The important thing is to post the JSON in the body of the request instead of putting it into the URL as a query parameter. Here is an example of doing just that: https://stackoverflow.com/questions/12059278/how-to-post-json-request-using-apache-httpclient – GreyBeardedGeek Oct 11 '18 at 17:35

score 0 · Answer 3 · answered Oct 10 '18 at 04:40

0

This is not a best practice for a REST API. Try to normalize your URLs in object oriented way to capture path variables.

if your object likes:

 param1:{ 
   param2:{ 
     param3: ""
          }
        }

use url pattern to capture attribute as:

class/param1/param2/{param3}

otherwise you will get more problems when altering front-end technologies while keeping back-end REST API same.

answered Oct 10 '18 at 04:40

Nilanka Manoj

3,527
4
17
48

Didn't get it. Can you make it more specific with the example "http://localhost:8090/extract?extractionInput={"inputInfo":{"inputText":"5.00%"}}"? Assuming 'InputInfo' has only two fields "inputText" and "id". – user697911 Oct 10 '18 at 17:02
What do you mean by "use url pattern to capture attribute as:"? – user697911 Oct 10 '18 at 17:02

How to resolve URI encoding problem in spring-boot?

3 Answers3

Linked