-1

I currently have a POST call with HTTPClient framework using JAVA, the method is not ours and i have found an interesting problem with the special characters.

  1. The call is sending a JSON Object, not using URL parameters.
  2. The call works well in my code and in postman but, if I put some special characters in the mix, the HTTPClient fails (but in postman keep working fine).

I am almost completely sure that my problem is that I am missing some header about my encoding but I don´t know which one i have to include in my call and i found very few info about this.

Example of the calls. (I will delete a few things for Data Security).

This are the Headers for the POSTMAN Call:

POSTMAN HEADERS:

And this is are the Ones for my HTTPClient call:

headers={Authorization=Bearer XXXXXXXXXXXXXXXX, Accept-Encoding=gzip, deflate, br, Content-Type=application/json}

As you can see, i am sure the problem is here, but i don´t know which parameter i have to inform and more importantly, why.

PD: I have checked before but didn´t found any similar questions around here.

Thanks for the help

Edit for clearance:

When i say "put some special characters in the mix" One of the multiple variables that i send iinside my JSON Object is a String that can contain special characters, to be more specific, we are sending first names and last names, and the actual problem is about the character 'Ñ'.

Thanks for the question Scary Wombat.

Edit2:

The api call in my LOG:

2022-05-11 10:20:45,118 ERROR [es.fiatc.http_client.GenericOperation.execute]- INFO: javax.net.ssl.trustStore = null 2022-05-11 10:20:45,495 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Executing method:post with uri: https://XXXXXXXXXXXXXXXXXXXXXXXX 2022-05-11 10:20:45,498 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Params: null 2022-05-11 10:20:45,501 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Headers: {Authorization=Bearer XXXXXXXXXXXXXXXXX, Content-Type=application/json} 2022-05-11 10:20:45,503 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Timeout = 30000 2022-05-11 10:20:45,506 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- User: null 2022-05-11 10:20:45,508 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Pwd: null 2022-05-11 10:20:45,511 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Charset: UTF-8 2022-05-11 10:20:45,513 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- TrustCertificate: null 2022-05-11 10:20:45,516 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- TrustCertificatePwd: null 2022-05-11 10:20:45,519 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- FiatcClientCertificate: false 2022-05-11 10:20:45,521 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Entity (class): java.lang.String 2022-05-11 10:20:45,524 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- EntityCharset: null 2022-05-11 10:20:45,526 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- EntityMimeType: application/json 2022-05-11 10:20:45,529 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Preemptive_authentication: false 2022-05-11 10:20:52,141 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- Executing request POST XXXXXXXXXXXXXXXXXXXXXX HTTP/1.1 2022-05-11 10:20:53,755 INFO [es.fiatc.http_client.dao.HTTPDao.execute]- RESULT = {headers={content-type=application/json, transfer-encoding=chunked, vary=Accept-Encoding, expires=Thu, 19 Nov 1981 08:52:00 GMT, cache-control=no-cache, pragma=no-cache, set-cookie=lang=es; expires=Fri, 10-Jun-2022 08:21:01 GMT; Max-Age=2592000; path=/; secure, date=Wed, 11 May 2022 08:21:01 GMT, strict-transport-security=max-age=31536000; includeSubDomains; preload, x-xss-protection=1;mode=block, x-content-type-options=nosniff, connection=close}, status_code=200, reason_phrase=OK, content=[B@166a7559} 2022-05-11 10:20:53,756 ERROR [es.fiatc.sld_guia_medica.dao.APIDao.callAPIPost]- {headers={content-type=application/json, transfer-encoding=chunked, vary=Accept-Encoding, expires=Thu, 19 Nov 1981 08:52:00 GMT, cache-control=no-cache, pragma=no-cache, set-cookie=lang=es; expires=Fri, 10-Jun-2022 08:21:01 GMT; Max-Age=2592000; path=/; secure, date=Wed, 11 May 2022 08:21:01 GMT, strict-transport-security=max-age=31536000; includeSubDomains; preload, x-xss-protection=1;mode=block, x-content-type-options=nosniff, connection=close}, status_code=200, reason_phrase=OK, content=[B@166a7559} 2022-05-11 10:20:54,935 INFO [es.fiatc.sld_guia_medica.tuotempo.actions.TuotempoAseguradoAction.execute]-

{"result":"ERROR","message":"A valid Userid is needed","sessionid":null,"execution_time":506,"net_execution_time":506}

An example of the JSONObject that i send:

{"email":"","userlid":"008368790204","idnumber":"XXXXXXX","idtype":"1","mobile_number":"666666666","privacy":"1","fname":"MARIA DEL CARMEN","lname":"OSUNA","gender":"F","date_of_birth":"1969-07-02","lname2":"NUÑEZ","language":"es"}

If the special character in lname2 disappear the api call works fine.

Edit 3:

while i can not post all of my java for the call these are the encoding relevant parts:

public static Map execute(String method, String url, Map<String, String> params, Map<String, String> headers, 
            String user, String pwd, String trustCertificate, String trustCertificateClave, String charset, 
            Integer timeout, Object entityRequest, String entityCharset, String entityMimeType, 
            boolean preemtiveAuth, boolean fiatcCertificate) throws Exception   {
        // Default values
        if (charset == null) charset = "UTF-8";
        if (timeout == null) timeout = 30000;

        // URI
        URI uri = new URI(url);
        

    // Creamos get o post
    if (StringUtils.equals(StringUtils.trim(method), "post"))
    {
        // Creamos post
        HttpPost httpPost = new HttpPost(url);

        // Insertamos params en post
        if (formparams != null)
        {
            UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formparams, charset);
            httpPost.setEntity(entity);                
        }
        
        // Insertamos entity en post (también multiparts,...)
        insertEntity(httpPost, null, entityRequest, entityMimeType, entityCharset);

        // Ponemos en uriRequest
        httpPost.setConfig(requestConfig);            
        uriRequest = httpPost;
    }

  private static void insertEntity(HttpPost httpPost, HttpPut httpPut, Object entityRequest, String entityMimeType, String entityCharset) throws UnsupportedEncodingException
    {
        // Miramos si existe entidad
        if (entityRequest == null) return;
        
        // Creamos contentType
        ContentType contentType = null;
        if (entityMimeType != null)
        {
            if (entityCharset == null)  contentType = ContentType.create(entityMimeType);
            else                        contentType = ContentType.create(entityMimeType, entityCharset);
        }

The postman call: enter image description here

Again, sorry for any format error in the way i make my question

Grismak
  • 192
  • 16
  • 1
    What does *if I put some special characters in the mix* mean? – Scary Wombat May 12 '22 at 06:16
  • 1
    So, maybe the person who wrote the API did not think to deal with *special characters* ? You need to explain what is happening. Is some exception thrown, if so please provide. – Scary Wombat May 12 '22 at 06:27
  • Sorry, it´s difficult because my main language is not english. The API is not well done, it return me an OK but instead of the data that i want it send a non relevant error message (the message says that the ID is wrong so it´s not even a good message) i will post the call request and the return. – Grismak May 12 '22 at 06:34
  • 1
    That's OK, but let's speak java. Unless we can see what is happening, we can only guess. – Scary Wombat May 12 '22 at 06:35
  • Thanks for your patience and help, i am still learning how to make the questions, but this is really helpful – Grismak May 12 '22 at 06:44
  • `"userlid":"008368790204"` - are you sure that this is correct? `userId` ? – Scary Wombat May 12 '22 at 06:47
  • See also https://stackoverflow.com/questions/6603928/should-i-url-encode-post-data – Scary Wombat May 12 '22 at 06:49
  • You need to post your code so that we can see how the String data (such as lname2) gets converted into bytes. Most likely the conversion is not using UTF-8, which the logs say it should do. – ewramner May 12 '22 at 06:55
  • Yes the UserId is correct, we can generate any sort of string of less that 16 characters as an ID, the message saying that the UserId is needed is something that they got wrong. About code, i will add it in the question too. thanks for the advise ewrammer – Grismak May 12 '22 at 06:59
  • mispellin userId. the correct one is userlid i will update with the postman call – Grismak May 12 '22 at 07:05
  • Finally solved. Thank you so much for your help – Grismak May 12 '22 at 09:51

1 Answers1

0

Finally i solved it.

The problem was deep in our framework but once spotted it was an easy fix:

When you send an Entity as a Post Call and it have special characters, yo usually encode it. Just as ewramner suggested i checked and i confirmed that we had the entity well encoded BUT at the moment to send the entity to the API, we lost the Charset configuration (not the global one but the entity one)

The solution was to add the specific charset of the entity inside the entity type like this:

data.put("entity", paramsCrear.toString());
data.put("entity_mime_type", "application/json");
data.put("entity_charset", "UTF-8");

It seems that postman automatically add this entity_charset to every form-data that you make that´s why I had problems looking for this specific point.

Thank you so much @ewramner and @Scary_wombat

And everyone, keep in mind that if you use a post call in java with entity, you might need to specify the charset in wich you encode the entity (event if it´s put in the headers too.

Grismak
  • 192
  • 16