3

is it possible to send UTF-8 character from a okhttp3 client ?

For the following string :

String fileName = "3$ Mù F'RANçé_33902_Country_5_202105";
String contentDisposition = "attachment;filename=" + "\"" +  fileName + "\"";

I've tried (for the contentDisposition header) :

Headers headers = new Headers.Builder()
                       .addUnsafeNonAscii("Content-Disposition", contentDisposition)
                       .add("Authorization", bearer)
                       .add("Content-type", "application/octet-stream")
                       .build();
             Request request = new Request.Builder()
                     .headers(headers)
                     .post(requestBody) 
                     .url(urlAddress)
                     .build();

But the server receive : 3$ Mù F'RANçé_33902_Country_5_202105

This request is send to a firm partner, so I have no access to the back-end.

application/octet-stream is needed by the back-end.

Body is created like this :

byte[] data = FileUtils.readFileToByteArray(file);
RequestBody requestBody = RequestBody.create(data);

It works perfectly fine with Postman.

Full MVCE (cannot be complete with file and back-end informations but it crashes before, anyway, so you can just start this exact code and it should throws the error) :

public class App 
{
    public static void main( String[] args ) throws IOException
    {
                OkHttpClient client = new OkHttpClient().newBuilder()
                    .build();
                MediaType mediaType = MediaType.parse("application/octet-stream");
                RequestBody body = RequestBody.create(mediaType, "");
                Request request = new Request.Builder()
                  .url("xxxx")
                  .method("POST", body)
                  .addHeader("Content-Type", "application/octet-stream")
                  .addHeader("content-disposition", "attachment;filename=\"3$ Mù F'RANçé_33902_Country_5_202105.csv\"")
                  .addHeader("Authorization", "Bearer xxxxx")
                  .addHeader("Cookie", "xxxxxx")
                  .build();
                Response response = client.newCall(request).execute();
    }
}

Error received : java.lang.IllegalArgumentException: Unexpected char 0xf9 at 25 in content-disposition value: attachment;filename="3$ Mù F'RANçé_33902_Country_5_202105.csv"

okhttp version : 5.0.0-alpha.2

Did I miss something ?

Thanks

kriegaex
  • 63,017
  • 15
  • 111
  • 202
Cesar
  • 453
  • 9
  • 21
  • Can you explain the reason why you are sending content type `application/octet-stream` and not some text type? – kriegaex Jan 24 '22 at 08:46
  • Many details are missing. What does the corresponding Postman request look like exactly? Are you sending this to a web form or some kind or REST service? How did you create `contentDisposition` and `requestBody`? An [MCVE](https://stackoverflow.com/help/mcve) would really be more helpful than a code snippet out of context and you wasting a bounty on a question with insufficient detail. – kriegaex Jan 24 '22 at 09:00
  • @kriegaex I've updated my question. You were right. Thx – Cesar Jan 24 '22 at 09:20
  • Oh, so it is not about UTF-8 characters in the post body, but in an HTTP header field. Please confirm. Because I thought you wanted UTF-8 in the body. That would have been simply `.add("Content-type", "application/octet-stream; charset=utf-8")`. – kriegaex Jan 24 '22 at 09:40
  • @kriegaex Yes, problem is in the "content-disposition" error, which contains the fileName. – Cesar Jan 24 '22 at 09:49

2 Answers2

11

The default character set for HTTP headers is ISO-8859-1. There is however RFC 6266, describing how you can encode the file name in a Content-Disposition header. Basically, you specify the character set name and then percent-encode the UTF-8 characters. Instead of fileName="my-simple-filename" you use a parameter starting with filename*=utf-8'' like

import java.net.URLEncoder;

// ...

String fileName = "3$ Mù F'RANçé_33902_Country_5_202105";
String contentDisposition = "attachment;filename*=utf-8''" + encodeFileName(fileName);

// ...

private static String encodeFileName(String fileName) throws UnsupportedEncodingException {
  return URLEncoder.encode(fileName, "UTF-8").replace("+", "%20");
}

Using the URL encoder and then modifying the result for "+" is a cheap trick I found here, if you want to avoid using Guava, Spring's ContentDisposition class or any other library and simply work with JRE classes.


Update: Here is a full MCVE, showing how to send an UTF-8 string both as a POST body and as a content disposition file name. The demo server shows how to decode that header manually - usually HTTP servers should do that automatically.

Maven POM showing used dependencies:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.example</groupId>
  <artifactId>SO_Java_OkHttp3SendUtf8_70804280</artifactId>
  <version>1.0-SNAPSHOT</version>

  <properties>
    <maven.compiler.source>11</maven.compiler.source>
    <maven.compiler.target>11</maven.compiler.target>
  </properties>

  <dependencies>
    <dependency>
      <groupId>com.squareup.okhttp3</groupId>
      <artifactId>okhttp</artifactId>
      <version>4.9.3</version>
    </dependency>
    <dependency>
      <groupId>org.nanohttpd</groupId>
      <artifactId>nanohttpd</artifactId>
      <version>2.3.1</version>
    </dependency>
  </dependencies>

</project>

OkHttp demo client:

import okhttp3.Headers;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;

import java.io.IOException;
import java.net.URL;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.Objects;

public class Client {
  public static void main(String[] args) throws IOException {
    String fileName = "3$ Mù F'RANçé_33902_Country_5_202105";
    String contentDisposition = "attachment;filename*=utf-8''" + encodeFileName(fileName);
    RequestBody requestBody = RequestBody.create(fileName.getBytes(StandardCharsets.UTF_8));
    Headers headers = new Headers.Builder()
      .add("Content-Disposition", contentDisposition)
      .add("Content-type", "application/octet-stream; charset=utf-8")
      .build();
    Request request = new Request.Builder()
      .headers(headers)
      .post(requestBody)
      .url(new URL("http://localhost:8080/"))
      .build();
    OkHttpClient client = new OkHttpClient();
    Response response = client.newCall(request).execute();
    System.out.println(Objects.requireNonNull(response.body()).string());
  }

  private static String encodeFileName(String fileName) {
    return URLEncoder.encode(fileName, StandardCharsets.UTF_8).replace("+", "%20");
  }
}

NanoHTTPD demo server:

import fi.iki.elonen.NanoHTTPD;

import java.io.IOException;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;

public class Server extends NanoHTTPD {

  public Server() throws IOException {
    super(8080);
    start(NanoHTTPD.SOCKET_READ_TIMEOUT, false);
    System.out.println("\nRunning! Point your browsers to http://localhost:8080/ \n");
  }

  public static void main(String[] args) throws IOException {
    new Server();
  }

  private static final String UTF_8_FILE_NAME_PREFIX = ";filename*=utf-8''";
  private static final int UTF_8_FILE_NAME_PREFIX_LENGTH = UTF_8_FILE_NAME_PREFIX.length();

  @Override
  public Response serve(IHTTPSession session) {
    try {
      Map<String, String> files = new HashMap<>();
      session.parseBody(files);
      String postBody = files.get("postData");
      String contentDisposition = session.getHeaders().get("content-disposition");
      String fileName = decodeFileName(
        contentDisposition.substring(
          contentDisposition.indexOf(UTF_8_FILE_NAME_PREFIX) + UTF_8_FILE_NAME_PREFIX_LENGTH
        )
      );
      System.out.println("POST body:           " + postBody);
      System.out.println("Content disposition: " + contentDisposition);
      System.out.println("UTF-8 file name:     " + fileName);
      return newFixedLengthResponse(postBody + "\n" + fileName);
    }
    catch (IOException | ResponseException e) {
      e.printStackTrace();
      return newFixedLengthResponse(e.toString());
    }
  }

  private static String decodeFileName(String fileName) {
    return URLDecoder.decode(fileName.replace("%20", "+"), StandardCharsets.UTF_8);
  }

}

If first you run the server and then the client, you will see this on the server console:

Running! Point your browsers to http://localhost:8080/ 

POST body:           3$ Mù F'RANçé_33902_Country_5_202105
Content disposition: attachment;filename*=utf-8''3%24%20M%C3%B9%20F%27RAN%C3%A7%C3%A9_33902_Country_5_202105
UTF-8 file name:     3$ Mù F'RANçé_33902_Country_5_202105

On the client console, you see:

3$ Mù F'RANçé_33902_Country_5_202105
3$ Mù F'RANçé_33902_Country_5_202105
kriegaex
  • 63,017
  • 15
  • 111
  • 202
  • I've been trying with 'filename*=' and saw this RFC, but I missed the 'utf-8' part. It works perfectly fine with your method ! Thanks ! – Cesar Jan 24 '22 at 10:29
  • **Update:** Just for reference, I added an [MCVE](https://stackoverflow.com/help/mcve) to my answer. – kriegaex Jan 24 '22 at 11:49
  • Sorry, I thought this was automatically send to the choosen answer. Just awarded you ! Have a good day sir ! – Cesar Jan 27 '22 at 07:39
3

Those who use spring framework and experience this error can do something like this:

HttpHeaders httpHeaders = new HttpHeaders();
ContentDisposition contentDisposition = ContentDisposition.builder("attachment")
    .filename("filename with unicode chars.csv", StandardCharsets.UTF_8).build();
httpHeaders.setContentDisposition(contentDisposition);
return ResponseEntity.ok().headers(httpHeaders)
    .contentType(MediaType.TEXT_PLAIN).body(byteArrayResourceObj);
Zaziro
  • 405
  • 4
  • 8