Send Special characters using Unicode code point in request body without conversion

Question

I am working on a Kotlin SpringBoot application.

I need to send a special character (Euro symbol for example) in the request. If I send the unicode code point then it is parsed and displayed correctly by the third party system.

I could do that with Postman tool but not able to do the same from my app.

Postman example:

{
    "comment": "The value is € 200 and \u20ac 300"
}

Third party system output:

{
    "comment": "The value is â‚¬ 200 and \u20ac 300"
}

Here we can see that \u20ac is sent correctly from Postman to the third party system.

My application code:

Controller:

import org.springframework.http.HttpStatus
import org.springframework.http.ResponseEntity
import org.springframework.web.bind.annotation.PostMapping
import org.springframework.web.bind.annotation.RestController

@RestController
class TestRestController(
  private val restClient: RestClient
) {

  data class Data(
    val comment: String
  )

  @PostMapping("/data-test")
  fun dataTest(): ResponseEntity<String> {
    val euroSymbol = "\u20ac"
    val escapedEuroSymbol = "\\u20ac"

    val mockData = Data(
      comment = "The value is $euroSymbol 200 and $escapedEuroSymbol 300"
    )

    restClient.createRequest(mockData)

    return ResponseEntity.status(HttpStatus.OK).body("ok")
  }
}

Rest client:

import org.springframework.cloud.openfeign.FeignClient
import org.springframework.http.MediaType
import org.springframework.web.bind.annotation.PostMapping
import org.springframework.web.bind.annotation.RequestBody
import javax.validation.Valid

@FeignClient(
  value = "restClient",
  url = "https://envpqi8gjjxx8.x.pipedream.net/",
)
interface RestClient {
  @PostMapping(
    value = ["/test"],
    consumes = [MediaType.APPLICATION_JSON_VALUE]
  )
  fun createRequest(
    @RequestBody request: @Valid TestRestController.Data
  ): String?
}

My object mapper config:

objectMapper.setTimeZone(TimeZone.getDefault())
objectMapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
objectMapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL)
objectMapper.registerModules(kotlinModule(), JavaTimeModule())

Third party system output:

{
    "comment": "The value is â‚¬ 200 and \\u20ac 300"
}

Now in the mockData variable you can see that I am using both \u20ac and \\u20ac. for the first one the value is corrupt and the second one is sent as \\u20ac. I want to be able to send \u20ac as plain text similar to how I was able to do it using Postman.

How to do that?

In JSON `€` and `\u20ac` represent the exact same character and the framework that produces your JSON will pick whichever it thinks is best. You might be able to configure which one it will prefer, but you almost certainly won't be able to get one string contain both variants (and there's basically no reason why you'd want that). The "third party system output" looks like something interprets UTF-8 data as some non-UTF-8 encoding. — Joachim Sauer, Mar 28 '23 at 19:57
I want `\u20ac`. I dont want other variant. I was just trying out with extra backslash and still it did not work. I dont know how to solve this, may be using some HttpMessageConverter — firstpostcommenter, Mar 28 '23 at 20:04
As Joachim says, this seems likely to be about how the output is _interpreted_, rather than its content: the receiver is interpreting it as ISO Latin-1 instead of UTF-8. Have you tried specifying `@PostMapping(/*…*/, produces = "text/plain;charset=UTF-8")`? Or see e.g. [this question](https://stackoverflow.com/questions/5649329/utf-8-encoding-problem-in-spring-mvc). — gidds, Mar 29 '23 at 00:53
text/plain is not working as I am sending a kotlin data class and not String. I get error — firstpostcommenter, Mar 29 '23 at 05:11

firstpostcommenter · Answer 1 · 2023-04-02T10:48:37.413

You can use Jackson's JsonGenerator writeRawValue for this.

Step1:

In dataTest() use escapedEuroSymbol and not the non escaped variable.

Step2 - Create a new serialiser:

import com.fasterxml.jackson.core.JsonGenerator
import com.fasterxml.jackson.databind.JsonSerializer
import com.fasterxml.jackson.databind.SerializerProvider
import java.io.IOException

//use Unicode code points(like escapedEuroSymbol) for special characters in the string variable if using this Serializer on the variable
class StringWithSpecialCharacterSerializer : JsonSerializer<String?>() {
  @Throws(IOException::class)
  override fun serialize(value: String?, gen: JsonGenerator, serializers: SerializerProvider) {
    if (value != null) {
      val newString = "\"" + value + "\""
      val newStringEscapeNewLine = newString
                  .replace(System.lineSeparator(), "\\n")
                  .replace("\t", "\\t")
                  .replace("\b", "\\b")
      gen.writeRawValue(newStringEscapeNewLine)
    }
  }
}

Step3 - And then in your data class you can use the serialiser on the String fields whose values contain special characters like euro symbol etc(always use escaped unicode code point and not the special character directly when using this serialiser):

data class Data(
    @JsonSerialize(using = StringWithSpecialCharacterSerializer::class)
    val comment: String
  )

Send Special characters using Unicode code point in request body without conversion

1 Answers1