Handling Unicode character in Mule's dataweave

Question

We have a DataWeave expression wherein the payload, for some of the fields, special unicode control characters are coming.

This is leading to a WeaveExecutionException with the following message:

Exception while executing:
"userComment": "CA*2249*0*1763335*54133300896010155

^ Unexpected character '\u000a' at payload@[8:116] (line:column), expected '"'.

Below is the payload which we are getting. Here, in the field usercomment, a \u000a (new line character) is coming:

{
    "dcsId":"11840000000000001",
    "accessToken":"jaskjadkjsa",
    "profile": {
        "comments": {
            "feedback": [{
                "feedBackType":
                    "5",
                    "userComment": "CA*2249*0*1763335*54133300896010155
1" }
                        ]
                    }
                },
    "platform": "iphone"
}

Below is the logged Exception which we are getting in the console:

com.mulesoft.weave.mule.exception.WeaveExecutionException: Exception 
while executing:                                                                 
"userComment": "CA*2249*0*1763335*54133300896010155

^
Unexpected character '\u000a' at payload@[8:116] (line:column), expected 
'"'.

Below is the DataWeave script we are executing:

%dw 1.0
%output application/java
---
payload

Does DataWeave have no support for unicode control characters?

score 0 · Answer 1 · edited Mar 14 '18 at 15:56

0

When you are doing transformations using either JSON to XML or XML to JSON please set Content-Type 'application/json; charset=UTF-8'.

edited Mar 14 '18 at 15:56

Attila

3,206
2
31
44

answered Mar 14 '18 at 15:36

Kamal Vamsi

128
4

score 0 · Answer 2 · answered Mar 14 '18 at 15:44

There is a related question from earlier: How to get more information about dataweave exception in muleosft

According to this: you are supposed to escape the special, "control unicode characters".

Instead of new line, you are supposed to have \n in your JSON.

Please see also: How do I handle newlines in JSON?

For details see the JSON standard; the ECMA - 404.

In this document on the page 9, in the chapter "String" you can see a detailed description how the unicode control characters should be represented in JSON.

A JSON string must not contain a new-line charachter. Your JSON data has invalid format, what leads to an Exception.

According to this: You have to make sure that the unicode control characters are properly escaped in your JSON strings.

score 0 · Answer 3 · edited Mar 16 '18 at 11:06

0

In datawave use the operation as :string

In your case:

"userComment": "CA*2249*0*1763335*54133300896010155 as :string

edited Mar 16 '18 at 11:06

Attila

3,206
2
31
44

answered Mar 15 '18 at 07:25

Swapnil Udamale

61
5

score 0 · Answer 4 · edited Jul 18 '22 at 02:04

If you are getting invalid whitespace character while converting your JSON to XML, then first we need to find in which it is reflecting; after that, we can use replace /{unicode value}/ with "".

We can use Java code to remove non-printable characters from Unicode.

package org.mycompany.utils;

public class whiteSpace {
    public static String myInvalidChar(String text) { 
        /* strips off all non-ASCII characters */
        //text = text.replaceAll("[^\\x00-\\x7F]", "");
        /* erases all the ASCII control characters */ 
        //text = text.replaceAll("[\\p{Cntrl}&&[^\r\n\t]]", "");
        /* removes non-printable characters from Unicode */ 
        //text = text.replaceAll("\\p{C}", "");
        //return text.trim();
    }
}

Which ever the error that you are getting, you can uncomment and use the code import to dataweave and use the method at the field level.

Handling Unicode character in Mule's dataweave

4 Answers4