String contains control character ERROR in script mediator - WSO2 EI 6.5.0

Question

I'm receiving JSON payload from customer, sometimes incoming payload value is lengthy and contains multiple special characters in some field eg (description).

Sample Payload:

    {
   "InstanceName":"clientDEV",
   "Direction":"InBound",
   "CustomerName":"client",
   "CustomerID":"6642671519xxx",
   "ObjectID":"c7cb6e97c399xxx",
   "ObjectName":"SCTASK0010101",
   "ObjectType":"Service Request",
   "OperationName":"UpdateObject",
   "ObjectData":{
      "description":"From: , Lene Bjørhei <Lene.Bjoerhei@client.com> , Sent: 19 August 2022 10:49 AM, To: Helpdesk <helpdesk@client.com>, Cc: Christensen, Helge <helge.christensen@client.com>, Subject: Power BI, , Hi, , , I have two names in Power BI, where the one marked with red is wrong. , , On the same list, Hanne Topland don, t work in client anymore and has no claims attached to her in PBI, and Kine Haaland is not in our department and has no claims here either. , ,    Kind reclients, Lene Bjørhei ,   Team leader,   P&I and Offshore Energy claims, client AS, o: +47 39 00 00 00  , m: +47 24 00 05 11, e: lene.bjoerhei@client.com",
      "cmdb_ci":"",
      "number":"SCTASK0010101",
      "contact_type":"Email",
      "state":"-5",
      "sys_created_on":"2022-08-19 10:05:39",
      "sys_created_by":"Maria@client.com",
      "priority":"4",
      "category":"inquiry",
      "subcategory":"internal application",
      "short_description":"Application - Power BI Claim handler names",
      "assignment_group":"Cognizant ICT Services - Service Desk",
      "caller":"Lene Bjørhei ",
      "close_code":"",
      "close_notes":"undefined",
      "comments":"",
      "work_notes":"Work notes: From: Christensen, Helge <helge.christensen@client.com>\n Sent: Monday, August 22, 2022 7:33 AM\n To: , Lene Bjørhei <Lene.Bjoerhei.@client.com>\n Cc: Helpdesk <helpdesk@client.com>\n Subject: FW: Power BI\n  \n Good morning Lene,\n \n Your old and misspelled name has been removed from the data source, sorry for the inconvenience.\n \n Kr\n Helge\n \n \n \n Additional comments: ",
      "u_queue":"",
      "on_hold_reason":"Awaiting Customer IT SPOC"
   }
}

API Code Snippet:

    <property expression="json-eval($.text)" name="received_payload" scope="default" type="STRING"/>
<property expression="base64Decode(get-property('received_payload'))" name="decoded_payload" scope="default" type="STRING" xmlns:ns="http://org.apache.synapse/xsd"/>
    <log level="custom">
        <property expression="get-property('decoded_payload')" name="DATA:DecodedPayload" xmlns:ns="http://org.apache.synapse/xsd"/>
    </log>
    <script language="js"><![CDATA[var customer_payload = mc.getProperty("decoded_payload");
var myJSON = JSON.parse(customer_payload);
//remianing business flow
]]></script>

Facing String contains control character ERROR in script mediator when some special character comes in the payload.

Log:

    [2022-08-22 05:29:38,840] [-1234] [] [pool-58-thread-68583] ERROR {org.apache.synapse.mediators.bsf.ScriptMediator} -  The script engine returned an error executing the inlined js script function mediate
com.sun.phobos.script.util.ExtendedScriptException: org.mozilla.javascript.EcmaError: SyntaxError: String contains control character (<Unknown Source>#2) in <Unknown Source> at line number 2
    at com.sun.phobos.script.javascript.RhinoCompiledScript.eval(RhinoCompiledScript.java:68)
    at javax.script.CompiledScript.eval(CompiledScript.java:92)
    at org.apache.synapse.mediators.bsf.ScriptMediator.mediateForInlineScript(ScriptMediator.java:394)
    at org.apache.synapse.mediators.bsf.ScriptMediator.invokeScript(ScriptMediator.java:289)
    at org.apache.synapse.mediators.bsf.ScriptMediator.mediate(ScriptMediator.java:257)
    at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:108)
    at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:70)
    at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:158)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectInbound(Axis2SynapseEnvironment.java:445)
    at org.wso2.carbon.inbound.amazonsqs.AmazonSQSPollingConsumer.injectMessage(AmazonSQSPollingConsumer.java:267)
    at org.wso2.carbon.inbound.amazonsqs.AmazonSQSPollingConsumer.poll(AmazonSQSPollingConsumer.java:206)
    at org.wso2.carbon.inbound.amazonsqs.AmazonSQSPollingConsumer.poll(AmazonSQSPollingConsumer.java:58)
    at org.wso2.carbon.inbound.endpoint.protocol.generic.GenericTask.taskExecute(GenericTask.java:41)
    at org.wso2.carbon.inbound.endpoint.common.InboundTask.execute(InboundTask.java:45)
    at org.wso2.carbon.mediation.ntask.NTaskAdapter.execute(NTaskAdapter.java:98)
    at org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.mozilla.javascript.EcmaError: SyntaxError: String contains control character (<Unknown Source>#2)
    at org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3687)
    at org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3665)
    at org.mozilla.javascript.NativeJSON.parse(NativeJSON.java:112)
    at org.mozilla.javascript.NativeJSON.execIdCall(NativeJSON.java:88)
    at org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:97)
    at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
    at org.mozilla.javascript.gen._Unknown_Source__79771._c_script_0(<Unknown Source>:2)
    at org.mozilla.javascript.gen._Unknown_Source__79771.call(<Unknown Source>)
    at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
    at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
    at org.mozilla.javascript.gen._Unknown_Source__79771.call(<Unknown Source>)
    at org.mozilla.javascript.gen._Unknown_Source__79771.exec(<Unknown Source>)
    at com.sun.phobos.script.javascript.RhinoCompiledScript.eval(RhinoCompiledScript.java:55)
    ... 21 more

Found this, But not sure this will help or not.

How to handle this ? is there any way to allow this characters in payload?

Answer 1

I don't think this has to do anything with the Payload. The error indicated that it detected control characters in your script. Also, I don't see any control characters in the shared JSON payload. If your JSON is valid here it should work.

So double-check the content of your script, maybe you can start with something simple and try to expand. Also, instead of language="js" try to use <script language="nashornJs"> instead.

If you really think the error is generated due to the payload, try to replace the control characters before parsing the string to a JSON. Something like below,

var customer_payload = mc.getProperty("decoded_payload");
customer_payload.replace(/[\u0000-\u001F\u007F-\u009F]/g, "")
var myJSON = JSON.parse(customer_payload);

Update

After looking at the full Stacktrace the error seems to be caused by the payload. It's thrown somewhere here in the source. So replacing the control character should work.