JasperReports Library Custom Visualization not rendering properly UTF8 components

Question

I'm using JasperReports library for generating pdfs from a custom Java application. However, when rendering € symbol in a Custom Visualization Component, the resulting pdf shows a � symbol instead (this happens with both PhantomJs and Chromium).The rest of € symbols in the report (TextFields) are shown properly, despite using the same font in all of them.

On the other hand,if the same report is rendered from JasperStudio with PhantomJs, the correct symbol is displayed.

I have checked the html file passed to chrome as it is temporarily stored in temp folder and it seems to be correctly encoded in UTF-8 (see below), so I think that it might be a problem related to the font file that jasper provides to Chrome or PhantomJs.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>

<head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
  <link rel="stylesheet" type="text/css" href="jr_script_3615012621213017572_feeRenderer.css" title="Style">
  <script src="jr_res_12941692111646429500_require.js"></script>
  <script src="jr_res_14211739430443032080_cv-component_static.js"></script>
  <script src="jr_res_10307871901394985341_feeRenderer.min.js"></script>
</head>

<body>
  <div id="element423553011047886097313196921978673211730" class="jr_cvc_element" style="width: 389px; height: 290px;" />
  <script class="jasperreports">
    require(["cv-component"], function(Comp) {
      new Comp({
        "id": "element423553011047886097313196921978673211730",
        "renderer": "d3Circle",
        "instanceData": {
          "series": [
            [{
              "Currency_symbol": "€",
              "Currency_height": "0.5",
              "Cents_height": "0.25",
              "Separation": "0",
              "Cents": "0",
              "Currency_margin": "0",
              "Units_height": "1",
              "Units": "8",
              "Margin": "0"
            }]
          ],
          "module": "d3Circle",
          "css_uri": "D:\\Dhom\\Documents\\Projects\\Myreport\\Test5\\feeRenderer.css",
          "width": 389,
          "id": "element423553011047886097313196921978673211730",
          "height": 290,
          "script_uri": "D:\\Dhom\\Documents\\Projects\\Myreport\\Test5\\feeRenderer.min.js",
          "animation": false
        }
      });
    });
  </script>
</body>

</html>

Has anyone faced this problem before?

Thanks in advance

Edit 20/5/2021 11:35:

I've been able to find the svg file sent to chrome and it contains the replacement character (\ufffd) instead of the € symbol, so it seems that the problem is happening during the svg generation stage:

<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>
<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\"><svg xmlns=\ "http://www.w3.org/2000/svg\" id=\ "element66576560284652498011372360525284671058svg\" width=\ "389\" height=\ "240\" style=\ "fill: transparent;\" version=\ "1.1\"><defs><style type=\"text/css\">\nsvg circle { fill: rgb(240, 0, 255); }\n.number-text { fill: white; font-family: arial; }</style></defs><text dominant-baseline=\"hanging\" aligment-baseline=\"hanging\" x=\"389\" y=\"0\" text-anchor=\"end\" class=\"number-text\" style=\"font-size: 60px;\">.95</text><text dominant-baseline=\"hanging\" aligment-baseline=\"hanging\" x=\"305.578125\" y=\"0\" text-anchor=\"end\" class=\"number-text\" style=\"font-size: 240px;\">8</text><text dominant-baseline=\"hanging\" aligment-baseline=\"hanging\" x=\"172.09375\" y=\"0\" text-anchor=\"end\" class=\"number-text\" style=\"font-size: 120px;\">\ufffd</text></svg>

Edit 20/5/2021 13:24:

By logging the variable that contains the currency symbol, it seems that it is being changed by the javascript interpreter. Therefore, if the report is exported as html and opened in Chrome, it gets rendered correctly, but when the javascript code is interpreted as part of the pdf report generation routine, the interpreter is unable to recognize the character and changes it to the Unicode replacement character. The logged value of that variable is shown at line 10547 in the log provided below of the java program:

3587 [main] DEBUG net.sf.jasperreports.customvisualization.export.ChromeCVElementImageDataProvider  - wrote CV render HTML page to C:\Users\Dhom\AppData\Local\Temp\cv_1203404104433554074.html
3589 [main] DEBUG net.sf.jasperreports.chrome.BrowserService  - page evaluation at file:/C:/Users/Dhom/AppData/Local/Temp/cv_1203404104433554074.html
3595 [main] INFO net.sf.jasperreports.chrome.ChromeInstance  - Launching Chrome instance 1 with configuration executable: D:\Dhom\Documents\Projects\Myreport\Test5\chrome.exe, headless: true, arguments: {}, idle timeout: 900000, live timeout: 7200000
3601 [main] INFO com.github.kklisura.cdt.launch.ChromeLauncher  - Launching chrome process D:\Dhom\Documents\Projects\Myreport\Test5\chrome.exe with arguments {no-first-run=true, remote-debugging-port=0, mute-audio=true, disable-client-side-phishing-detection=true, disable-popup-blocking=true, disable-default-apps=true, disable-extensions=true, metrics-recording-only=true, no-default-browser-check=true, disable-background-timer-throttling=true, disable-translate=true, safebrowsing-disable-auto-update=true, headless=true, hide-scrollbars=true, disable-background-networking=true, disable-prompt-on-repost=true, user-data-dir=C:\Users\Dhom\AppData\Local\Temp\cdt-user-data-dir3328865212396958053, disable-hang-monitor=true, disable-sync=true, disable-gpu=true}
8633 [main] DEBUG net.sf.jasperreports.chrome.ChromeInstanceRepository  - schedule chrome instance 1 idle timeout check after 900000
8635 [main] DEBUG net.sf.jasperreports.chrome.ChromeInstanceRepository  - schedule chrome instance 1 timeout after 7200000
8637 [main] DEBUG net.sf.jasperreports.chrome.ChromeInstance  - using chrome instance 1
9724 [main] DEBUG net.sf.jasperreports.chrome.StandardPageCreator  - created tab 902F248522DDE30E16DE2D3CD552FBD9
10020 [main] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Connecting to ws server ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10350 [Grizzly(1)] INFO com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Connected to ws ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10438 [main] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Sending message {"id":1,"method":"Log.enable","params":{}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10443 [Grizzly(2)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"id":1,"result":{}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10462 [main] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Sending message {"id":2,"method":"Runtime.enable","params":{}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10466 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"://","name":"","uniqueId":"-1037893551376530969.-1504717943394719259","auxData":{"isDefault":true,"type":"default","frameId":"902F248522DDE30E16DE2D3CD552FBD9"}}}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10467 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"id":2,"result":{}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10483 [main] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Sending message {"id":3,"method":"Page.enable","params":{}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10484 [Grizzly(2)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"id":3,"result":{}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10486 [main] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Sending message {"id":4,"method":"Page.navigate","params":{"url":"file:/C:/Users/Dhom/AppData/Local/Temp/cv_1203404104433554074.html"}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10489 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"id":4,"result":{"frameId":"902F248522DDE30E16DE2D3CD552FBD9","loaderId":"82EC151841E6F98D209D3B011A2A2FBF"}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10490 [Grizzly(2)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Page.frameStartedLoading","params":{"frameId":"902F248522DDE30E16DE2D3CD552FBD9"}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10494 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Runtime.executionContextsCleared","params":{}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10495 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Page.frameNavigated","params":{"frame":{"id":"902F248522DDE30E16DE2D3CD552FBD9","loaderId":"82EC151841E6F98D209D3B011A2A2FBF","url":"file:///C:/Users/Dhom/AppData/Local/Temp/cv_1203404104433554074.html","domainAndRegistry":"","securityOrigin":"file://","mimeType":"text/html","adFrameType":"none","secureContextType":"Secure","crossOriginIsolatedContextType":"NotIsolated","gatedAPIFeatures":["SharedArrayBuffers","SharedArrayBuffersTransferAllowed"]}}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10497 [Grizzly(2)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Runtime.executionContextCreated","params":{"context":{"id":2,"origin":"file://","name":"","uniqueId":"-4551564477363548089.1093106506060346497","auxData":{"isDefault":true,"type":"default","frameId":"902F248522DDE30E16DE2D3CD552FBD9"}}}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10497 [main] DEBUG net.sf.jasperreports.chrome.BrowserService  - waiting for result, timeout 60000
10514 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Page.loadEventFired","params":{"timestamp":247828.898653}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10515 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Page.frameStoppedLoading","params":{"frameId":"902F248522DDE30E16DE2D3CD552FBD9"}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10516 [Grizzly(2)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Page.domContentEventFired","params":{"timestamp":247828.898826}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10521 [pool-1-thread-1] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Sending message {"id":5,"method":"Runtime.evaluate","params":{"expression":"renderResult(true)"}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10522 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"id":5,"result":{"result":{"type":"object","subtype":"promise","className":"Promise","description":"Promise","objectId":"-3590625203254494070.2.1"}}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10547 [Grizzly(2)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"method":"Runtime.consoleAPICalled","params":{"type":"log","args":[{"type":"string","value":"\ufffd"}],"executionContextId":2,"timestamp":1.621509394264413e+12,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"6","url":"file:///C:/Users/Dhom/AppData/Local/Temp/jr_res_8667172309765663436_feeRenderer.min.js","lineNumber":4,"columnNumber":19892},{"functionName":"","scriptId":"4","url":"file:///C:/Users/Dhom/AppData/Local/Temp/jr_res_8570279741294855884_cv-component_static.js","lineNumber":47,"columnNumber":16},{"functionName":"execCb","scriptId":"3","url":"file:///C:/Users/Dhom/AppData/Local/Temp/jr_res_8609129502099932916_require.js","lineNumber":1695,"columnNumber":32},{"functionName":"check","scriptId":"3","url":"file:///C:/Users/Dhom/AppData/Local/Temp/jr_res_8609129502099932916_require.js","lineNumber":882,"columnNumber":50},{"functionName":"enable","scriptId":"3","url":"file:///C:/Users/Dhom/AppData/Local/Temp/jr_res_8609129502099932916_require.js","lineNumber":1175,"columnNumber":21},{"functionName":"init","scriptId":"3","url":"file:///C:/Users/Dhom/AppData/Local/Temp/jr_res_8609129502099932916_require.js","lineNumber":787,"columnNumber":25},{"functionName":"","scriptId":"3","url":"file:///C:/Users/Dhom/AppData/Local/Temp/jr_res_8609129502099932916_require.js","lineNumber":1459,"columnNumber":35}]}}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10552 [pool-1-thread-1] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Sending message {"id":6,"method":"Runtime.awaitPromise","params":{"returnByValue":true,"generatePreview":false,"promiseObjectId":"-3590625203254494070.2.1"}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10574 [Grizzly(1)] DEBUG com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Received message {"id":6,"result":{"result":{"type":"string","value":"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?><!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\"><svg xmlns=\"http://www.w3.org/2000/svg\" id=\"element1445529659355478804511607219085981653206svg\" width=\"389\" height=\"240\" style=\"fill: transparent;\" version=\"1.1\"><defs><style type=\"text/css\">\nsvg circle { fill: rgb(240, 0, 255); }\n.fee-text { fill: white; font-family: arial; }</style></defs><text dominant-baseline=\"hanging\" aligment-baseline=\"hanging\" x=\"389\" y=\"0\" text-anchor=\"end\" class=\"fee-text\" style=\"font-size: 60px;\">.95</text><text dominant-baseline=\"hanging\" aligment-baseline=\"hanging\" x=\"305.578125\" y=\"0\" text-anchor=\"end\" class=\"fee-text\" style=\"font-size: 240px;\">8</text><text dominant-baseline=\"hanging\" aligment-baseline=\"hanging\" x=\"172.09375\" y=\"0\" text-anchor=\"end\" class=\"fee-text\" style=\"font-size: 120px;\">\ufffd</text></svg>"}}} on ws://localhost:1906/devtools/page/902F248522DDE30E16DE2D3CD552FBD9
10577 [main] INFO com.github.kklisura.cdt.services.impl.WebSocketServiceImpl  - Web socket connection closed NORMAL_CLOSURE,
10581 [pool-1-thread-1] DEBUG net.sf.jasperreports.chrome.BrowserService  - Page console LOG: ?

Using &euro just results in the text "&euro" being shown in the generated pdf file as I'm using d3.js and it does not allow to include html elements inside svg elements. However, I'm not sure if it would be possible to solve this problem using that html encoding with other libraries like RaphaelJs or using just plain javascript or if this is a limitation caused by the fact that the file generated by jasper needs to be an svg component (the web page shown in my question is automatically generated by jasper and, thus, I cannot modify it) — Dhom, May 20 '21 at 09:24
might be a font problem, can you check this out http://jasperstarter.cenote.de/unicode-pdf-export.html https://stackoverflow.com/questions/40402377/some-currency-symbol-in-jasper-report-is-not-coming . to see if the first one works, change the font to DejaVu Sans — tremendous7, May 20 '21 at 10:28
Initially I thought about that, but it just fails in the case of the CVC component (in text fields of the report the symbol is rendered correctly and both, the CVC and those texts, use Arial, which is a Unicode font). Additionally, the Unicode code of the symbol is being changed before passing it to Chrome, so it does not seem to be related to the font used (see edit). I have also tried with other fonts and the problem persists — Dhom, May 20 '21 at 10:38

score 1 · Answer 1 · answered May 21 '21 at 08:49

This problem has finally resulted to be a bug in Jasper Reports Library upto version 6.17 caused by not properly setting the communication stream between Chrome and Jasper to use UTF-8 (see https://github.com/TIBCOSoftware/jasperreports/issues/189#issuecomment-845341796 and https://github.com/TIBCOSoftware/jasperreports/issues/190). As a result, the default character encoding of JVM is used, leading to data corruption when some characters are used.

Workaround 1: The Jasper Reports maintainer that found the cause (github user dadza) has suggested a workaround to solve it until a corrected version is released, consisting in setting the correct character encoding by adding the following option when running the program: -Dfile.encoding=UTF-8

Workaround 2: According to https://stackoverflow.com/a/14987992/15974273, in some JVM versions it is possible to change the default character encoding at runtime using a dirty hack. Although it is strongly not recommended to use it, adding the following lines of code in your main program may make those symbols to be rendered properly (security manager might require to be turned off according to some comments of the original answer providing this solution):

System.setProperty("file.encoding","UTF-8");
Field charset = Charset.class.getDeclaredField("defaultCharset");
charset.setAccessible(true);
charset.set(null,null);

JasperReports Library Custom Visualization not rendering properly UTF8 components

1 Answers1