3

I have a similar question to this: ColdFusion, CFDirectory and the French which was not given a satisfactory answer.

We have upgraded from Coldfusion 9 to Coldfusion 11. So far no major problems except the following:

When using CFdirectory to display file names that contain non ASCII characters in their names (eg: accents, umlauts) we get to see the file name with replacement characters � instead of the correct UTF equivalent. For example a file named L’État, c’est moi.pdf is displayed as L�����tat, c���est moi.pdf.

We are confident that this is a Coldfusion issue as nothing has changed but the Coldfusion version. With Coldfusion 9 CFdirectory worked OK when listing the same accented filenames. Our OS is Redhat 7.0 and the file names are also displayed correctly on the terminal with the ls command. I have also created a quick PHP script to see if PHP can read correctly the directory with the "readdir" command and there no problems there either, filenames are rendered correctly.

So I believe this has to be a Coldfusion 11 issue. I have added the -Dfile.encoding=UTF-8 -Dencoding=UTF-8 parameters in the JVM settings from the Coldfusion administrator server interface but it made no difference.

Any suggestions on how to rectify this would be appreciated.

example of code used follows:

<cfdirectory
action="list"
directory="#ExpandPath( './' )#/pdfs"
listinfo="name"
name="qFile"
/>  
<cfdump
var="#qFile#"
label="All Files"
/>

This the output of the cfdump:

Community
  • 1
  • 1
user1201723
  • 115
  • 2
  • 8
  • I'm dead keen to know how you're "using CFdirectory to display files". Any chance you can actually provide us with some code that demonstrates what you're doing? (reading: http://blog.adamcameron.me/2013/09/short-self-contained-correct-compilable.html) – Adam Cameron Nov 21 '14 at 09:31
  • Sorry Adam as English is not my first language perhaps I did not express my self correctly. I meant we use CFdirectory to list the file names of the selected directory. I have added relevant code in my question. I hope I am clear this time :-) – user1201723 Nov 21 '14 at 09:46
  • Yes it is, cheers. I'm tied up just now, but will have a look @ this @ lunchtime (about 3h time) if no-one else gets back to you first. – Adam Cameron Nov 21 '14 at 10:38
  • Did you restart CF after making the arg change? – Mark A Kruger Nov 21 '14 at 14:35
  • 1
    Wait - your problem might be with CFDUMP. Set up your page with the correct display encoding and then use cfoutput to output the files. Those question marks just indicate out of bounds ascii characters right? – Mark A Kruger Nov 21 '14 at 14:36
  • @Mark A Kruger Yes, I did restart CF and have also used cfoutput, no change. I have narrowed down the problem to be tomcat (which CF11 uses) related. Someone had the same issue with railo and fixed it by changing the setenv.sh settings file of tomcat. See here: https://issues.jboss.org/browse/RAILO-3090. In my case , I am not using Railo and CF seems to be using a custom tomcat implementation and I cant modify any of its settings or at least it's not obvious/documented how to do so for a CF 11 / tomcat instance. – user1201723 Nov 21 '14 at 15:00
  • Sorry for late response, @user1201723. Just tried your code with that exact file name on Windows, and had no issue. I don't have Linux available to test on, so not sure I can be much more help here. :-( – Adam Cameron Nov 23 '14 at 14:20
  • @Adam Cameron, quick question -what is your JAVA version on Windows? Is it anything but 1.7 by any chance? – user1201723 Nov 24 '14 at 10:14

4 Answers4

1

Have you tried setting the cfprocessingdirective tag?

<cfprocessingdirective pageencoding="utf-8">

CF 11 WikiDocs

Also, In the Chrome Network Inspector, make sure the encoding is being returned correctly. Eg:

Content-Type:text/html; charset=UTF-8
BuzzCloudAU
  • 153
  • 7
  • Yes I have tried this, no difference. The encoding is returned as utf-8 either with or without – user1201723 Nov 21 '14 at 09:22
  • 2
    `` just tells CF how to compile the source code files.It has no bearing on data processing or output. Reading: http://blog.adamcameron.me/2013/10/cfcamp-and-how-not-to-use-it.html – Adam Cameron Nov 21 '14 at 09:33
  • I have run this up in CF11 on OSX 10.10.1 and I can loop/output your example file name, and if I dump the cfdirectory result, the name also displays. Could you update your question with some sample code? – BuzzCloudAU Nov 21 '14 at 09:42
  • @SlickRick I have added the relevant code and a screenshot of the cfdump output. – user1201723 Nov 21 '14 at 09:56
  • I am still unable to duplicate your results on my system. For a sanity check, can you run this code and check its output. #stm.getProperty("sun.jnu.encoding")# – BuzzCloudAU Nov 21 '14 at 10:18
  • @SlickRick Sure no problem, I ran the code and the output is : UTF-8. – user1201723 Nov 21 '14 at 10:25
  • 1
    By the way I have also tried an alternative to CFdirectory in order to list the files, It is a JAVA version of CFdirectory I found here: http://www.anujgakhar.com/2007/11/06/java-version-of-cfdirectory/. Unfortunately the error with the replacement characters still persists :-( – user1201723 Nov 21 '14 at 10:29
  • If thats the case, then maybe this question will point you in the right direction: http://stackoverflow.com/questions/3610013/file-listfiles-mangles-unicode-names-with-jdk-6-unicode-normalization-issues – BuzzCloudAU Nov 21 '14 at 10:36
  • @SlickRick Thanks I will investigate this – user1201723 Nov 21 '14 at 10:40
0

If your environment is Linux, you need to have a clean UTF-8 configuration.

Please have a look here.

I had the same problem, I just add into the file ~/.bashrc these lines: 

export LC_ALL=en_US.UTF-8 
export LANG=en_US.UTF-8 
export LANGUAGE=en_US.UTF-8

After that, don't forget to restart your Coldfusion Server

sudo /opt/coldfusion11/cfusion/bin/coldfusion restart
kvorobiev
  • 5,012
  • 4
  • 29
  • 35
Denis
  • 1
  • Our Linux locale configuration has always been like the example you posted. When Coldfusion 9 was installed there were no file name display problems. The problem began after we upgraded to Coldfusion 11. Therefore , at least in our case, this is not a Linux locale misconfiguration issue. – user1201723 Apr 21 '16 at 07:33
0

Please see: Why are certain characters not being injected correctly to SQL Server from a CFQUERY?

Make sure your file is saved with encoding Unicode UTF-8.

Also make sure your JVM arguments will process that as well. Admin > Server Settings > Java and JVM. Add " -Dfile.encoding=UTF-8" to the Arguments.

enter image description here

Community
  • 1
  • 1
Jules
  • 1,941
  • 15
  • 18
  • In my question I already mentioned that I the "-Dfile.encoding=UTF-8 -Dencoding=UTF-8" parameter is included in the JAVA and JVM settings. The files are Unicode UTF-8 encoded. In any case the problem is not with the file contents which display fine but with the file names which CFDIRECTORY does not show correctly as already explained. – user1201723 Sep 29 '16 at 08:30
  • Sorry I missed those comments. Unfortunately I am not able to reproduce the problem... cfdir and cfdump show me the file name `L’État, c’est moi.pdf` properly. – Jules Sep 30 '16 at 03:53
0

I had the same problem this solved my bug
/.bashrc LC_ALL="de_DE.UTF-8" on linux and after change restart coldfusion application

DucaBgd
  • 276
  • 3
  • 4