Turkish character not rendering properly

Question

I have a problem in Perl with a Turkish character. I have set the Turkish character set but my Turkish character isn't displaying properly.

This is my Perl script.

#!"c:\xampp\perl\bin\perl.exe"

use strict;
use warnings;

use CGI qw(:standard);

my $name    = param('name');
my $surname = param('surname');
my $age     = param('age');
my $gender  = param('gender');

my $q = new CGI;

# We're outputting plain text, not HTML
print $q->header(-content_type => 'text/plain' ,  -charset => 'ISO-8859-9');

my $text = $name." ".$surname." ".$age." ".$gender." kaydı, sistemde yapılacak olan güncellemelerden sonra sisteme başarıyla eklenecektir."; 

# printf("%s %s %d %d kaydı, sistemde yapılacak olan güncellemelerden sonra sisteme başarıyla eklenecektir.", $name , $surname , $age , $gender);

print $text;

How can I fix the problem?

What encoding does your code file use? Did you tell Perl about it? — choroba, May 02 '17 at 09:43
I didn't understand what excatly you say. But in perl code print $q->header(-content_type => 'text/plain' , -charset => 'ISO-8859-9'); you can see this code block. — mehfatitem, May 02 '17 at 09:47
Saying that you use `ISO-8859-9` is great but you also need to actually use it. Your editor will possibly have an "encoding" setting somewhere. — Álvaro González, May 02 '17 at 09:49
By default Perl source code is Latin-1. This means all string literals are Latin-1. — ceving, May 02 '17 at 10:15
I assume the character at fault is the `ş` in `başarıyla`? What are you getting instead? — Borodin, May 02 '17 at 10:24
@mehfatitem: Your comment is very ill-mannered. Please answer my questions. — Borodin, May 02 '17 at 11:10
@SinanÜnür: Thank you. I have never encountered Apache on Windows and wasn't aware of that. — Borodin, May 02 '17 at 11:14
@mehfatitem: I know you want to use ISO-8859-9, but how is your source file encoded? It is generally much more common to use UTF-8 as it is not language-specific and doesn't restrict you to 128 non-English letters, but you may have it set to save in ISO-8859-9. If you're unsure then please run this code `use Data::Dumper; $Data::Dumper::Useqq = 1; $Data::Dumper::Terse = 1; print Dumper $text;` and show the result. — Borodin, May 02 '17 at 11:21
@mehfatitem: And please answer my question about what output you are getting instead of the correct one. — Borodin, May 02 '17 at 11:24
It's not really your Perl program is it? It's mostly my Perl program :-) If you found my answer useful, please consider [accepting it](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work). If you didn't find it useful, please add a comment explaining why. — Dave Cross, May 02 '17 at 13:05
Let me rephrase it as a question. If you found [my answer to your previous question useful](http://stackoverflow.com/questions/43713053/a-simple-form-application-with-perl/) (and I assume you did, as you're using my code now) why haven't you accepted my answer? You don't, of course, have to accept answers. But if you make a habit of not accepting useful answers then people might be less inclined to help you. — Dave Cross, May 02 '17 at 13:17
Sorry , I only forgot to click accept button. Otherwise , your answer was very helpful. — mehfatitem, May 02 '17 at 13:26
@DaveCross Dou u have any idea about fixing my Turkish character problem ? — mehfatitem, May 02 '17 at 13:34
You need to [edit your question](http://stackoverflow.com/posts/43734497/edit) to add the answers to these two questions. 1/ Exactly what bad output do you see? 2/ What is the encoding of your Perl program file? Without that information, it is impossible to do anything but guess at an answer here. — Dave Cross, May 02 '17 at 13:37

Sinan Ünür · Answer 1 · 2017-05-02T11:33:07.010

First and foremost: Do not use the HTML generation functionality in CGI.pm. Using those leads to an unmaintainable mess of a presentation layer. Instead, use Template Toolkit based templates to separate presentation and functionality.

Second, do not use indirect object notation. That is, do not write:

my $cgi = new CGI;

Instead, write

my $cgi = CGI->new

The use of $q or $query to refer to the CGI object is weird and has it roots in the early days of the WWW. There is no reason to perpetuate it when you are learning things from scratch.

In addition, given that you have just instantiated an object, don't use plain subs such as param and don't pollute the namespace of your script. Access parameter values using:

 my $value = $cgi->param('surname');

Finally, if you are going to use "interesting" characters in your source code such as Ş, save your source code as UTF-8 and specify

 use utf8;

at the top of your script.

Finally, do also save all your HTML templates in UTF-8 and generate all output from your script encoded in UTF-8 and specify the document encoding as UTF-8. All other paths lead to insanity.

Also, don't use sexual as a parameter name. Use nouns as parameter and variable names. And, most infuriating to me as a Turk, Mr. (Bay) and Ms. (Bayan) are just titles and they are not appropriate choices for an input field asking about the sex of the respondent.

See also How can I deal with diverse gender identities in user profiles? That may not be on your radar in Turkey at the moment, but you will eventually encounter the issue.

Here is an untested script that might work for you:

#!"c:\xampp\perl\bin\perl.exe"

use utf8;
use strict;
use warnings;
use warnings  qw(FATAL utf8);

# Provides only object oriented interface
# without HTML generation cruft

use CGI::Simple;

run( CGI::Simple->new );

sub run {
    my $cgi = shift;
    binmode STDOUT, ':encoding(UTF-8)';

    my ($name, $surname, $age, $gender) = map $cgi->param($_), qw(name surname age gender);
    print $cgi->header(-type => 'text/plain', -charset => 'utf-8');

    printf("%s %s %d %d kaydı, sistemde yapılacak olan güncellemelerden sonra sisteme başarıyla eklenecektir.\n",
        $name , $surname , $age , $gender);

    return;
}

@mehfatitem: You have done an excellent job of alienating most people who would answer you. You've asked five questions and have a total of -4 votes. You're behaving like a surly nightclub bouncer, and it's hard to believe in you as a programmer. Perhaps try writing a new question and not being so unpleasant? If you want help then you really need to answer the questions that I've asked: what is the encoding of your Perl program file, and what are you seeing in the browser instead of what should be there? — Borodin, May 02 '17 at 20:35
I'm not a programmer firstly fix this fault. I'm a software engineer. I have been researching about perl for a few days.I think You must give more satisfying answer insted of criticising people. You are not a moral police officer. I asked a question about caharcter encoding about Turkish. You went away from the topic. Told that You must delete #!"c:\xampp\perl\bin\perl.exe" this code. But it is very far way from onur topic. And you told that the problem is about "ş" character. And I told you that ş is character in Turkish alphabe.You made the topic too exaggerated and show yourself as a victim — mehfatitem, May 02 '17 at 21:39
@victim Borodin and after this time , please stop adding comment if it wouldn't be about software. Because you steal of my time. — mehfatitem, May 02 '17 at 21:46
@mehfatitem: People like Borodin and I are trying to help you. But we can't help you unless you answer the questions that we ask. — Dave Cross, May 03 '17 at 07:25
@mehfatitem: I hope you take note of the best advice in my answer (use UTF-8) and don't just apply the hacky `encode()` solution. — Dave Cross, May 03 '17 at 11:42
@DaveCross: When I set header -charsert => "utf-8" instead of "ISO-8859-9" , it solved my problem. You are right :-) — mehfatitem, May 03 '17 at 11:55
@mehfatitem: I'm pretty sure that's working accidentally. You should add `use utf8` to your code and encode the output correctly. — Dave Cross, May 03 '17 at 12:07

score 1 · Answer 2 · edited May 03 '17 at 12:00

I think you are misunderstanding the purpose of the charset attribute on the Content-type header. Your program is emitting this header:

Content-Type: text/plain; charset=ISO-8859-9

This says to an HTTP client (a browser, for example) "I am going to send you some plain text which is encoded as ISO-8859-9". But it's important to note that the header is purely informational. It tells the world that your text is encoded as ISO-8859-9. It does not do the encoding for you. That is up to you.

This is why Borodin and others have been asking you questions which you have not been answering. Most editors will create text that is encoded either as ISO-8859-1 or UTF-8. Unless you have a special Turkish editor or you have changed the configuration of your editor, it seems very unlikely to me that you are producing text in ISO-8859-9.

If you are determined to emit ISO-8859-9 text, then you need to do that encoding yourself. You can use the encode() function from the Encode module to do that.

use Encode;

my $encoded_text = encode('ISO-8859-9', $text);
print $encoded_text;

But I wonder why you want to use a relatively obscure encoding like ISO-8859-9. UTF-8 covers all of the characters used in Turkish. Why not use that instead? Your life will become far easier if you embrace the same standards as the web.

As an aside, you have introduced a small strangeness in your code. You use CGI.pm in "functions" mode and load it in a way which imports a number of its functions into your namespace.

use CGI qw(:standard);

And then you use the param() function a few times in this way. But after that you create a CGI query object in order to call the header() method on it.

my $q = new CGI;
print $q->header(...);

You probably don't realise it, but the header() function is included in the :standard set of imports, so you can call it without creating a CGI object.

print header(...);

I used it that way in my answer to your previous question. I'm not sure why you changed the code to make it more complicated.

I should also point out that if you do want to create a CGI query object, then you shouldn't use the indirect object notation:

my $q = new CGI;

This will cause you problems at some point. Far better to write:

my $q = CGI->new;

(As demonstrated in the CGI.pm documentation)

Turkish character not rendering properly

2 Answers2