How to use utf8 encode with open pragma

Question

I have problem with utf8::encode when use pragma use open qw(:std :utf8);

Example

#!/usr/bin/env perl

use v5.16;
use utf8;
use open qw(:std :utf8);

use Data::Dumper;

my $word = "+банк";
say Dumper($word);
say utf8::is_utf8($word) ? 1 : 0;

utf8::encode($word);
say Dumper($word);
say utf8::is_utf8($word) ? 1 : 0;

Output

$VAR1 = "+\x{431}\x{430}\x{43d}\x{43a}";
1
$VAR1 = '+Ð±Ð°Ð½Ðº';
0

When I remove this pragma use open qw(:std :utf8);, everything is OK.

$VAR1 = "+\x{431}\x{430}\x{43d}\x{43a}";
1
$VAR1 = '+банк';
0

Thank you in advanced!

Using `:std :utf8` adds an UTF-8 layer to `STDOUT`. Therefore the last `say` will try to encode `$word` once more.. — Håkon Hægland, Jul 09 '15 at 13:21

score 6 · Accepted Answer · answered May 10 '13 at 14:09

6

If you're going to replace utf8::encode($word); with use open qw(:std :utf8);, you'll actually need to remove the utf8::encode($word);. In the version that doesn't work, you're encoding twice.

answered May 10 '13 at 14:09

ikegami

367,544
15
269
518

score 2 · Answer 2 · answered May 10 '13 at 16:20

utf8::encode is not what you want if you are going to print to a filehandle upon which perl expects to output utf8.

utf8::encode says take this string and give me a string where each character is a byte of the utf8 encoding of the input string. This would normally be only done if you are then going to use that string in some way where perl won't be automatically converting to utf8 if necessary.

If you add a say length($word); after the encode, you will see that $word is 9 characters, not the original 5.

How to use utf8 encode with open pragma

2 Answers2

Linked