String to UTF-16 using perl

Question

I need to convert my message string to UTF-16 using Perl.

Example Convert:

Dear User, Please enter your password to verify your mobile no.

into :

0044 0065 0061 0072 0020 0055 0073 0065 0072 002C 0020 0050 006C 0065 0061 0073 0065 0020 0065 006E 0074 0065 0072 0020 0079 006F 0075 0072 0020 0070 0061 0073 0073 0077 006F 0072 0064 0020 0074 006F 0020 0076 0065 0072 0069 0066 0079 0020 0079 006F 0075 0072 0020 006D 006F 0062 0069 006C 0065 0020 006E 006F 0020 002E

as http://macchiato.com/unicode/convert.html does . paste the message into input and select UTF-16M in output . what the output you get i want that using perl script.. — kanishka, Jul 04 '16 at 06:09
This may interest you: http://stackoverflow.com/questions/15449469/create-utf-16-string-from-char — Otterbein, Jul 04 '16 at 06:14
The above is a bit simple case, as 7 bits ASCII just gets a byte 00 before (UTF-16 Big Endian) or after (Little Endian). If the code text too is non-ASCII, say French, special quotes, then you need to know the source encoding, to encode from. — Joop Eggen, Jul 04 '16 at 08:45
It's unclear what encoding you want (UTF-16le or UTF-16be) since you provided the output in an ambiguous format. Please provide the desired output as *bytes* (their hex representation is ok) — ikegami, Jul 04 '16 at 16:54

score 2 · Answer 1 · answered Jul 04 '16 at 06:29

Just specify the output encoding for the handle you output to:

#! /usr/bin/perl
use warnings;
use strict;

my $input = 'Dear User, Please enter your password to verify your mobile no .';
binmode *STDOUT, 'encoding(UTF-16)';
print $input;

Test:

$ 1.pl | xxd
0000000: feff 0044 0065 0061 0072 0020 0055 0073  ...D.e.a.r. .U.s
0000010: 0065 0072 002c 0020 0050 006c 0065 0061  .e.r.,. .P.l.e.a
0000020: 0073 0065 0020 0065 006e 0074 0065 0072  .s.e. .e.n.t.e.r
0000030: 0020 0079 006f 0075 0072 0020 0070 0061  . .y.o.u.r. .p.a
0000040: 0073 0073 0077 006f 0072 0064 0020 0074  .s.s.w.o.r.d. .t
0000050: 006f 0020 0076 0065 0072 0069 0066 0079  .o. .v.e.r.i.f.y
0000060: 0020 0079 006f 0075 0072 0020 006d 006f  . .y.o.u.r. .m.o
0000070: 0062 0069 006c 0065 0020 006e 006f 0020  .b.i.l.e. .n.o. 
0000080: 002e                                     ..

If you want to avoid the BOM (The feff at the beginning), specify the endianness (UTF-16LE or UTF16-BE).

score 1 · Answer 2 · edited May 23 '17 at 10:28

If this is for Windows then you will want UTF-16LE

If you need to print encoded data to an output file, then by far the best way is to work with characters within your program, and set the encoding for the output handle as choroba has described

However, if you need a UTF-16-encoded string for any other reason, such as to name a file or to store in the clipboard, then you will need to do the encoding separately

The Encode module will help you here, and this is a sample program to demonstrate. I have unpacked and the encoded string as a series of unsigned 16-bit values so that you can see that the contents match what you expect

use strict;
use warnings 'all';

use Encode qw/ encode :fallbacks /;


my $s = 'Dear User, Please enter your password to verify your mobile no.';

my $encoded = encode( 'UTF-16LE', $s, FB_CROAK);

print join ' ', map { sprintf '%04X', $_ } unpack 'S*', $encoded;

output

0044 0065 0061 0072 0020 0055 0073 0065 0072 002C 0020 0050 006C 0065 0061 0073 0065 0020 0065 006E 0074 0065 0072 0020 0079 006F 0075 0072 0020 0070 0061 0073 0073 0077 006F 0072 0064 0020 0074 006F 0020 0076 0065 0072 0069 0066 0079 0020 0079 006F 0075 0072 0020 006D 006F 0062 0069 006C 0065 0020 006E 006F 002E

score 0 · Answer 3 · answered Jul 05 '16 at 07:31

For me this worked

#! /usr/bin/perl
use warnings;
use strict;
use Encode qw(encode);

my $message = 'Dear User, Please enter your password to verify your mobile no.';
my $message_hex=unpack "H*",encode( 'utf-16be', $message );
print $message_hex ;

OUTPUT : 004400650061007200200055007300650072002c00200050006c006500610073006500200065006e00740065007200200079006f00750072002000700061007300730077006f0072006400200074006f002000760065007200690066007900200079006f007500720020006d006f00620069006c00650020006e006f002e

String to UTF-16 using perl

3 Answers3

output