Perl Split Binary Data into Chunks with Unpack

Question

I'm using Transfer-Encoding: chunked to write an HTTP response.

The response is split into pieces via the following:

my $template = "a$buffer_size" x int(length($response)/$buffer_size) . 'a*';

foreach my $buffer (unpack $template, $response){
    ...
}

This works fine when the content type is text/html, but it is corrupting binary data, such as application/pdf.

Can unpack be used to split binary data into equal lengths?

When you say that the content type is causing the corruption, have you confirmed that by trying to access the same data, send with different MIME types? — Dancrumb, Jun 01 '12 at 12:58
@Dancrumb the "corruption" is that the file is incorrectly flagged as UTF-8 and not ANSI. Strange thing is if I simulate `unpack` with `grep {/\S/} split /(.{$buffer_size})/`, everything is fine. Also, the `unpack` logic is OK if done at the command line (i.e., not via mod_perl or ActiveState PerlEx). — xpsd300, Jun 01 '12 at 13:43
[Edit your question](http://stackoverflow.com/posts/10850622/edit) to provide your [full code and input data](http://sscce.org). When you just paraphrase parts of it in English prose, it is impossible to help you because we are not able to [reproduce the problem](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html#showmehow). — daxim, Jun 01 '12 at 14:02
Using the Plack streaming interface for chunked encoding: http://codepad.org/531qBASm — daxim, Jun 01 '12 at 14:36

score 1 · Accepted Answer · answered Jun 01 '12 at 19:45

1

Still not sure why unpack is failing in this context, but I stumbled upon a solution.

If I manipulate the response with an in-memory file, unpack works correctly:

my $resp;
open (my $fh, '>', \$resp);
my $fh_old = select($fh);
print $response;
close $fh;
select($fh_old);
$response = $resp;

Any insight into why this works?

answered Jun 01 '12 at 19:45

xpsd300

1

Finally tracked down the root of the problem. The PDF file was being created on-the-fly from a unicode database. I was converting wide characters into `cp1252` to match `WinAnsiEncoding` of the font object. Perl switched into `utf8` mode when it encountered a wide character. This caused `unpack` to fail since it was set to operate in `ascii` mode. – xpsd300 Jun 02 '12 at 15:48

score 0 · Answer 2 · answered Jun 01 '12 at 16:32

0

That works perfectly fine with binary data. The problem is elsewhere. (Did you binmode all relevant handles?)

answered Jun 01 '12 at 16:32

ikegami

2 Answers2