Perl check if nonblocking file read/ write has finished

Question

I am learning about non blocking io in perl. I have the following script, where I am trying to write to a file in a non blocking fashion.

Is this the correct way to do it ? If not what is the correct way ? I am having trouble simulating a write that doesn't happen in an instant, so I can't really test it if it works or not.

use strict;
use warnings;

use POSIX qw(:errno_h);

open(my $fh, ">>", "b.txt") or die "$! \n";

my $buffer = "aaaaaaaaaaaaaaaaaaaaaaa\n";

my $rv = syswrite($fh, $buffer, length $buffer);
#start more non blocking operations
while(1)
{
    if (!defined($rv) && $! == EAGAIN) {
        print "Would block \n";
    } elsif ($rv != length $buffer) {
        print "Incomplete write \n";
    } else {
    print "Successfull write so we can do what we want with the read data\n";
    }
   #check the other operations

   #sleep for a bit
    sleep 1;
}

Minor side-note: `sleep` only allows integer values, so `sleep 0.005` is actually sleeping for 0 seconds. You can use `usleep` or `nanosleep` from [`Time::HiRes`](http://perldoc.perl.org/Time/HiRes.html) to sleep for a certain number of microseconds or nanoseconds, respectively. — ThisSuitIsBlackNot, Feb 20 '15 at 18:58
EAGAIN?? There is no filehandle named EAGAIN in your code. In any case, `$!` is a string. — 7stud, Feb 20 '15 at 19:43
@7stud `EAGAIN` is not a filehandle, it is a constant exported by `use POSIX qw(:errno_h);`. See [What does EAGAIN mean?](http://stackoverflow.com/q/4058368/176646) for details. — ThisSuitIsBlackNot, Feb 20 '15 at 19:53
@7stud As for `$!` being a string, see [`perldoc perlvar`](http://perldoc.perl.org/perlvar.html): *"When referenced, `$!` retrieves the current value of the C errno integer variable...When referenced as a string, `$!` yields the system error string corresponding to errno ."* — ThisSuitIsBlackNot, Feb 20 '15 at 20:00
@ThisSuitIsBlackNot, Yeah, I looked that up first, but there were 39 matches on the page. I didn't get far enough. Tricky perl. — 7stud, Feb 20 '15 at 20:15

Steffen Ullrich · Accepted Answer · 2015-02-20T20:18:13.017

First, non-blocking I/O works on sockets and pipes, but not on regular files. That is not a limit of Perl but of the underlying system. What you instead want for regular files would not be non-blocking but asynchronous I/O:

non-blocking I/O: operation will not block when it can not be finished immediately but it will do as much as it can and then return.
asynchronous I/O: the operation will be done in the background and the application will be notified once it is done.

Unfortunately support for real asynchronous I/O is system specific and often not well supported so you would need to use threads here (again, that's not a problem of Perl but of the underlying system). You might want to take a look at IO::AIO which tries to give you the feeling of asynchronous I/O by using whatever the system supports. But note also that it is not very efficient and if you could use non-blocking sockets directly (i.e. with sockets but not with regular files) you should do it.

To use non-blocking you have to mark the file descriptor as non-blocking and then you read, write, connect, accept etc on it like you do normally, except that it will never block but return EAGAIN/EWOULDBLOCK if the operation can not be executed immediately. To wait until a file descriptor becomes readable/writable again you can use select, poll, kqueue or similar mechanisms.

That's for a short introduction. Some things you have to watch out for:

Setting a file descriptor to non-blocking is different on UNIX and Windows. IO::Socket->blocking handles the major differences for you.
On todays UNIX EAGAIN is the same as EWOULDBLOCK but on Windows this is different and you have usually to use EWOULDBLOCK.
Again, non-blocking does not work on regular files. It might not throw an error but will simply block.
You have to watch out if you want to use non-blocking with file descriptors which do not reside entirely in the kernel (like with SSL sockets). Look at the documentation of IO::Socket::SSL for more information.

That is the best definition of the difference between non-blocking and asynchronous that I've seen, ever. — Stanimirovv, Feb 20 '15 at 20:30

score 2 · Answer 2 · answered Feb 20 '15 at 19:02

I'll go at a bit of a tangent - I'm not too familiar with doing nonblocking IO like that, because my approach tends to be to either thread or fork and do it that way.

Nonblocking writes to a file though, aren't all that common - it's usually not relevant when doing so with small files, so your test case probably isn't going to work. I'd suggest you aim for several megabytes of writes, just to be able to see it happen.

As for a threading approach - I tend to do it like this:

#!/usr/bin/perl

use strict;
use warnings;

use threads;
use Thread::Queue;

my $output_q = Thread::Queue->new();

sub writer {
    open( my $output_fh, ">", "output_file.txt" ) or die $!;
    while ( my $line = $output_q->dequeue ) {
        print {$output_fh} $line;
    }
    close($output_fh) or die $!;
}

## main bit.

my $writer = threads->create( \&writer );

my $done = 0;

while ( not $done ) {
    $output_q->enqueue( "a" x 1024 );

    # do something else.
    #repeat until $done

}

$output_q->end();
$writer->join();

First of all thank you for the answer. This is a pretty convenient approach. I am trying to do things in a single thread (for now). I will test with bigger files. — Stanimirovv, Feb 20 '15 at 19:05
There's no wrong answer, so whichever you prefer. Just bear in mind that filesystem writes will often go straight to cache, so the write operation will complete very quickly, until you start overrunning that cache. A 1s sleep after small number of bytes won't do that. You might find that `$buffer = "a" x 1048576;` is a useful construct :) — Sobrique, Feb 20 '15 at 19:06
Come now, perl is one language that got it right: it's `1_048_576`. — 7stud, Feb 20 '15 at 19:22

Perl check if nonblocking file read/ write has finished

2 Answers2