Simulating streaming data

Question

I'm trying to write a program that will read from a flat file of data and simulate streaming it so I can test a program that reads streaming data without having to connect and start up the streaming hardware.

What are the more realistic ways to accomplish this? I need it to stream at variable speeds depending on the hardware im simulating.

My two ideas so far are a program that writes to a named pipe, or a program that writes to a virtual serial port at the rates I need.

Is there a better (more realistic) way of simulating streaming data?

Answer 1

Set up a background process that writes the file to a socket at whatever rate you want. In Perl,

use Socket;
socketpair my $A, my $B, AF_UNIX, SOCK_STREAM, PF_UNSPEC;

if (fork() == 0) {
    stream();
    exit;
}

while (<$A>) {
    print;
}

sub stream {
    # output 1024 bytes/sec
    select $B; $| = 1;          # disable output buffering
    open my $fh, '<', '/file/to/stream';
    my $buffer;
    while (my $n = read $fh, $buffer, 1024) {
        sleep 1;
        print $B $buffer;
    }
    close $fh;
}

Answer 2

I'd go with simulating the stream in a way that is closest to the original source of stream data. E.g. if you normally get your stream from a network socket, then use a network socket to simulate; if you have a serial port, then use a virtual serial port, etc. Otherwise, you will need an adapter for your stream consumer code (but you already know all this I presume).

Other than that, you'll need a rate limiting algorithm that will allow you to control how much data to send. A quick search led me to this but there are surely more complex and sophisticated algorithms. YMMV.

This assumes of course that you know how to create valid data/packets for the consumer of your stream.

Answer 3

You might try saving some coding time by looking at trickle (a userspace bandwidth shaper).