How to run tshark in Java to get packets in real-time?

Question

I have a problem with running tshark in Java. It seems that packets arrive in bulk instead of truly real-time (as it happens when run from terminal). I tried a few different approaches:

ArrayList<String> command = new ArrayList<String>();
command.add("C:\\Program Files\\Wireshark\\tshark.exe");
ProcessBuilder pb = new ProcessBuilder(command);
Process process = pb.start();

BufferedReader br = null;
try {
    //tried different numbers for BufferedReader's last parameter
    br = new BufferedReader(new InputStreamReader(process.getInputStream()), 1);
    String line = null;
    while ((line = br.readLine()) != null) {
        System.out.println(line);
    }
} catch...

also tried using InputStream's available() method as seen in What does InputStream.available() do in Java?

I also tried NuProcess library with the following code:

NuProcessBuilder pb = new NuProcessBuilder(command);
ProcessHandler processHandler = new ProcessHandler();
pb.setProcessListener(processHandler);
NuProcess process = pb.start();
try {
    process.waitFor(0, TimeUnit.SECONDS);
} catch (InterruptedException e) {
    e.printStackTrace();
}

private class ProcessHandler extends NuAbstractProcessHandler {
    private NuProcess nuProcess;

    @Override
    public void onStart(NuProcess nuProcess) {
        this.nuProcess = nuProcess;
    }

    @Override
    public void onStdout(ByteBuffer buffer) {
        if (buffer == null)
            return;

        byte[] bytes = new byte[buffer.remaining()];
        buffer.get(bytes);
        System.out.println(new String(bytes));
    }
}

None of the methods work. Packets always arrive, as if buffered, only when about 50 were sniffed.

Do you have any idea why this may be happening and how to solve it? It's pretty frustrating. I spent a lot of time looking at similar questions at SO, but none of them helped.

Do you see any errors in my code? Is it working in your case?

Answer 1

As the tshark man page says:

   −l  Flush the standard output after the information for each packet is
       printed.  (This is not, strictly speaking, line‐buffered if −V was
       specified; however, it is the same as line‐buffered if −V wasn’t
       specified, as only one line is printed for each packet, and, as −l
       is normally used when piping a live capture to a program or script,
       so that output for a packet shows up as soon as the packet is seen
       and dissected, it should work just as well as true line‐buffering.
       We do this as a workaround for a deficiency in the Microsoft Visual
       C++ C library.)

       This may be useful when piping the output of TShark to another
       program, as it means that the program to which the output is piped
       will see the dissected data for a packet as soon as TShark sees the
       packet and generates that output, rather than seeing it only when
       the standard output buffer containing that data fills up.

Try running tshark with the -l command-line argument.

Answer 2

I ran some tests to see how much Buffering would be done by BufferedReader versus just using the input stream.

        ProcessBuilder pb = new ProcessBuilder("ls", "-lR", "/");

        System.out.println("pb.command() = " + pb.command());

        Process p = pb.start();
        byte ba[] = new byte[100];
        InputStream is = p.getInputStream();
        int bytecountsraw[] = new int[10000];
        long timesraw[] = new long[10000];
        long last_time = System.nanoTime();
        for (int i = 0; i < timesraw.length; i++) {
            int bytecount = is.read(ba);
            long time = System.nanoTime();
            timesraw[i] = time - last_time;
            last_time = time;
            bytecountsraw[i] = bytecount;
        }
        try (PrintWriter pw = new PrintWriter(new FileWriter("dataraw.csv"))) {
            pw.println("bytecount,time");
            for (int i = 0; i < timesraw.length; i++) {
                pw.println(bytecountsraw[i] + "," + timesraw[i] * 1.0E-9);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        BufferedReader br = new BufferedReader(new InputStreamReader(is));
        int bytecountsbuffered[] = new int[10000];
        long timesbuffered[] = new long[10000];
        last_time = System.nanoTime();
        for (int i = 0; i < timesbuffered.length; i++) {
            String str = br.readLine();
            int bytecount = str.length();
            long time = System.nanoTime();
            timesbuffered[i] = time - last_time;
            last_time = time;
            bytecountsbuffered[i] = bytecount;
        }
        try (PrintWriter pw = new PrintWriter(new FileWriter("databuffered.csv"))) {
            pw.println("bytecount,time");
            for (int i = 0; i < timesbuffered.length; i++) {
                pw.println(bytecountsbuffered[i] + "," + timesbuffered[i] * 1.0E-9);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }

I tried to find a command that would just keep printing as fast as it could so that any delays would be due to the buffering and/or ProcessBuilder rather than in the command itself. Here is a plot of the result.

You can plot the csv files with excel although I used a Netbeans plugin called DebugPlot. There wasn't a great deal of difference between the raw and the buffered. Both were bursty with majority of reads taking less than a microsecond separated by peaks of 10 to 50 milliseconds. The scale of the plot is in nanoseconds so the top of 5E7 is 50 milliseconds or 0.05 seconds. If you test and get similar results perhaps it is the best process builder can do. If you get dramatically worse results with tshark than other commands, perhaps there is an option to tshark or the packets themselves are coming in bursts.