tcl : Execute a windows command line and see the "flow" of the application

Question

I have tried the following code :

set my_cmd "|tracert google.com"

puts "Now executing the command '$my_cmd':\n"

set f [open $my_cmd "r"]

#process command output
while {[gets $f line] != -1} {
  # print line
  puts $line
}

close $f

and it works for tracert I see the program running flow line by line, but with my application (the one runs in the attached image), it's wait for the application to finish, only then shows all the output. Tried several ways also with "exec" and no luck.

it is only works if i do as so : enter image description here

What tcl code can make it work ?

You've got two levels of stdout buffering to deal with. The application you are executing may not flush its buffer, so your program may not see its data line-by-line -- not much you can do about this, and your program above is also buffering the output (add a flush as in wolfhammer's answer). — Brad Lanam, Apr 02 '15 at 17:17

score 2 · Answer 1 · edited May 23 '17 at 12:13

When a subprocess writes to a pipe (or a file) as opposed to the terminal, it is usually the case that the output is buffered; the output is only written once it has accumulated to a certain point (often 4kB) or when the output channel is closed or the program exits (both of which will trigger a flush of everything that remains). If the program's behaviour is to slowly trickle a few bytes out, it will seem as if the program has hung. It hasn't; it's just that you're waiting for the program to do its magic.

Why doesn't tracert have this behaviour? Simple: it's either explicitly flushing its internal buffers after every line, or not buffering at all and just writing the bytes directly as it has them. When a program is producing a lot of output that's slower (because it ends up doing a lot more system calls) but when there's not much output it's not a Big Deal in the first place. In short, tracert isn't a good model for the program you've got a problem with.

I notice from your screen capture that the real problem may well be with a Python program that is buffering its output. There are a number of techniques on dealing with this described in this Stack Overflow question: Disable output buffering. I would try setting the environment variable first, as that's easy to do from Tcl:

set ::env(PYTHONUNBUFFERED) yes

However, in general there are other ways to tackle this from the Tcl side. The main one to mention is to use the Tcl extension, Expect. That does some evil OS hackery behind the scenes to make it appear as if the program is running in interactive mode, triggering different buffering behaviour.

package require Expect

set my_cmd "tracert google.com"
puts "Now executing the command '$my_cmd':"

spawn {*}$my_cmd

#process command output
expect {
    -re {([^\r\n]+)[\r\n]} { ### Match a whole line
        set line $expect_out(1,string)
        ### This is where your code from inside your loop goes
        puts $line
        ### Match the next line
        exp_continue
    }
    eof { ### Special clause for End-Of-File; which is effectively empty
    }
}

close

That's a very simple use of Expect (it can also interact with the subprogram on your behalf) but it is a basic translation of your program.

This Expect code crashes after first line printed, I think in the "exp_continue" instruction. — Sion C, Apr 05 '15 at 08:04
I am having troube with "\r" since it closes the application in the middle, and the new line in you example is not working it is crashing, can you match me a "new line" regex ? tnx — Sion C, Apr 05 '15 at 16:12

score 0 · Answer 2 · answered Apr 02 '15 at 17:08

Here's a program that allows you to capture the output line by line. I'm using a mac so it's "traceroute" instead of "tracert."

trace.tcl

#!/usr/bin/tclsh

proc FollowTrace fd {
  if {[gets $fd line] < 0} {
    # there was problem getting the next line
    set code [catch {close $fd} err]

    if {$code == 0} {
      set ::result 0
    } else {
      puts stderr $err
      set ::result 1
    }

    return
  }

  # print data to standard output
  puts stdout "MYOUTPUT:[clock seconds] - $line"

  # flush standard output
  flush stdout
  return
}

# capture command output
set fd [open [list | traceroute google.com 2>@1]]

# Wireup the standard input handler.
fileevent $fd readable [list FollowTrace $fd] ;#localHandler

# Keep running until ctrl-c
vwait __forever__

output:

./trace.tcl
MYOUTPUT:1427994488 - traceroute: Warning: google.com has multiple addresses; using 173.194.33.137
MYOUTPUT:1427994488 - traceroute to google.com (173.194.33.137), 64 hops max, 52 byte packets

tcl : Execute a windows command line and see the "flow" of the application

2 Answers2