0

I created a simple asyncssh app that starts a process that runs continuously. I would like to get all stdout from the app as it runs but for some reason it ends part ways. I'm not sure if this means there is a limit of how much I can get from stdout. The code is similar to the following:

async def watch_process(command): 
   async with asyncssh.connect(host=host, user=user, pass=pass) as conn: 
      async with conn.create_process(command) as proc: 
         async for line in proc.stdout: 
            printf(f"{host}: {line}")

asyncio.run(watch_process(command))

UPDATE1: I just took a look at the asyncssh email list and it appears the above code is correct, so I'm a bit perplexed that my process prints out only a portion of the lines and then hangs. https://groups.google.com/g/asyncssh-users/c/DiwRpZpYQjM/m/5uklhe01AwAJ

UPDATE2: After letting the application run for a few hours with asyncssh, I got console output every X# of lines. It seems like it had to fill a buffer past a threshold before it would print out. Is there a theshold setting in SSH that I need to configure so that it prints out each line instead of every X# of lines?

LeanMan
  • 474
  • 1
  • 4
  • 18
  • Seems odd. I ran up a https://stackoverflow.com/help/minimal-reproducible-example using ```ping localhost``` as the command, and fixed the arguments you used for connect. ```line in proc.stdout``` clears the receiving buffer fine. I wonder if its something else in your code that is the issue as your snippet won't run. – jwal Oct 29 '21 at 21:17
  • Yea I can work on my MPE and try creating something similar that will run and produce the same behavior. Interesting you were able to get it to work. I was running my example within a conda environment and through TMUX. – LeanMan Oct 30 '21 at 01:13
  • I did little more than fix the username and password and used one of my debian dev VMs to use as a server. I expected proc.stdout to drain the local buffer fast, which it did. cpython 3.9 on my system. – jwal Oct 30 '21 at 01:54
  • So I did not change my code much and changed the command to `ping 8.8.8.8` to compare to your result. And it seemed fine to print every ping so I thought perhaps I need to have something run continuously. So I ran another command to increment a counter every second in bash. That ran a little over 3 hours with a count of 11593 => asyncssh good! So this leads to my app and how it outputs data. I tried to mimic what it does (print out bursts of data) and cat a bigfile (10 GB) that had lines filled from 0 to 10M (worked fine). Makes me wonder if it is something to do with my stderr and stdout...? – LeanMan Oct 30 '21 at 22:23
  • I think I may have figured out the problem. Unix by default doesn't flush to file on each printf during program execution. It fills up a buffer and then flushes to file. Prints to terminal flushes on each new line. Ping and other utilities probably flush on each print so that's why they work. But in my case, my program might be buffering before printing to a file and therefore asyncssh doesn't show anything until that action occurs. – LeanMan Oct 31 '21 at 16:25
  • What an ordeal... following this SO Post solved the problem regarding this issue: https://stackoverflow.com/questions/1716296/why-does-printf-not-flush-after-the-call-unless-a-newline-is-in-the-format-strin – LeanMan Oct 31 '21 at 16:53

0 Answers0