Read stdin line by line

Question

I need to pipe a command into a batch file and do some processing, while preserving the output of the original command. So that for example on running the following command, the output would still be as if there was no piping at all:

ping 127.0.0.1 -n 4 | my_process

The best workaround I have found so far is https://stackoverflow.com/a/6980605/6094503. But my problem is that I need the output line by line. Using that solution the output is flushed only after the ping command is done executing. I found https://stackoverflow.com/a/21567535/6094503 which says it is because of the in (inside the for in loop).

This is a line by line example:

ping 127.0.0.1 -n 4 | findstr $

Actually if Windows was an open source project we could probably find the answer inside findstr or similar commands.

Well, in the [answer you linked](https://stackoverflow.com/a/6980605), `for /F` is used, which processes the text line by line, because the loop body is executed per each line of incoming text, so I do not really get what your problem is… — aschipfl, Aug 03 '21 at 13:06
@aschipfl: Navid's problem is `the output is flushed only after the ping command is done executing` in other words, the output is not in real time. — Stephan, Aug 03 '21 at 13:10
Oh, I see, @Stephan, thanks for clarification; not sure if there is a possibility with batch scripting since `set /P` cannot be used in pipes, and `for /F` awaits the whole text before continuing… — aschipfl, Aug 03 '21 at 13:13
@aschipfl But you can flush a pipe with `findstr` into a file and use `set /p` to read from that file — jeb, Aug 03 '21 at 13:20

jeb · Accepted Answer · 2021-08-03T14:16:14.697

You can flush the pipe line by line with more into a file and with a second cmd.exe instance read from that file.

@echo off
REM *** This is a trampoline to jump to a function when a child process shall be invoked
for /F "tokens=3 delims=:" %%L in ("%~0") do goto %%L

setlocal DisableDelayedExpansion

break > pipe.tmp
REM *** Create a new cmd.exe process, and calling :async in this batch file, uses the trampoline
start "" /b "%~d0\:async:\..%~pnx0"
( 
    more
    echo END
) >> pipe.tmp

REM Wait for a clean exit of the async thread
ping localhost -n 2 > nul
echo END
exit /b

:async
echo async

set lineCnt=0
< pipe.tmp (
    for /L %%n in ( infinite ) do (
    set "line="
    set /p line=
    if defined line (
        set /a lineCnt+=1
        setlocal EnableDelayedExpansion
        if "!line:~0,3!" == "END" (
            exit
        )
        echo( READ[!lineCnt!]: !line!
        endlocal
    )
    )
)

findstr fails to read from a pipe and store the output into a file asynchronously. But it works when reading from a file, but then you need two asynchronous processes.

@echo off
REM *** This is a trampoline to jump to a function when a child process shall be invoked
for /F "tokens=3 delims=:" %%L in ("%~0") do goto %%L

setlocal DisableDelayedExpansion

break > pipe1.tmp
break > pipe2.tmp
REM *** piperun.tmp is used as a signal for :async1 to detect when to stop the infinite loop
break > piperun.tmp

REM *** Create a new cmd.exe process, and calling :async1 in this batch file, uses the trampoline
start "" /b "%~d0\:async1:\..%~pnx0"
start "" /b "%~d0\:async2:\..%~pnx0"

more >> pipe1.tmp
del piperun.tmp

REM Wait for a clean exit of the async thread
ping localhost -n 2 > nul
del pipe1.tmp
del pipe2.tmp

echo END
exit /b


:async1
< pipe1.tmp > pipe2.tmp (
    for /L %%n in ( infinite ) do (
        findstr /n "^"
        if not exist piperun.tmp (
            REM *** The "raw" END is the signal for :async2 to stop the infinite loop
            echo END
            @REM echo EXIT %0 > CON
            exit
        )
    )
)
exit /b

:async2
set lineCnt=0
< pipe2.tmp (
    for /L %%n in ( infinite ) do (
    set "line="
    set /p line=
    if defined line (
        set /a lineCnt+=1
        setlocal EnableDelayedExpansion
        if "!line:~0,3!" == "END" (
            @REM echo EXIT %0 > CON
            exit
        )
        @REM set "line=!line:*:=!"
        echo( READ[!lineCnt!]: !line!
        endlocal
    )
    )
)
exit /b

I needed a minute or three to fully appreciate (the beauty of) `"%~d0\:async:\..%~pnx0"`. — Stephan, Aug 03 '21 at 14:08
@Stephan Okay, I added some comments and a second solution which can handle also empty lines — jeb, Aug 03 '21 at 14:17
@jeb Thank you for your answer, I will check it out. But does it really have to be this much painful? — navid, Aug 04 '21 at 05:59

Read stdin line by line

1 Answers1

Linked