How can you use timeit with commands containing EOL characters?

Question

I'm attempting to learn how to use timeit to time command line arguments driven by python.

from subprocess import *
import timeit

p = Popen( ["cmd.exe"], stdin=PIPE, stdout=PIPE )
dir_time = timeit.timeit('p.stdin.write("dir\n"),p.stdin.write("exit\n")')
print p.stdout.read()

Unfortunately, when I run it I get
"exceptions.SyntaxError: EOL while scanning string literal (line 6, offset 26): 'p.stdin.write(""dir'"

I've seen a similar question about EOL syntax error when using timeit and cmd and it's solution was replacing single quotation marks however I'm not feeding code snippets directly into cmd, their commands never contain an end line character in it and double quotations produce a syntax error.

Is there any way to delimit the end of line character but still be valid in cmd? Have I used the incorrect syntax for two arguments in timeit?

Wouldn't that time how long it took to send the commands to the shell, rather than how long it took the commands to run? — Harry Johnston, Apr 02 '15 at 04:16
I assume that it has to wait for the stdout to arrive from the first command before it sends the exit but that's just my logical as opposed to technical understanding — Tesenka, Apr 02 '15 at 04:34
There isn't really any way for it to know how much output to expect. — Harry Johnston, Apr 02 '15 at 04:45
I guess EOL is posed by `\n`so you should escape `\ ` in `p.stdin.write("dir\n")` etc. as follows: `p.stdin.write("dir\\n")` — JosefZ, Apr 02 '15 at 07:32

score 1 · Accepted Answer · edited Mar 20 '17 at 10:29

To fix the immediate problem with EOL, you could use raw string literals: r'p.stdin.write("dir\n"),..' (notice: r'') otherwise \n is interpreted as a newline that leads to the error you see:

>>> import timeit
>>> timeit.timeit('"a" + "\n"')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/timeit.py", line 230, in timeit
    return Timer(stmt, setup, timer).timeit(number)
  File "/usr/lib/python2.7/timeit.py", line 136, in __init__
    code = compile(src, dummy_src_name, "exec")
  File "<timeit-src>", line 6
    "a" + "
          ^
SyntaxError: EOL while scanning string literal

vs.

>>> timeit.timeit(r'"a" + "\n"')
0.021812915802001953

The more general issue is that:

timeit(r'p.stdin.write("dir\n"),p.stdin.write("exit\n")') won't work

p probably is not accessible (NameError)
even if it had worked it would not measure the time it takes to run dir command in cmd.exe

It would measure how long it takes to write the command to an internal python file buffer and/or push it through the pipe to cmd.exe
or it would raise IOError/OSError

the pipe is probably closed after exit is executed for the first time. timeit runs the code multiple times (1000000 by default).

It looks like XY problem. If you want to know how to measure the time it takes to run an internal command in cmd.exe then ask about it directly as a new question.

timeit is not a suitable tool here.

How can you use timeit with commands containing EOL characters?

1 Answers1