0

I'm working on a script parser that expands embedded Python code into script code by extracting all Python code into a single file so that it can remember Python variables defined earlier in the script. In order to substitute the Python output in the appropriate places in the script, it therefore cares about both Python's output, and the line number that produced this output. So I would like to generate Python code like this:

sys.stdout = ... # Write function reads _lineNumber and includes it in the output
_lineNumber = 1 # this line automatically inserted
print("qwerty") # from line 1
_lineNumber = 2 # this line automatically inserted
def foobar(x):
    print(x)
_lineNumber = 4
if True:
    foobar("fdsa") # from line 4
_lineNumber = 6
foobar("asdf") # from line 6

Note however that I have to skip lines 3 and 5 because they are not at the top-level context--changing _lineNumber during the function call would be incorrect behavior, and changing _lineNumber during the if would require matching the indentation and is unnecessarily complicated when I can simply require that the user complete all code blocks such as functions and conditionals in the same block of script code. My problem is ensuring that it is always valid to perform the assignment to _lineNumber when I do--the lines between the assignments are arbitrary code. My question is, if the following line has no indentation, does that guarantee that it is syntactically valid to insert an assignment to _lineNumber on the previous line, or is there any other python grammar that would make the scheme of inserting an assignment before every unindented line be invalid?

Vitruvie
  • 2,327
  • 18
  • 25
  • 1
    Why not rewrite `print` calls to include a line number argument, or use `inspect.stack()` and pull out the line number from the caller's stack frame? Also, `sys.stdout.write` probably isn't a good place to add the line numbers to the output, and if you're putting this file together from a bunch of code snippets embedded into something else, line numbers may not be that useful. – user2357112 Dec 14 '15 at 02:05
  • @user2357112 Essentially, the line number tells me which code snippet the output came from, so that I can put the output in place of that snippet. Adding an argument onto print would force redundancy onto the user. The actual line number isn't that important; just the correspondence between snippets and outputs. What would be a better place to add this information than `sys.stdout.write`? – Vitruvie Dec 14 '15 at 02:10
  • No, I mean why don't *you* rewrite `print` calls to include a line number? Too complex a parsing job? Also, if the line number is supposed to indicate which snippet a particular bit of output came from, is it really a line number, or a snippet number? If it's a snippet number, that eliminates a lot of problematic cases, like line continuations and multi-line string literals, neither of which should cross snippets. – user2357112 Dec 14 '15 at 02:25
  • @user2357112 I don't want to rewrite the print call because they might call print by any means, including calling another function, and I want to get the number of the top-level caller of the function(s), not of the function(s) being called. It is really a snippet number; however most snippets are one line long--line continuations may cross between snippets. (Not all snippets output a value). – Vitruvie Dec 14 '15 at 02:40
  • Well, if you're going to allow foo = \ in one snippet and then 'bar' in the next, this approach is pretty much screwed. – user2357112 Dec 14 '15 at 02:45
  • @user2357112 Ah, thank you. So part of the answer would be "you have to also check that the previous line does not end in an odd number of backslashes." It's acceptable that all output from line continuations would be as if it came from the snippet that started the line, just like how output from `if True:\n foobar("fdsa")` comes from the `if True:` snippet. – Vitruvie Dec 14 '15 at 02:51
  • 1
    That's not going to cut it. You'd still need to deal with stuff like `foo = [`, `1,`, `2`, and `]` on consecutive lines, and multi-line string literals, and not injecting assignments before an `else`, `except`, or `finally`. The `ast` module would be pretty much mandatory for handling the parsing. – user2357112 Dec 14 '15 at 03:01

1 Answers1

0

No; while indentation at the start of the line can tell you the the depth of context of that statement, it is still possible for many things such as lists to cause one line to span multiple code lines, and other statements such as else can be connected to earlier statements despite having no indentation. There's no simple way to do this; short of getting into the python parsing, you'd have to rely on the user to return to the top level context at well-defined points.

Vitruvie
  • 2,327
  • 18
  • 25