I was thinking more about this today. Ideally, there's a tool out there to do this, but if not, I think this might work, depending on how much it is worth to you to script it:
Comment-preserving diff algorithm:
1 . For file1 and file2, process them and create 2 new files for each:
i. A version of each file with the comments removed, (file1.py.nocom).
Lines containing only a comment would not be removed. Just the comment
removed. The line numbering would need to stay the same.
ii. A file containing the locations for all the comments as well as the
actual comment text. Something like:
1,1:# File 1 code
4,15:# comment 1
2. Do the diff between file1.py.nocom and file1.py.nocom, but without the -y
flag. This will be easier to parse. Even easier, use the -c flag with a
really high value. Hopefully you can get the whole file in the diff
without any missing "common" lines that way.
3. Go through the output from #2 and add back in the comments using the info
from 1.ii. I experimented with manually editing the diff from #2 and
applying it with vim, but it didn't seem to like one of the "common" lines
having a comment change. But there may be some tool that will allow you to
view it. Barring that:
4. Use the commented diff output to recreate yourself the -y flag style
output. I guess the tricky part will be determining the width of the
left side and printing out the right column. If on #2 you weren't able
to get all the common lines into the diff output using the -c flag, then
here you'll have to re-add those missing common lines.
The above won't (easily) work with docstrings, and there are probably other cases I haven't thought of. I guess it might need to be tweaked if you have additional/removal of comment lines between files as well. But there's my two cents. It seems doable, but definitely a chunk of work.