Tracking changes in python source files?

Question

I'm learning python and came into a situation where I need to change the behvaviour of a function. I'm initially a java programmer so in the Java world a change in a function would let Eclipse shows that a lot of source files in Java has errors. That way I can know which files need to get modified. But how would one do such a thing in python considering there are no types?! I'm using TextMate2 for python coding.

Currently I'm doing the brute-force way. Opening every python script file and check where I'm using that function and then modify. But I'm sure this is not the way to deal with large projects!!!

Edit: as an example I define a class called Graph in a python script file. Graph has two objects variables. I created many objects (each with different name!!!) of this class in many script files and then decided that I want to change the name of the object variables! Now I'm going through each file and reading my code again in order to change the names again :(. PLEASE help!

Example: File A has objects x,y,z of class C. File B has objects xx,yy,zz of class C. Class C has two instance variables names that should be changed Foo to Poo and Foo1 to Poo1. Also consider many files like A and B. What would you do to solve this? Are you serisouly going to open each file and search for x,y,z,xx,yy,zz and then change the names individually?!!!

You start your application and see what goes wrong, or you use a profiler/syntax check to do the checks for you? Or you keep track of your code yourself, no one understands your code better than you.. Also what you're describing is not a Java or Python thing, it's an IDE thing. It's the IDE that keeps track of the changes and presents them to you. You could achieve the same thing by using Eclipse for Python i guess? — Torxed, Mar 19 '14 at 07:19
@Torxed: it sounds like a Java thing, because Java polymorphism on methods sucks: we need to write like 99 signatures for the same method. Instead of the Python way: define the method once, with default parameters, then figure out the correct thing to do based on which type/class the arg(s) passed were. — smci, Mar 20 '14 at 10:37
related: [What refactoring tools do you use for Python?](http://stackoverflow.com/q/28796/4279) — jfs, Mar 20 '14 at 10:38
My reaction to your code example: you shouldn't be referencing a member variable directly from client code (i.e. Files A,B). They should call a [getter/setter method on the object, or to be more Pythonic, a property](http://stackoverflow.com/questions/6618002/python-property-versus-getters-and-setters). Your client code should never need to know the internal data member names (no *myobj.mymember* references!). **But you're still describing the symptom of the problem, not the problem itself. What is your client code actually trying to _do_ with the data members Poo,Poo1?** — smci, Mar 20 '14 at 10:42
"`Graph` has two object variables". That's not telling us much. But **what are they for**? successors and payload? predecessors and successors? ...? — smci, Mar 20 '14 at 10:50
@smci Getters/setters and properties for the sake of properties are not Pythonic - Python is not Java. Specifically, a property that merely mirrors an internal slot is unnecessary and reduces performance for no good reason. The whole point of properties is that you can add them **later**, when and *if* you decide that you need additional processing when getting or setting the data. Using a bare attribute is perfectly fine in Python. — user4815162342, Mar 20 '14 at 14:01

score 1 · Answer 1 · edited May 23 '17 at 11:57

Whoa, slow down. The coding process you described is not scalable. How exactly did you change the behavior of the function? Give specifics, please.

UPDATE: This all sounds like you're trying to implement a class and its methods by cobbling together a motley patchwork of functions and local variables - like I wrongly did when I first learned OO coding in Python. The code smell is that when the type/class of some class internal changes, it should generally not affect the class methods. If you're refactoring all your code every 10 mins, you're doing something seriously wrong. Step back and think about clean decomposition into objects, methods and data members.

(Please give more specifics if you want a more useful answer.)

If you were only changing input types, there might be no need to change the calling code. (Unless the new fn does something very different to the old one, in which case what was the argument against calling it a different name?)
If you changed the return type, and you can't find a common ancestor type or container (tuple, sequence etc.) to put the return values in, then yes you need to change its caller code. However...
...however if the function should really be a method of a class, declare that class and the method already. The previous paragraph was a code smell that your function really should have been a method, specifically a polymorphic method.
Read about code smells, anti-patterns and When do you know you're dealing with an anti-pattern?. There e.g. you will find a recommendation for the video "Recovery from Addiction - A taste of the Python programming language's concision and elegance from someone who once suffered an addiction to the Java programming language." - Sean Kelly
Also, sounds like you want to use Test-Driven Design and add some unittests.

If you give us the specifics we can critique it better.

I updated the answer, and also added more comment/requests for clarification underneath your question - please respond. Basically what are your data members **for**? Explain what you need to access them for, from client code (that screams 'method' at me)? — smci, Mar 21 '14 at 13:03

aychedee · Answer 2 · 2014-03-19T08:36:56.980

1

Sounds like you can only code inside an IDE!

Two steps to free yourself from your IDE and become a better programmer.

Write unit tests for your code.
Learn how to use grep

Unit tests will exercise your code and provide reassurance that it is always doing what you wanted it to do. They make refactoring MUCH easier.

grep, what a wonderful tool grep -R 'my_function_name' src will find every reference to your function in files under the directory src.

Also, see this rather wonderful blog post: Unix as an IDE.

edited Mar 19 '14 at 08:36

answered Mar 19 '14 at 08:28

aychedee

24,871
8
79
83

Tongue might be slightly in cheek... :) And I do use an Integrated Development Environment. It's a great one, called Linux. – aychedee Mar 19 '14 at 08:53

Aaron · Answer 3 · 2014-03-19T07:35:01.993

You won't get this functionality in a text editor. I use sublime text 3, and I love it, but it doesn't have this functionality. It does however jump to files and functions via its 'Goto Anything' (Ctrl+P) functionality, and its Multiple Selections / Multi Edit is great for small refactoring tasks.

However, when it comes to IDEs, JetBrains pycharm has some of the amazing re-factoring tools that you might be looking for.

The also free Python Tools for Visual Studio (see free install options here which can use the free VS shell) has some excellent Refactoring capabilities and a superb REPL to boot.

I use all three. I spend most of my time in sublime text, I like pycharm for refactoring, and I find PT4VS excellent for very involved prototyping.

Despite python being a dynamically typed language, IDEs can still introspect to a reasonable degree. But, of course, it won't approach the level of Java or C# IDEs. Incidentally, if you are coming over from Java, you may have come across JetBrains IntelliJ, which PyCharm will feel almost identical to.

One's programming style is certainly different between a statically typed language like C# and a dynamic language like python. I find myself doing things in smaller, testable modules. The iteration speed is faster. And in a dynamic language one relies less on IDE tools and more on unit tests that cover the key functionality. If you don't have these you will break things when you refactor.

user4815162342 · Answer 4 · 2014-03-19T08:50:02.643

0

One of the tradeoffs between statically and dynamically typed languages is that the latter require less scaffolding in the form of type declarations, but also provide less help with refactoring tools and compile-time error detection. Some Python IDEs do offer a certain level of type inference and help with refactoring, but even the best of them will not be able to match the tools developed for statically typed languages.

Dynamic language programmers typically ensure correctness while refactoring in one or more of the following ways:

Use grep to look for function invocation sites, and fix them. (You would have to do that in languages like Java as well if you wanted to handle reflection.)
Start the application and see what goes wrong.
Write unit tests, if you don't already have them, use a coverage tool to make sure that they cover your whole program, and run the test suite after each change to check that everything still works.

edited Mar 19 '14 at 08:50

answered Mar 19 '14 at 07:23

user4815162342

141,790
18
296
355

A question on point 3: if the common code is correctly tested, would instantiations also have to be "unit"-tested? Or is it just a way to enforce correct implementation of the common code at the first time, *before* writing multiple implementations? – Joël Mar 19 '14 at 08:28
Good unit tests, test that the behaviour of a class or function is as expected. You can then have a different layers of tests that are integration or functional tests. These might drive the application like a black box, or they might just see what happens when you call top level functionality. – aychedee Mar 19 '14 at 08:34
@Joël The distinction between *common code* and *instantiations* feels arbitrary and is probably subject to change as the code evolves. You get to choose what your unit tests cover, and a typical place are the lines that divides the interface and the implementation at various abstraction levels. – user4815162342 Mar 19 '14 at 09:13

score 0 · Answer 5 · answered Mar 19 '14 at 08:24

One answer only specific to your edit:

if your old code was working and does not need to be modified, you could just keep old names as alias of the new ones, resulting in your old code not to be broken. Example:

class MyClass(object):
    def __init__(self):
        self.t = time.time()
    # creating new names
    def new_foo(self, arg):
        return 'new_foo', arg
    def new_bar(self, arg):
        return 'new_bar', arg
    # now creating functions aliases
    foo = new_foo
    bar = new_bar

if your code need rework, rewrite your common code, execute everything, and correct any failure. You could also look for any import/instantiation of your class.

Tracking changes in python source files?

5 Answers5