Python: Am I missing something?

Question

I'm in the process of learning Python while implementing build scripts and such. And for the moment everything is working fine in that the scripts do what they need to do. But I keep having the feeling I'm missing something, such as "The Python Way". I know build scripts and glue scripts are not really the most exciting development work and may hardly be a candidate for revealing the true power of Python but I'd still like the opportunity to have my mind blown. I develop mostly in C# and I find that my Python code looks awfully similar in structure and style to a lot of my C# code. In other words I feel like I'm thinking in C# but writing in Python.

Am I really missing something?

(Note: I realize this isn't so much a programming question and it's quite broad and there may not be a definitive answer so mod me down into oblivion if you have to.)

Answer 1

I would recommend that you read up on Generators, Iterators, itertools and above all List Comprehensions.

These are the pillars of anything Pythonic. And for everything else, there is PEP-8.

Read up on these concepts and try using them wherever appropriate. All the best!

PS: Don't forget to import this ;)

Addendum: I would also aggregate some excellent suggestions given by others in this thread here:

• Read Code Like a Pythonista: Idiomatic Python (from David Grant)
• Solving problems from Project Euler is a great idea (from Nick Fortescue)
• This excellent presentation on Generators and Iterators by David Beazley (from André)
• Learn about Scons (from the venerable S.Lott)

Answer 2

No - this is common for folks who move to Python from other C-like languages. I believe what you are looking for is ways to make your code more "Pythonic". The good news is the more Python you write the more Pythonic your code will become. It is a natural overflow of the "how can I do this more simply" attitude.

Another good place to look at is The Zen of Python. These attitudes towards Python development will also help you in the same regard.

Answer 3

You've gotten good suggestions so far. I'd only add Dive Into Python.

EDIT: As of Oct 4, 2011, this work can be found here. Dive Into Python 3, here. "Why" this is: see here, here and here.

Answer 4

You should definitely take a look at this talk, when you start doing systems programming with python: http://www.dabeaz.com/generators/

Answer 5

Recently I've been learning/improving my python by solving the Project Euler problems in python. This has worked really well for me because:

1. It is fun and competitive, so I'm motivated to keep going
2. It forces me to use the python data structures in a really natural way to get the performance I need, so has taught me a lot about lists, sets, strings, iteration etc.
3. Most of the problems need less than a page of code to solve, so you have more time to think about polishing or rewriting in a more elegant way
4. Python copes with large integers really easily, and so it just feels like the right language to use

I'd thoroughly recommend this.

Answer 6

Are you reading Python you haven't written?

Here's a script from the Python 2.6.1 distribution that deletes .pyc and .pyo files.

#!/usr/local/bin/python
"""Recursively zap all .pyc and .pyo files"""
import os
import sys

# set doit true to actually delete files
# set doit false to just print what would be deleted
doit = 1

def main():
    if not sys.argv[1:]:
        if os.name == 'mac':
            import EasyDialogs
            dir = EasyDialogs.AskFolder(message='Directory to zap pyc files in')
            if not dir:
                sys.exit(0)
            zappyc(dir)
        else:
            print 'Usage: zappyc dir ...'
            sys.exit(1)
    for dir in sys.argv[1:]:
        zappyc(dir)

def zappyc(dir):
    os.path.walk(dir, walker, None)

def walker(dummy, top, names):
    for name in names:
        if name[-4:] in ('.pyc', '.pyo'):
            path = os.path.join(top, name)
            print 'Zapping', path
            if doit:
                os.unlink(path)

if __name__ == '__main__':
    main()

How many Python idioms can you find in that?

Answer 7

Think like this:

• If you are writing too much for little work, something is wrong, this is not pythonic.

Most Python code you will write is very simple and direct. Usually you don't need much work for anything simple. If you are writing too much, stop and think if there is a better way. (and this is how I learned many things in Python!)

Answer 8

To add to the answers of Andrew Hare and Baishampayan Ghose...

To learn the idiom of any language must involve reading code written in that idiom. I'm still learning the Python idiom, but I've been through this with other languages. I can read about list comprehensions, but the lightbulb only really comes on when you see such things in use and say, "Wow! That's awesome! Two lines of code and it's crystal clear!" So go find some pythonic code that you find interesting and start reading it and understanding it. The knowledge will stay in your head better if you see everything in the context of a working program.

Answer 9

Study well-designed pythonic APIs / frameworks. Some of the best live outside the standard library. Note what they allow you to not do. My favorites:

• py.test is more pythonic than unittest (which was based on JUnit). Just write test_foo() function instead of a class inheriting from unittest.TestCase. Just do assert x == y instead of self.assertEqual(x, y). Many more goodies...

  • Doctest is also more pythonic than unittest. Many give up on it due to some practical annoyances, but the idea is brilliant.

  • If you're deeply into testing, the mock module's action -> assertion approach is more Pythonic than competing libraries record -> replay paradigm.

• requests is one of the cleanest API's I've ever seen, way better than httplib + urllib + urllib2.

• JSON and YAML are more pythonic than XML.

  • JSON-RPC is more pythonic than XMLRPC or shudder SOAP.

  • ElementTree is much more pythonic API than DOM. By optimizing for tags having either .text or children it reduces the XML <-> data structures impedance mismatch (but can still round-trip mixed content via the .tail hack).

• Scapy is an astonishingly terse framework for sending and dissecting network packets.

• I believe Storm is the most pythonic Object-Relational Mapping out there. Disclaimer: I have nearly no experience with ORMs.

• heapq is a beautifully implemented module. Just read the code.

• itertools is very powerful. Read all the examples in the documentation.
  Opinions vary on whether using it extensively is pythonic or a weird dialect :-)

Read "what's new in Python", at least for the last version you're using. Read some PEPs. Understand why the changes are an improvement.

Finally, make sure you make the most of interactive Python. The best way to learn APIs is to try them out and see what happens! Multi-line editing, completion and easy help() access are must-have features - check out ipython (including the notebook!), dreampie, bpython.

Answer 10

Another powerhouse resource: Code Like a Pythonista: Idiomatic Python.

Answer 11

Write some Python code and post it on SO for review and feedback whether it is pythonic.

Answer 12

Apart from stuffs like Generators, Iterators, List Comprehensions etc mentioned by other folks here, I would like to suggest couple of more concepts which I feel definitely adds value to someone attempting to do things the 'pythonic' way. These are Decorators and Meta Classes.

For Decorators, this stackoverflow answer is highly recommended and for Meta Classes you can go through this.

Answer 13

While build scripts and clue scripts [sic] are not really the most exciting development work, they are completely a candidate for revealing the true power of Python.

First, look at the various make-like tools already written in Python.

The big fish in this pond is Scons: http://www.scons.org/. Learn about this.

This blog post talks about the alternatives http://farmdev.com/thoughts/46/the-python-make-tool/

"I'd still like the opportunity to have my mind blown" Start with Google: find Python tools that already do some or all of what you're trying to do. Code less, download and read more.

Answer 14

import this

If you can put your code next to the Zen of Python and feel good about it you are half way there. Also check the this.py module: a little bit of irony.

The other half is learning about stuff like: regular expressions, list comprehensions, generators, meta-classes, argument unpacking, decorators, list stepping, 'with' statement, conditional assignment, powerful built-ins like enumerate(), map(), zip(), reduce(), any() and pow(), and many others(check Hidden features of Python for more)

Along the way you can read other people's code and modify it.

Finnaly: Learn about finally and import antigravity

Answer 15

I would suggest finding a personal python guru. Show them some of your code and have them review/rewrite it into idiomatic python. Thus will you be enlightened.

Answer 16

To echo TLHOLADAY, read the standard library. That's where the "pythonic" stuff is. If you're not getting a good feel there, then read the source for sqlachemy or django or your project of choice.