Python: Extracting number data in certain ranges

Question

I am a Python beginner trying to count the number of certain sizes in a big data set. The original data is in a text file separated by tabs. I have "Names" (string, but each row seems like a list) of different animals and "Sizes" (integer number) of them in a different row. I would like to count the number of all the animals that fall in certain size ranges, between 10-30.

So far, I have successfully counted how many of each "Name" I have but failing in specifying the "Size". The code I have is below, and I dot get any error but it just gets ignored. Could somebody please help me why the codes are being ignored? Thank you for your help in advance!

import csv, collections

reader=csv.reader(open('C:\Users\Owl\Desktop\Data.txt','rb'), delimiter='\t')
counts=collections.Counter()

for line in reader:
   Name=line[1]
   Size=line[10]
   counts[Name]+=1

for (Name, count) in counts.iteritems():
   if 10<=Size<=30:
      print '%s: %s' % (Name, count)

Answer 1

As written, Size will be permanently set to the last size value in the file, it's not stored along with Name.

Each round through the for loop, Size is set to line[10], but it's not stored in anything outside of the scope of the loop. Name is indirectly stored in the counter. So the next time the loop runs, the value of Size changes to the next animal's size.

Does each animal appear more than once in the data?

You will either need a slightly more complex data structure or to look at the size while looping through the file.

If you don't mind ignoring the animals outside of the size range:

for line in reader:
    size = float(line[10])
    if 10 <= size <= 30:
        name = line[1]
        counts[name] += 1

for name, count in counts.iteritems():
    print '%s: %s' % name, count

(Note: I've changed the case and whitespace of your original code to match Python's recommended style guide, pep8.)

Answer 2

Size=line[10]

makes Size a string.

10<=Size<=30

compares ints with a string (Size).

In [3]: 10 <= '20' <= 30
Out[3]: False

To fix this use:

try:
    Size = float(line[10])
except (ValueError, IndexError):
    continue

The try...except above will cause your program to skip lines in your csv file that either does not have an 11th column or has a string there which can not be converted to a float.

---

In Python2, ints compare less than strings.

In [4]: 10 <= '1'
Out[4]: True

(Believe it or not, because i as in int comes before s as in string in the alphabet...)

In Python3, a TypeError is raised.

Python 3.2.2 (default, Sep  5 2011, 22:09:30) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 10 <= '1'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: int() <= str()

Hallelujah.

Answer 3

One of the cool features of python is that keys in dictionaries can be pretty advanced things such as... tadaaa!... tuples (or dates, or a lot of stuff... as long it's hashable, as J.F. Sebastian pointed out -nothing illegal with hashes, here-). Combine that with regular expressions and you have a pretty fancy "Size classifier" :-) :

sizesFromFile = [
    "Name: Cat, Size: 3.2",
    "Name: Dog, Size: 4.2",
    "Name: BigFoot, Size: 12",
    "Name: Elephant, Size: 31.4",
    "Name: Whale, Size: 85.99",
]

import re
import sys
regex = re.compile(r"^Name:\s*(?P<name>\w+),\s+Size:\s+(?P<size>[\d\.]+)")

myRanges = {
    (0, 10): list(),
    (11, 20): list(),
    (21, 30): list(),
    (31, sys.maxint): list()
}

for line in sizesFromFile:
    match = regex.match(line)
    if match is not None:
        print "Success parsing %s, %s" % (match.groupdict()["name"], match.groupdict()["size"])
        name = match.groupdict()["name"]
        size = float(match.groupdict()["size"])
        for myRange in myRanges:
            if size >= myRange[0] and size <= myRange[1]:
                myRanges[myRange].append(name)

print "This is what I got: %s" % (myRanges)

That outputs:

This is what I got: {(21, 30): [], (11, 20): ['BigFoot'], (0, 10): ['Cat', 'Dog'], (31, 2147483647): ['Elephant', 'Whale']}

Although I'm pretty sure this is very non-optimal, speed speaking... but it's still kinna cool, right?