"python way" to parse and conditionally replace every element in 2D list

Question

I have a list which consists of further lists of strings which may represent words(in the alphanumeric sense) or ints, e.g.

myLists = [['5','cat','23'], 
           ['33','parakeet','scalpel'], 
           ['correct','horse','battery','staple','99']]

I want to parse the array so that all integer representations are converted to ints. I have a simple function, numParser(string) to this end:

def numParser(s):
    try:
        return int(s)
    except ValueError:
        return s

With my c/java background I would normally just iterate through both arrays, changing each value (the fact that those arrays would be homogeneous notwithstanding). But I'm new to python and thought there might be a better way to do this, so I searched around and found a couple SO posts regarding map() and list comprehension. List comprehension didn't seem like the way to go because the lists aren't uniform, but map() seemed like it should work. To test, I did

a=['cat','5','4']
a = map(numParser, a)

Which works. Without thinking, I did the same on a nested loop, which did not.

for single_list in myLists:
    single_list = map(numParser, rawData)

Now, after receiving an unchanged 2D array, it occurs to me that the iterator is working with references to the array, not the array itself. Is there a super snazzy python way to accomplish my goal of converting all integer representations to integers, or do I need to directly access each array value to change it, e.g. myLists[1][2] = numParser(myLists[1][2])?

Aside: using a bare `except` is usually a bad idea. It hides errors (typos -- say you typed `striing` instead, or accidentally rebound `int` beforehand, etc). Specify your expected exceptions instead. — DSM, Jul 08 '15 at 15:34
Yes, I'm a bad person :) This is just a quick script for myself though: I could say that that removes the chance for user error but knowing me, it probably wouldn't. I'll edit so I'm not spreading bad practice around. — Daniel B., Jul 08 '15 at 15:36
@DSM it's even worse than that, it can catch BaseException and that can cause your program to behave really weird. As for Daniel B. and your list of lists I think you can't escape from direct modification as you can't de-reference your list 'pointer' — user3012759, Jul 08 '15 at 15:40
I'd also avoid using `list` and `string` as variable names - both of those are names defined in the standard library — Moshe, Jul 08 '15 at 16:24
I have list as something else in my code but not string, thanks for catching that! I'll edit. — Daniel B., Jul 08 '15 at 16:28

score 4 · Answer 1 · answered Jul 08 '15 at 15:30

4

You can do this with list comprehension:

>>> [map(numParser, li) for li in myLists]
[[5, 'cat', 23], [33, 'parakeet', 'scalpel'], ['correct', 'horse', 'battery', 'staple', 99]]

answered Jul 08 '15 at 15:30

Rohit Jain

209,639
45
409
525

I was about to ask why this would avoid the problem I have, but list comprehension creates a new list from the operations performed by the comprehension. Or am I mistaken and it's done in place? Would this be ill advised if I had a particularly large data set? – Daniel B. Jul 08 '15 at 15:34
It will create a new list comprehension.. And that's the tradeoff. In general if it is creating a bottleneck in your application, then you can look for other way. But I think that would rarely be the case that you run out of memory. I guess it should be able to handle fairly large datasets. – Rohit Jain Jul 08 '15 at 15:38
I doubt anything I'll be doing will even leave L3 or maybe L2. Thanks :) – Daniel B. Jul 08 '15 at 15:39

score 2 · Answer 2 · answered Jul 08 '15 at 15:30

You can use a nested list comprehension and use str.isdigit() method to check of your string is a digit, then convert it to int:

>>> [[int(i) if i.isdigit() else i for i in j] for j in myLists]
[[5, 'cat', 23], [33, 'parakeet', 'scalpel'], ['correct', 'horse', 'battery', 'staple', 99]]

Or if you want to use your function you can pass it to map function or use in a list comprehension.

score 2 · Accepted Answer · answered Jul 08 '15 at 15:33

2

Here is one way to do it:

for single_list in myLists:
    for i, item in enumerate(single_list):
        single_list[i] = numParser(item)

The advantage of this method is you actually replace the elements in the existing lists, not creating new lists.

answered Jul 08 '15 at 15:33

Hai Vu

37,849
11
66
93

This has spawned another question: http://stackoverflow.com/questions/31298416/why-will-one-loop-modify-a-list-of-lists-but-the-other-wont – Daniel B. Jul 08 '15 at 16:26

score 0 · Answer 4 · answered Jul 08 '15 at 16:02

def numParser(string):
    try:
        return int(string)
    except ValueError:
        return string

if __name__ == '__main__':
    myLists = [['5','cat','23'],
           ['33','parakeet','scalpel'],
           ['correct','horse','battery','staple','99']]

    print list(map(numParser, _list) for _list in myLists)


[[5, 'cat', 23], [33, 'parakeet', 'scalpel'], ['correct', 'horse', 'battery', 'staple', 99]]

"python way" to parse and conditionally replace every element in 2D list

4 Answers4

Linked