Inverting floats in a list of lists

Question

I have the output of a Markov transition table, which is a list of 59 lists each with 59 floats. I want to invert each of the non-0 floats, and then normalise the output so that again I have a list of probabilities which add up to 1.

I have read the textbook on list comprehensions, and that seems relevant, but I can't for the life of me understand how to implement it.

The list of lists is m

for i in range(59):
    [1/item for item in m[i] if item > 0.]
    i += 1

This runs, but it doesn't change m. Am I wrong to be using item in this code? Should I be using some other reference?

Thanks Graipher. I'm not entirely sure what that means. Do you mean something like I need to change the values in the list? Could you show me how to do that? — Richard Hames, Mar 15 '18 at 10:19
Although this calculation in itself doesn't demonstrate a need, I recommend you use numpy arrays rather than Python lists for numerical analysis: [here's why](https://stackoverflow.com/questions/993984/why-numpy-instead-of-python-lists). — jpp, Mar 15 '18 at 10:41
This is a (not-exact, but similar enough) duplicate of a question I answered earlier today. — cs95, Mar 15 '18 at 13:15

score 8 · Answer 1 · answered Mar 15 '18 at 10:23

[1/item for item in m[i] if item > 0.] creates a new list. Then you do nothing with that list. What Graipher is telling you is that you need a reference to the newly created list:
m[i] = <your list comprehension> will assign back to row i of your list of lists.

Also, consider numpy for element-wise operations. Demo:

>>> import numpy as np
>>> m = np.array([[1,2], [3, 0]])
>>> m = 1.0/m
>>> m
array([[ 1.        ,  0.5       ],
       [ 0.33333333,         inf]])
>>> m[m == np.inf] = 0
>>> m
array([[ 1.        ,  0.5       ],
       [ 0.33333333,  0.        ]])

Graipher · Answer 2 · 2018-03-15T10:32:25.410

5

You need to assign it back to m[i] in the loop:

for i in range(len(m)):
    m[i] = [1./item if item != 0. else item for item in m[i]]

And the i+=1 is not needed because the for loop does that for you.

In order to avoid the sublist being shorter than the original sublist, you need to move the if from the end of the list comprehension (which basically filters which items are processed) to the front (so the value itself is a ternary expression that evaluates to 1/item or item, depending on the value of item).

At this point you should probably watch the presentation Loop like a Native and rewrite it for example as:

for i, x in enumerate(m):
    m[i] = [1./item if item != 0. else item for item in x]

edited Mar 15 '18 at 10:32

answered Mar 15 '18 at 10:20

Graipher

6,891
27
47

Thanks so much. I now have a list of lists, but they are of different lengths. Is there a way to return a 0 value in the same place if item = 0? There is no 'else' possible after that 'if', as far as I can see. – Richard Hames Mar 15 '18 at 10:25
@RichardHames There is. I also changed `> 0` to `!= 0` to allow negative values. – Graipher Mar 15 '18 at 10:27
1

My bad. Apparently I'm having a bad reading day today and should stop answering questions. I just had to completely rewrite an answer to an XML question for similar reasons. Deleted. – holdenweb Mar 15 '18 at 10:36

Jean-François Fabre · Answer 3 · 2018-03-15T12:50:20.847

[1/item for item in m[i] if item > 0.] alone on a line creates a throwaway object. Not really useful.

My suggestion: rebuild your list of lists using a double comprehension:

m = [[1./item for item in sm if item > 0.] for sm in m]

this solution is Python 2/3 compliant in regards to floating point division whatever the list contains (integers or floats)

EDIT: quoting the question: "I want to invert each of the non-0 floats", the formula is incorrect since it doesn't include the zeroes. So a more accurate version would be not to filter, but to test expression with a ternary so it keeps zeroes.

m = [[1./item if item else 0. for item in sm] for sm in m]

(if item is the shortcut to test item against 0)

I'd go with this solution, but aren't you skipping items `<= 0.`? Could `[[1./item if item > 0. else item for item in sm] for sm in m]` improve the solution? — Gsk, Mar 15 '18 at 10:49

jpp · Answer 4 · 2018-03-15T10:37:15.860

I recommend you use, if possible, tools specifically designed for this purpose.

In the below vectorised solution, inversion occurs in place via numpy, and sklearn.preprocessing.normalize is used to normalize.

import numpy as np
from sklearn.preprocessing import normalize

arr = np.array([1, 2, 0, 1, 1, 0, 4, 0], dtype=float)

def inv(A):
    m = (A != 0)
    A[m] = 1/A[m]
    return A

res = normalize(inv(arr))

# [[ 0.54944226  0.27472113  0.          0.54944226  0.54944226  0.
#    0.13736056  0.        ]]

Inverting floats in a list of lists

4 Answers4