finding standard deviation down a column of data in python

Question

A.append([(float(a) + float(b) + float(c))/3,
          (float(d) + float(e) + float(f))/3,
          (float(g) + float(h) + float(i))/3,
          (float(j) + float(k) + float(l))/3,
          (float(m) + float(n) + float(o))/3,
          (float(p) + float(q) + float(r))/3,
          (float(s) + float(t) + float(u))/3])


def mean(A):
    positives = [b for b in A if b >= 0]
    E.append((len(positives)) / (len(A))*100)
    if positives:
        return sum(positives) / len(positives)
    else:
        return 0

C = map(mean, zip(*A))
print C

#3 sigma check
def sigma(A):
    positives = [b for b in A if b >= 0]
    if positives:
        F.append((positives - C) / len(A))

print F

I am looking to find the standard deviation of the out put of the first chunk of code. It results in lists of seven numbers ie: [[-9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0], [-9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0, -9999.0], [0.040896, 0.018690, 0.0056206, -9999.0, 0.038722, 0.018323, -9999.0], [0.03944364, -9999.0, 0.037885, 0.014316, -9999.0]]

The second chunk of code finds the mean of the columns(['0.040170', '0.018057', '0.004782', '0.000000', '0.037378', '0.014778', '0.000000'])

I began writing the third chunk of code to find standard deviation but F prints out blank. Also I do not think I am writing the function properly to subtract each positive number by the average, any help would be appreciacted

inspectorG4dget · Accepted Answer · 2013-07-17T19:00:16.830

5

If I understand your question correctly, you have a list of lists, where each sublist contains multiple floats.

If you want to compute the standard deviation of a list of numbers:

import numpy
numpy.std(myList)

If you want to compute the standard deviation of all the numbers in the ith "column" of a list of lists:

import numpy
numpy.std(zip(*myList)[i])

If you want to exclude negative numbers within a column:

import numpy
import itertools
numpy.std([i for i in itertools.izip(*myList)[i] if i>=0])

Hope this helps

edited Jul 17 '13 at 19:00

answered Jul 16 '13 at 18:43

inspectorG4dget

110,290
27
149
241

This is probably a stupid question but can you explain the syntax of `*myList` or point me in the right direction of that operator? I keep ending up at C++ references – sedavidw Jul 16 '13 at 18:55
1

It's a very good question, actually. It turns a list into a sequence of the list's elements. Check [this](http://stackoverflow.com/q/2921847/198633) out – inspectorG4dget Jul 16 '13 at 18:58
@inspectorG4dget, when I use this syntax how would I exclude the numbers in the column that are negative? – KJo Jul 16 '13 at 20:49
@inspectorG4dget I'm getting this error: `NameError: name 'itertools' is not defined` – KJo Jul 17 '13 at 13:25
@KJo: `import itertools` – inspectorG4dget Jul 17 '13 at 18:59

finding standard deviation down a column of data in python

1 Answers1