2

I am getting an issue where I am trying to run (on Python):

#Loading in the text file in need of analysis
x,y=loadtxt('2.8k to 293k 15102014_rerun 47_0K.txt',skiprows=1,unpack=True,dtype=float,delimiter=",")

C=-1.0      #Need to flip my voltage axis

yone=C*y    #Actually flipping the array

plot(x,yone)#Test

origin=600.0#Where is the origin? i.e V=0, taking the 0 to 1V elements of array

xorg=x[origin:1201]# Array from the origin to the final point (n)

xfit=xorg[(x>0.85)==True] # Taking the array from the origin and shortening it further to get relevant area

It returns the ValueError. I have tried doing this process with a much smaller array of 10 elements and the xfit=xorg[(x>0.85)==True] command works fine. What the program is trying to do is to narrow the field of vision, of some data, to a relevant point so I can fit a line of best fit a linear element of the data.

I apologise for the formatting being messy but this is the first question I have asked on this website as I cannot search for something that I can understand where I am going wrong.

Ashwini Chaudhary
  • 244,495
  • 58
  • 464
  • 504
SmallBarracoder
  • 31
  • 2
  • `(x>0.85)==True` is better written as `x>0.85` – YXD Nov 06 '14 at 11:40
  • `xorg` has 600 elements, `x` has, at least, 1200 elements. When you index `xorg` using the expression `x<0.85` you are indexing a 600 element array using an array of booleans that is, at least, of 1200 elements. `python` loudly complains... – gboffi Nov 06 '14 at 13:21

3 Answers3

3

This answer is for people that don't know about numpy arrays (like me), thanks MrE for the pointers to numpy docs.

Numpy arrays have this nice feature of boolean masks.

For numpy arrays, most operators return an array of the operation applied to every element - instead of a single result like in plain Python lists:

>>> alist = range(10)
>>> alist
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> alist > 5
True

>>> anarray = np.array(alist)
>>> anarray
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

>>> anarray > 5
array([False, False, False, False, False, False,  True,  True,  True,  True], dtype=bool)

You can use an array of bool as the index for a numpy array, in this case you get a filtered array for the positions where the corresponding bool array element is True.

>>> mask = anarray > 5
>>> anarray[mask]
array([6, 7, 8, 9])

The mask must not be bigger than the array:

>>> anotherarray = anarray[mask]
>>> anotherarray
array([6, 7, 8, 9])

>>> anotherarray[mask]
ValueError: too many boolean indices

So you cant use a mask bigger than the array you are masking:

>>> anotherarray[anarray > 7]
ValueError: too many boolean indices

>>> anotherarray[anotherarray > 7]
array([8, 9])

Since xorg is smaller than x, a mask based on x will be longer than xorg and you get the ValueError exception.

Paulo Scardine
  • 73,447
  • 11
  • 124
  • 153
0

Change 

xfit=xorg[x>0.85]

to

xfit=xorg[xorg>0.85]

x is larger than xorg so x > 0.85 has more elements than xorg

YXD
  • 31,741
  • 15
  • 75
  • 115
  • Doesn't make a lot of sense either. Probably he wants `[x for x in xorg if x < 0.86]` – Paulo Scardine Nov 06 '14 at 11:43
  • 1
    It makes sense with numpy arrays, not with Python lists – YXD Nov 06 '14 at 11:43
  • Interesting, what is the mechanism behind this? Does the `>` operator return a lazy expression instead of True/False? – Paulo Scardine Nov 06 '14 at 11:47
  • 1
    Have a look at http://stackoverflow.com/q/993984/553404, http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html and http://scipy-lectures.github.io/intro/numpy/array_object.html#fancy-indexing for more info! – YXD Nov 06 '14 at 11:50
  • Interesting indeed, I will make my discoveries an answer for people like me. – Paulo Scardine Nov 06 '14 at 12:29
0

Try the following: replace your code 

xorg=x[origin:1201]
xfit=xorg[(x>0.85)==True]

with

mask = x > 0.85
xfit = xorg[mask[origin:1201]]

This works when x is a numpy.ndarray, otherwise you might end up in problems as advanced indexing will return a view, not a copy, see SciPy/NumPy documentation.

I'm unsure whether you like to use numpy, but when trying to fit data, numpy/scipy is a good choice anyway...

jkalden
  • 1,548
  • 4
  • 24
  • 26
  • I see! This works now but I was curious on why the original code did not work. Treat me like a python-noob because I pretty much am. Thank you very much so far. – SmallBarracoder Nov 06 '14 at 12:04
  • Sorry, I forgot to link the documentation on advanced indexing. I'll add it in the answer. As also stated by @Mr E, only numpy arrays work properly with this kind of indexing, as for python lists, only views are returned instead of copies. That means that you might see it in an iteractive session, but your `xint` array is not altered. – jkalden Nov 06 '14 at 12:17