0

Using NumPy, I am trying to reshape data that looks like the following once loaded:

(0.00017299999308306724, 0.00020900000527035445, ... 0.9700180292129517)

It has 54 items and I'm trying to ultimately reshape it into a 27x2 array. It does load as an array but returns a shape of: () I have tested with NumPy doing the following: 

    test = np.arange(54)
    print(test)
    print(test.shape)
    test = test.reshape(27,2)

That works as expected, giving an initial shape of (54,) and then a new shape of (27,2). The data mentioned above does not return a shape and I'm unable to reshape it. The test data that does work looks like this when first initialized [0 1 2 3 ...53] Why does the data format look so different (not the types but the [] with spaces vs () with , and how would one go about changing the format so the array may be reshaped?

Chris Townsend
  • 3,042
  • 27
  • 31
  • 1
    Can you provide the code you've used to load your data in? – Ffisegydd Mar 16 '15 at 16:03
  • did you try first to define `a = np.array([...])`? and then `a.resample(...)`? – plonser Mar 16 '15 at 16:10
  • No, I didn't do a resample. I'm loading data from a large NumPy binary file and grabbing two lines out of there. That initially returns a len of 2 and a shape of (2,) but each of those records have 54 values. I can reference them like data[0][n] but when I try to load a single 54 value, record for lack of a better word, I get the problem of returning a shape like () even though I can reference each of the items in the array and it loaded with a call to asarray and other methods. The only difference I see is the way the data looks in my initial description - thank you so much for your time. – Chris Townsend Mar 16 '15 at 16:19
  • 2
    `()` in Python indicates a `tuple` as opposed to a list. In `numpy`, this tuple notation is used for an element of a structured array. What is the `dtype` of the original array? – hpaulj Mar 16 '15 at 17:39
  • Thank you hpaulj - I realized that shortly after posting and called np.tolist within np.asarray and it now gives me a shape. – Chris Townsend Mar 16 '15 at 17:46
  • I don't think it is the best way but right now calling: new_test = np.asarray(test.tolist()) converts it in such a way that I get a valid shape returned and can then go on to reshape. I'm not sure why this would be necessary when working with data stored in the NumPy format. – Chris Townsend Mar 16 '15 at 17:49
  • 1
    Chris, hpaulj is right. You seem to be having a [structured array](http://docs.scipy.org/doc/numpy/user/basics.rec.html), which you might have gotten from a call to `numpy.genfromtxt`. If your entire array is numeric, you could simply fiddle with the options of `genfromtxt` or [convert](http://stackoverflow.com/q/5957380/2476444) it, if the fields of the records have the same byte interpretation and bytesize. – Oliver W. Mar 18 '15 at 00:31
  • Hpaulj & Oliver, thank you. That appears to be the case - that it is a structured array from data that was converted via genfromtxt. It is a large data set ~ 8GB in binary format so I'm unsure of how best to convert at the moment but will investigate that - thank you. The dtype consists of three integers followed by 54 floats per "record". The integer values are used to locate the data, it is sorted and a separate index file is used to then load chunks of this data at a time as needed. – Chris Townsend Mar 18 '15 at 16:44

0 Answers0