Numpy savetxt error

Question

Below is the code I am struggling with as a beginner:

commonfields = arcpy.ListFields(gt_pnts)
names = [f.name for f in commonfields]
needed_names = names[3:]

gt_pnts_arr = arcpy.da.FeatureClassToNumPyArray(gt_pnts, needed_names)
gt_pnts_arr = gt_pnts_arr.reshape((gt_pnts_arr.shape[0],1))
eq_smpled_pnts_arr = arcpy.da.FeatureClassToNumPyArray(eq_smpled_pnts, needed_names)
eq_smpled_pnts_arr = eq_smpled_pnts_arr.reshape((eq_smpled_pnts_arr.shape[0],1))
master_table = np.concatenate((gt_pnts_arr, eq_smpled_pnts_arr), axis=0)
np.savetxt(outputcsvfilename,master_table, fmt="%.1f")

Error is as below:

TypeError: float argument required, not numpy.void

I searched and found this exception raised when datatype is not defined. But when I test as below I am in the dilemma-

>>>master_table.dtype.descr
>>>[('grid_code', '<i4'), ('b1_Clip_ProximityRaster1', '<f4'), ('b2_Clip_ProximityRaster1', '<f4'), ('b3_Clip_ProximityRaster1', '<f4'), ('b4_Clip_ProximityRaster1', '<f4'), ('b5_Clip_ProximityRaster1', '<f4'), ('b6_Clip_ProximityRaster1', '<f4'), ('b7_Clip_ProximityRaster1', '<f4'), ('b8_Clip_ProximityRaster1', '<f4'), ('b9_Clip_ProximityRaster1', '<f4'), ('b10_Clip_ProximityRaster1', '<f4'), ('b11_Clip_ProximityRaster1', '<f4'), ('b12_Clip_ProximityRaster1', '<f4'), ('b13_Clip_ProximityRaster1', '<f4'), ('resp', '<U2')]
>>>master_table
array([[ (13, 13.0, 3810.0, 3810.0, 1982.952392578125, 3873.71923828125, 34869.9140625, 5483.3564453125, 7272.138671875, 4409.591796875, 872.0665283203125, 36238.62109375, 4441.62109375, 6775.2861328125, u'1')],
       [ (1, 1.0, 3601.99951171875, 3603.12353515625, 1626.9295654296875, 3725.922607421875, 34595.9453125, 5810.5595703125, 7592.90478515625, 4476.0361328125, 576.2811889648438, 36462.984375, 4499.0, 7164.47509765625, u'1')],
       [ (13, 13.0, 3721.93505859375, 3723.02294921875, 1642.3458251953125, 3842.928466796875, 34713.43359375, 5702.3681640625, 7597.17041015625, 4562.07177734375, 657.9513549804688, 36343.12890625, 4586.9599609375, 7111.0126953125, u'1')],.....................
>>>master_table.shape
>>>(50, 1)
>>>gt_pnts_arr.shape
>>>(25, 1)

Even I can not load this master_table into pandas dataframe as below-

df = pd.DataFrame(data=master_table[1:,1:], index=master_table[1:,0],columns=master_table[0,1:])

My table datatypes: There are 13 colums and 50 rows in the master_table.First and last column datatype is respectively integer and integer but all other(11) data type is float.

looks to me like your `master_table` is non numerical because of the last column of `u'1'`'s, and it also contains python tuples instead of being multidimensional with floats. You probably just need to reformat your data... — Julien, Nov 22 '16 at 07:11
2d numeric arrays are easiest for `savetxt`. It can handle 1d structured arrays also, but getting the `fmt` right can trickier. The code isn't complicated - just iterate and format the 'row' with `fmt%tuple(row)`. The numpy void is a record of your structured array. — hpaulj, Nov 22 '16 at 08:29

hpaulj · Answer 1 · 2016-11-22T18:17:52.840

Recreating your data (thaxs for enough information), I make a data array

In [33]: data.shape
Out[33]: (3, 1)
In [34]: len(data.dtype.fields)

Trying savetxt:

In [28]: np.savetxt('test.csv',data, fmt="%.1f")
-> 1158   fh.write(asbytes(format % tuple(row) + newline))
   1159      except TypeError:

TypeError: cannot convert to a float; scalar object is not a number

and a secondary error:

TypeError: Mismatch between array dtype .... and format specifier ('%.1f')

savetxt iterates on data

for row in data:
    f.write(fmt%tuple(row))

But with this shape row is a (1,) array.

But if I get rid of the unnecessary 2nd dimension (size 1)

In [37]: np.savetxt('test.csv',data[:,0], fmt="%.1f")
TypeError: a float is required

and

TypeError: Mismatch between array dtype .... and format specifier ('%.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f')

Note that is has taken your fmt, and replicated it to match the number of fields. Copy that last format specifier, and tweak it to work with the final string column

In [38]: fmt='%.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %.1f %s'

Now it works:

In [39]: np.savetxt('test.csv',data[:,0], fmt=fmt)
In [40]: cat test.csv
13.0 13.0 3810.0 3810.0 1983.0 3873.7 34869.9 5483.4 7272.1 4409.6 872.1 36238.6 4441.6 6775.3 1
1.0 1.0 3602.0 3603.1 1626.9 3725.9 34595.9 5810.6 7592.9 4476.0 576.3 36463.0 4499.0 7164.5 1
13.0 13.0 3721.9 3723.0 1642.3 3842.9 34713.4 5702.4 7597.2 4562.1 658.0 36343.1 4587.0 7111.0 1

So there are 2 corrections - a structured array has to be 1d to work with savetxt - rows and fields. And the fmt has to work with the dtype. In this case you have to handle the string field.

If the last field had been numeric (e.g. int) as opposed to U2, the data[:,0] would have worked with the simple %.1f fmt. The initial int field saves just fine (though the fmt could also be tweaked for that).

Numpy savetxt error

1 Answers1