0

Similar to Meenakshi I am trying to append data to a file using numpy's savetxt function.

I have a .data file that I want to append more float32 data to.

responses = np.array(responses,np.float32)
responses = responses.reshape((responses.size,1))

# save as new training results
np.savetxt(sampleData,samples)
np.savetxt(responseData,responses)

# want the option to append to previously written results

I am able to append as binary with the following but I need to append in float32. 

# append to old training results 
    with open(sampleData, 'ab+') as fs:
        fs.write(samples)
    with open(responseData, 'ab+') as fr:
        fr.write(responses)

When I try 

# append to old training results 
        with open(sampleData, 'a+') as fs:
            fs.write(samples)
        with open(responseData, 'a+') as fr:
            fr.write(responses)

I get "TypeError: write() argument must be str, not numpy.ndarray"

What is the syntax/ extension I should be using to interact with this .data file type in python given the above?

Nina Moorman
  • 36
  • 4

1 Answers1

1

Update:

Didn't see your comment on appending originally. The answer in your linked question shows you were on the right track:

The following will append your data as expected, but without dumping "gibberish" (bytes). np.savetxt apparently takes care of doing the appropriate formatting/encoding so that what's written is human readable.

with open(some_file, 'ab+') as fo:
    np.savetxt(fo, responses)

Original - left here to explain why the OP approach wasn't working

Your comment hints into what's happening:

The following does append but it inputs gibberish (i assume because of the binary, but without the b it tells me I need to input a string) --> with open(sampleData, 'ab+') as fs: fs.write(samples) with open(responseData, 'ab+') as fr: fr.write(responses)

When you try to write without b, it complains appropriately, because you need to give it a string in normal write-mode - you can't just write a list/array (which is what samples and responses are). When you use b, you're writing in binary/bytes mode, so whatever you pass to write gets coerced into bytes. This is what I see if I write the following in binary mode:

resp = np.array([1, 2, 4, 5], np.float32)
resp = resp.reshape((resp.size, 1))
np.savetxt(file1, resp)
with open(file2, 'ab+') as fo:
    fo.write(resp)

# From Hex view of written file
00 00 80 3F 00 00 00 40 00 00 80 40 00 00 A0 40

which is identical to calling bytes(...) on the array I made:

import binascii
binascii.hexlify(bytes(resp))

# produces:
b'0000803f00000040000080400000a040' -> '00 00 80 3f 00 00 00 40 00 00 80 40 00 00 a0 40'

So, you just need to format your data into a str friendly representation, such as joining into a string (for example):

>>> ', '.join(str(x) for x in resp)
'[1.], [2.], [4.], [5.]'

...but how you format it is of course dependent on your requirements.

b_c
  • 1,202
  • 13
  • 24