0

When trying to write numpy matrix M to binary file as:

from io import open
X = [random.randint(0, 2 ** self.stages - 1)for _ in range(num)]
Matrix = np.asarray([list(map(int, list(x))) for x in X])

file_output = open('result.bin', 'wb')
M = np.ndarray(Matrix, dtype=np.float64)
file_output.write(M)
file_output.close()

I get this error:

Traceback (most recent call last):
  File "experiments.py", line 164, in <module>
    write_data(X, y)
  File "experiments.py", line 39, in write_data
    arr = np.ndarray(Matrix, dtype=np.float64)
ValueError: sequence too large; cannot be greater than 32

Can I know how to fix this? Thank you

Medo
  • 952
  • 3
  • 11
  • 22
  • pls fix your code (e.g. file name) first. – patrick Jul 19 '17 at 14:15
  • You forgot to open a string. Please fix that first. – Willem Van Onsem Jul 19 '17 at 14:15
  • Oh! Thank you . but still have same problem – Medo Jul 19 '17 at 14:16
  • The error occurs in `arr = np.ndarray(Matrix, dtype=np.float64)`. We need to know what `Matrix` is. Please create a [minimal, complete and verifiable example](https://stackoverflow.com/help/mcve). – Warren Weckesser Jul 19 '17 at 14:27
  • 1
    It seems the problem is with using `np.ndarray`. An answer to [this](https://stackoverflow.com/questions/15879315/what-is-the-difference-between-ndarray-and-array-in-numpy) question suggests to not use this, instead, use `np.array` – DavidG Jul 19 '17 at 14:28
  • I even cut-and-pasted the line, and somehow missed the `nd` in `ndarray`. @DavidG is correct--don't use `np.ndarray` like that, use `np.array`. – Warren Weckesser Jul 19 '17 at 14:30
  • @WarrenWeckesser I have updated my post. Thank you – Medo Jul 19 '17 at 14:31
  • This question isn't about saving an array, it's about correctly generating some sort of random array. – hpaulj Jul 19 '17 at 17:06

2 Answers2

1

You can do this in one of two equivalent ways:

import numpy as np

a = np.random.normal(size=(10,10))
a.tofile('test1.dat')

with open('test2.dat', 'wb') as f:
    f.write(a.tobytes())

# diff test1.dat test2.dat

See the docs for tofile. However from the original example, it looks like Matrix is failing to be convert into an ndarray.

JoshAdel
  • 66,734
  • 27
  • 141
  • 140
1

Replace:

M = np.ndarray(Matrix, dtype=np.float64)

with

M = Matrix.astype(np.float64)

np.array(Matrix, dtype=np.float64) would also work, but the astype is simpler.

I'm having some problems recreating your Matrix variable. What's its shape?

np.save is the best way of saving a multidimensional array to a file. There are other methods, but they don't save the shape and dtype information.

ndarray is wrong because the first (positional) argument is supposed to be the shape, not another array. The data if any is provided in the buffer parameter. ndarray is not normally used by beginners or even advanced numpy users.

What is Matrix supposed to be. When I try your code with a couple of parameters, I get an error in the map step:

In [495]: X = [np.random.randint(0, 2 ** 2 - 1)for _ in range(4)]
     ...: Matrix = np.asarray([list(map(int, list(x))) for x in X])
     ...: 
-----> 2 Matrix = np.asarray([list(map(int, list(x))) for x in X])
....
TypeError: 'int' object is not iterable
In [496]: X
Out[496]: [1, 2, 1, 0]

Is X just a list of numbers? Not some sort of list of arrays? And why the Matrix step? Is it trying to convert a nested list of lists into integers? Even though randint already creates integers? Then you follow with a conversion to float?

hpaulj
  • 221,503
  • 14
  • 230
  • 353