46

Looking for a way to reliably identify if a numpy object is a view.

Related questions have come up many times before (here, here, here), and people have offered some solutions, but all seem to have problems:

  • The test used in pandas now is to call something a view if my_array.base is not None. This seems to always catch views, but also offers lots of false positives (situations where it reports something is a view even if it isn't).
  • numpy.may_share_memory() will check for two specific arrays, but won't answer generically
    • (@RobertKurn says was best tool as of 2012 -- any changes?)
  • flags['OWNDATA']) is reported (third comment first answer) to fail in some cases.

(The reason for my interest is that I'm working on implementing copy-on-write for pandas, and a conservative indicator is leading to over-copying.)

Community
  • 1
  • 1
nick_eu
  • 3,541
  • 5
  • 24
  • 38
  • 4
    relevant discussion on the numpy github tracker: https://github.com/numpy/numpy/issues/5604 – ali_m Nov 04 '15 at 20:00
  • 3
    The situations described as false positives seem to be when an operation that "creates a copy" actually returns a view of a copy. If this is to be considered "not a view", you might try checking the refcount of the array's `base` to see whether there are any other references to it, but that won't be perfect. I don't think it's possible to reliably detect this kind of thing. – user2357112 Nov 04 '15 at 20:05
  • 7
    Also see this recent SO question, http://stackoverflow.com/questions/33467477/how-to-find-all-variables-with-identical-id. It mentions a new function, `np.shares_memory`. But I streesed there that it's one thing to compare 2 known variables, quite another to ask is an array shares the data buffer with any other array. – hpaulj Nov 04 '15 at 20:11
  • 1
    How about an numpy array a view onto which is created later, how do you want to treat that original array? – Dima Tisnek Jan 18 '16 at 10:37
  • 1
    For my purposes, original is ok to treat as "not a view". – nick_eu Jan 20 '16 at 19:48
  • My five cents: it is pointless to have a view with offset zero, default strides and a size computed from shape and itemsize that's equal to the size of the buffer. So anything having zero offset, default strides and a shape that matches the buffer is probably no view. Default strides given by: strides = [itemsize]; for dim in reversed (shape [1 : ]): strides.insert (0, strides [0] * dim) – Jacques de Hooge Mar 25 '16 at 13:14

1 Answers1

1

Depending on your usages, flags['OWNDATA'] would do the job. In fact, there's no problem with your link. It does not fail in some cases. It will always do what it's supposed to do.

According to http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.require.html: the flag "ensure an array that owns its own data".

In your "counterexample", they use the code: 

print (b.flags['OWNDATA'])  #False -- apparently this is a view
e = np.ravel(b[:, 2])
print (e.flags['OWNDATA'])  #True -- Apparently this is a new numpy object.

But, it's the normal behaviour to be True in the second case.

It comes from the definition of ravel (from http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.ravel.html).

Return a contiguous flattened array. A 1-D array, containing the elements of the input, is returned. A copy is made only if needed.

Here, a copy is needed, so a copy is made. So, the variable e really owns its own data. It's not a "view of b", "a reference to b", "an alias to a part of b". It's a real new array that contains a copy of some elements of b.

So, I think that it's impossible without tracking the entire origin of the data to detect that kind of behaviour. I believe you should be able to build your program with that flag.

Alexis Clarembeau
  • 2,776
  • 14
  • 27