3

I tried to run the code below, but takes a long time, is there any way to improve this code? I tried with a lot of python functions but all look like are created to work with 1D arrays only, sorry for the code I am very new on this:

Function that returns an array that contains unique elements:

def get_unique_values(dataset):
    uniqueimages=np.empty(dataset.shape, dtype=np.float32)
    u=0

    for i in range(dataset.shape[0]):
        if (check_if_exists(dataset[i,:,:],uniqueimages)==0):
            uniqueimages[u,:,:]=dataset[i,:,:]
            u=u+1

    return uniqueimages[0:u,:,:]

function that verifies if a given element is already in the array:

def check_if_exists( q,a ):
    r=0
    for i in range(a.shape[0]):
        if np.array_equal(q,a[i,:,:]):
            r=1
            break

    return r

then I run this giving a 3D array as input (shape= 2000,28,28):

vadu=get_unique_values(3Darray)

EDIT: this question was solved in the other linked question, thanks Divakar!

Piero G.
  • 41
  • 3
  • 1
    This is very very broad. You can use np.unique() on some flattened view or hash and work on hashes. But that decision is hard to do blindly. – sascha Aug 30 '17 at 23:50
  • Thanks Sascha, I tried with np.unique but it compares the 2D arrays not as a whole element but by its internal numbers and the comparison is not accurate. Think that function is more useful with 1D arrays. Will investigate about Hashes and see what can i find – Piero G. Aug 30 '17 at 23:55
  • Again too broad. You should mention if those might differ by shapes and size and if this is relevant here. We don't know that. (and is not accurate sounds strange; this is fully deterministic and correct; but of course it might not be doing what you think it's doing) – sascha Aug 30 '17 at 23:55
  • ok, the 3D array is an array of images, so 2D arrays are the elements inside. When I run np.unique it is converting my 3D array to a 1D array with shape (256,). I suponse as the 2D arrays are representing images they contain values only from 0 to 256 and the np.unique is comparing only the numbers not the 2D elements as a whole. Thats why I build the functions. – Piero G. Aug 31 '17 at 00:07
  • Then start by reading the docs, especially the argument: ```axis```. – sascha Aug 31 '17 at 00:09
  • Yes, there is no 'axis' parameter in python3 for np.unique. Documentation says that "Parameter: Input array - This will be flattened if it is not already 1-D." Don't know how to fix this really. – Piero G. Aug 31 '17 at 00:17
  • The `axis` argument was added to `unique` in numpy 1.13.0. – Warren Weckesser Aug 31 '17 at 00:43
  • Thanks all!, as marked by Divakar, this question was perfectly answered in the linked question. But don't have a way to mark this as solved with a linked question. – Piero G. Sep 01 '17 at 00:54

1 Answers1

1

If you want to get the unique elements from a numpy array, you can use the numpy unique function.

import numpy as np

a = np.array([1,2,2,3,4,2,4])
np.unique(a)
array([1, 2, 3, 4])
James
  • 32,991
  • 4
  • 47
  • 70
  • I tried that I I got the 3D array converted to a 1D array. >>trd.shape (200000, 28, 28) >>newarray=np.unique(trd) >>v.shape (256,) I want to tell that the 3D array contains images as 2D arrays, so the values are from 0 to 256 each element. Dont know why is converting the original array to a 1D array. Thats why I built the functions above – Piero G. Aug 31 '17 at 00:03