0

I'm trying to create a new list of data from some existing data. Currently though I am having the problem where each element of my new list is being saved as its own single-element list. I can't see why this is. My code is currently something like:

old_data = np.reshape(old_data, (49210, 1)) #Reshape into 1D array
new_data = [None] #Create empty list to be filled
max_value = np.nanmax(old_data)
threshold = 0.75 * max_value #Create threshold to be used as condition for new list.

for x in range(len(old_data)):
    if old_data[x] > threshold:
        new_data.append(old_data[x])

Please note that old_data was stored as a 2D array, which is why I have included that np.reshape command at the start.

As I said above, each time the condition is fulfilled the value is stored as a new element in new_data, but the element is an array of type float32 with size (1,). So new_data has ended up being an array of arrays. I'm not sure why this is. I would really just like a regular array/list output from this so that new_data is tractable. Any advice is appreciated.

imwellhowareyou
  • 49
  • 7
  • @JakobVinkas thank you, that's very useful for me to know as I am new to Python (clearly). I am actually getting the same result. I suspect then that my issue must be with how I have stored/structured `old_data`. I will investigate and report back. – imwellhowareyou Aug 14 '20 at 06:04

3 Answers3

0

I do not have the data but this might work:

old_data = np.reshape(old_data, (49210, 1)) #Reshape into 1D array
threshold = np.nanmax(old_data) #Create threshold to be used as condition for new 
new_data = [element for element in old_data if element > threshold]

List comprehension is both faster and much prettier to use instead of for loops with append.

In case you have ended up with a array of arrays you might want to try the following instead:

old_data = np.reshape(old_data, (49210, 1)) #Reshape into 1D array
threshold = np.nanmax(old_data) #Create threshold to be used as condition for new 
new_data = [element[0] for element in old_data if element[0] > threshold]
JakobVinkas
  • 1,003
  • 7
  • 23
0

So I see, you are using numpy library to flatten a 2D array to a 1D array.

What you can do is flatten your 2D array into a 1D array and then append your old list to your new list as you would like.

To flatten a 2D array to a 1D array -

import numpy as np 
  
ini_array1 = np.array([[1, 2, 3], [2, 4, 5], [1, 2, 3]]) 
   
# Multiplying arrays 
result = ini_array1.flatten() 
  
# printing result 
print("New resulting array: ", result)

The result will be your old 2D array as a normal 1D array or a python list which you can append to your new list to get your final list.

Innomight
  • 556
  • 3
  • 15
0

Thank you both @JakobVinkas and @Innomight. I've tried both solutions and both have worked. I'm still slightly confused as to why I was originally getting this problem but I will just read up on that I guess.

For anyone who may have come across this question later I have implemented what Jakob suggested using element[0] as my solution.

Also I have found this question and this question to be a good explanation of what is fundamentally happening here in the first place.

imwellhowareyou
  • 49
  • 7