numpy array add value

Question

I am trying to add new value into a numpy array with the following example

dict_created = {"A": [0,0], "B": [0,0], "C": [0,0], "D": [0,0], 
                "E": [0,0], "F": [0,0], "G": [0,0]} 

res_array = np.array(list(dict_created.items()))

now, I want to add "Q": [1,2] into each item expect the result like:

{"AQ": [1,2], "BQ": [1,2], "CQ": [1,2], "DQ": [1,2], "EQ": [1,2], "FQ": [1,2], "GQ": [1,2]}

I have tried with + sign but returned as [0, 0, 1, 2] for the value of each item, rather than add up with given values.

what is the correct method to return the expected result?

Thanks you

Your `numpy.array` object is a strange dtype=object array. Why not just use a `list`? IOW, **why are you working with numpy at all?** — juanpa.arrivillaga, Mar 20 '21 at 20:21
I don't quite understand why you expect Numpy to help here. Fundamentally, you are trying to create and then modify a dictionary. Numpy doesn't deal in those. You could use Numpy to represent and then update the *values* in that dict (which currently are lists, and could be Numpy arrays), but it's clearly overkill for lists that short. — Karl Knechtel, Mar 20 '21 at 20:21
Print `res_array` so both you and we have a clear idea what it is. — hpaulj, Mar 20 '21 at 20:32

Renan Lopes · Answer 1 · 2021-03-21T13:37:07.960

If I understood correctly you want to update both the key and values of a dictionary, this is what I thought (using numpy to add the lists instead of appending them):

import numpy as np

new_key = "Q"
new_value = [1,2]

result = dict((k+new_key, np.add(v, new_value)) for k, v in dict_created.items())

Expected output:

{'AQ': array([1, 2]), 'BQ': array([1, 2]), 'CQ': array([1, 2]), 'DQ': array([1, 2]), 'EQ': array([1, 2]), 'FQ': array([1, 2]), 'GQ': array([1, 2])}

You can also do the following (alternative way using dictionary comprehension):

result = {k+new_key:np.add(v, new_value) for k, v in dict_created.items()}

Edit:

According to your comment, the combination of keys and values would be in the order of billions. You shouldn't expect a fast solution even with numpy, but, in any case, Python dicts are well optimized.

You can check more about performance on the following questions:

Python dictionary vs list, which is faster?

Python lists/dictionaries vs. numpy arrays: performance vs. memory control

Why is this loop faster than a dictionary comprehension for creating a dictionary?

For the filtering and sorting examples I've modified the dict_created to this:

dict_created = {"A": [1,0], "B": [0,0], "C": [0,0], "D": [0,0], "E": [0,0], "F": [0,0], "G": [3,0]}
# Result
{'AQ': array([2, 2]), 'BQ': array([1, 2]), 'CQ': array([1, 2]), 'DQ': array([1, 2]), 'EQ': array([1, 2]), 'FQ': array([1, 2]), 'GQ': array([4, 2])}

Filtering

# Filter elements with x > 1
filtered_dict = {k:v for k, v in result.items() if v[0] > 1}
# {'AQ': array([2, 2]), 'GQ': array([4, 2])}

Sorting

# Sorts keys
sorted_keys = sorted(result)
# ['AQ', 'BQ', 'CQ', 'DQ', 'EQ', 'FQ', 'GQ']

# Sorting by dict keys
sorted_with_kv = {k:v for k, v in sorted(result.items())}
# {'AQ': array([2, 2]), 'BQ': array([1, 2]), 'CQ': array([1, 2]), 'DQ': array([1, 2]), 'EQ': array([1, 2]), 'FQ': array([1, 2]), 'GQ': array([4, 2])}

# Sorting by x given [x, y]
sorted_with_kv_lambda = {k:v for k, v in sorted(result.items(), key=lambda elem: elem[1][0])}
# {'BQ': array([1, 2]), 'CQ': array([1, 2]), 'DQ': array([1, 2]), 'EQ': array([1, 2]), 'FQ': array([1, 2]), 'AQ': array([2, 2]), 'GQ': array([4, 2])}

You should try to stick with dictionary comprehension since it yields the best performance when working with Python dictionaries.

For a more in-depth explanation, I recommend this reading: How Do Dictionaries and Sets Work

Thanks you. it works, just extent the question. for performance concern. the dict_created would be as huge as more than 100K, the new_key_value pair choice would be also another 10Kish... I am expecting to have the product of the combination of all of them then do some filtering and sorting. What would be the best approach for this circumstance? — user4603876, Mar 21 '21 at 06:02

hpaulj · Answer 2 · 2021-03-20T21:13:48.917

In [88]: dict_created = {"A": [0,0], "B": [0,0], "C": [0,0], "D": [0,0],
    ...:                 "E": [0,0], "F": [0,0], "G": [0,0]}
    ...: 
    ...: res_array = np.array(list(dict_created.items()))
<ipython-input-88-bc8f2c25c347>:4: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.

In [89]: res_array
Out[89]: 
array([['A', list([0, 0])],
       ['B', list([0, 0])],
       ['C', list([0, 0])],
       ['D', list([0, 0])],
       ['E', list([0, 0])],
       ['F', list([0, 0])],
       ['G', list([0, 0])]], dtype=object)

You have created an object dtype array containing letters and lists.

You weren't clear about the addition, but let's assume you did:

In [91]: res_array+list({"Q": [1,2]}.items())
<ipython-input-91-1bdaa5aac0d7>:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.

Out[91]: 
array([['AQ', list([0, 0, 1, 2])],
       ['BQ', list([0, 0, 1, 2])],
       ['CQ', list([0, 0, 1, 2])],
       ['DQ', list([0, 0, 1, 2])],
       ['EQ', list([0, 0, 1, 2])],
       ['FQ', list([0, 0, 1, 2])],
       ['GQ', list([0, 0, 1, 2])]], dtype=object)

The first column is strings, and the + is string concatenate.

The second is lists, which has a similar concatenate.

What you want array addition for the second column.

Let's turn that column into arrays:

In [93]: res_array[:,1]=[np.array(i) for i in res_array[:,1]]
In [94]: 
In [94]: res_array
Out[94]: 
array([['A', array([0, 0])],
       ['B', array([0, 0])],
       ['C', array([0, 0])],
       ['D', array([0, 0])],
       ['E', array([0, 0])],
       ['F', array([0, 0])],
       ['G', array([0, 0])]], dtype=object)
In [95]: 
In [95]: res_array+list({"Q": np.array([1,2])}.items())
<ipython-input-95-2703d0356b14>:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.

Out[95]: 
array([['AQ', array([1, 2])],
       ['BQ', array([1, 2])],
       ['CQ', array([1, 2])],
       ['DQ', array([1, 2])],
       ['EQ', array([1, 2])],
       ['FQ', array([1, 2])],
       ['GQ', array([1, 2])]], dtype=object)

I'm leaving the 'ragged array' warning that np version 1.19 started adding. You should be aware that you are creating a non-standard numpy array, and treat it appropriately. Making a numpy array from a dict, especially if you want both keys and values is a bit odd. numpy is best for arrays with uniform dtype - all character or all numbers. This mixed stuff is awkward.

Here is a non-numpy way of making a new dict:

In [97]: newdict= {key+'Q': [i+j for i,j in zip(value,[1,2])] for key, value in dict_created.items()}

In [98]: newdict
Out[98]: 
{'AQ': [1, 2],
 'BQ': [1, 2],
 'CQ': [1, 2],
 'DQ': [1, 2],
 'EQ': [1, 2],
 'FQ': [1, 2],
 'GQ': [1, 2]}

Cv4niak · Answer 3 · 2021-03-22T10:16:07.367

0

You can try to use this method:

new_item = {"Q":[1,2]}
new_item = np.array(list(new_item.items()))
res_array = np.concatenate((res_array, new_item))

edited Mar 22 '21 at 10:16

answered Mar 20 '21 at 21:05

Cv4niak

71
7

@user4603876 So sorry, I made a mistake while changin name from my sketch code. Try now. – Cv4niak Mar 22 '21 at 10:15

numpy array add value

3 Answers3

Edit:

Filtering

Sorting