1

The following code is part of a function but the problem I have is on these lines.

# Global variables
NUM_MECS          = 2
MAX_MEC_CAPACITY  = 2
VNF_TYPES         = 2

# Mec capacity to hold a vnf
served_vnfs = list(range(MAX_MEC_CAPACITY+1)) # 0 does not  count as capacity
# All possible mec states as far as holded vnfs
mec_capacity_states = [list(s) for s in itertools.product(served_vnfs, repeat=VNF_TYPES)]
# All possible states with defined number of mecs
mecs_states = [list(p) for p in itertools.product(mec_capacity_states, repeat=NUM_MECS)]

and this generates a list of lists as:

out: [[[0, 0], [0, 0]], [[0, 0], [0, 1]], [[0, 0], [0, 2]], [[0, 0], [1, 0]], [[0, 0], [1, 1]], [[0, 0], [1, 2]], [[0, 0], [2, 0]], [[0, 0], [2, 1]], [[0, 0], [2, 2]], [[0, 1], [0, 0]], [[0, 1], [0, 1]], [[0, 1], [0, 2]], [[0, 1], [1, 0]], [[0, 1], [1, 1]], [[0, 1], [1, 2]], [[0, 1], [2, 0]], [[0, 1], [2, 1]], [[0, 1], [2, 2]], [[0, 2], [0, 0]], [[0, 2], [0, 1]], [[0, 2], [0, 2]], [[0, 2], [1, 0]], [[0, 2], [1, 1]], [[0, 2], [1, 2]], [[0, 2], [2, 0]], [[0, 2], [2, 1]], [[0, 2], [2, 2]], [[1, 0], [0, 0]], [[1, 0], [0, 1]], [[1, 0], [0, 2]], [[1, 0], [1, 0]], [[1, 0], [1, 1]], [[1, 0], [1, 2]], [[1, 0], [2, 0]], [[1, 0], [2, 1]], [[1, 0], [2, 2]], [[1, 1], [0, 0]], [[1, 1], [0, 1]], [[1, 1], [0, 2]], [[1, 1], [1, 0]], [[1, 1], [1, 1]], [[1, 1], [1, 2]], [[1, 1], [2, 0]], [[1, 1], [2, 1]], [[1, 1], [2, 2]], [[1, 2], [0, 0]], [[1, 2], [0, 1]], [[1, 2], [0, 2]], [[1, 2], [1, 0]], [[1, 2], [1, 1]], [[1, 2], [1, 2]], [[1, 2], [2, 0]], [[1, 2], [2, 1]], [[1, 2], [2, 2]], [[2, 0], [0, 0]], [[2, 0], [0, 1]], [[2, 0], [0, 2]], [[2, 0], [1, 0]], [[2, 0], [1, 1]], [[2, 0], [1, 2]], [[2, 0], [2, 0]], [[2, 0], [2, 1]], [[2, 0], [2, 2]], [[2, 1], [0, 0]], [[2, 1], [0, 1]], [[2, 1], [0, 2]], [[2, 1], [1, 0]], [[2, 1], [1, 1]], [[2, 1], [1, 2]], [[2, 1], [2, 0]], [[2, 1], [2, 1]], [[2, 1], [2, 2]], [[2, 2], [0, 0]], [[2, 2], [0, 1]], [[2, 2], [0, 2]], [[2, 2], [1, 0]], [[2, 2], [1, 1]], [[2, 2], [1, 2]], [[2, 2], [2, 0]], [[2, 2], [2, 1]], [[2, 2], [2, 2]]]

If for example I want to modify only the second value in the second item from the second list:

print(mecs_states[1], '-', mecs_states[1][1], '-', mecs_states[1][1][1])

mecs_states[1][1][1] = 'A'

Is not only changing the desired value but also other lists values in the same index position.

out: [[[0, 0], [0, 0]], [[0, 0], [0, 'A']], [[0, 0], [0, 2]], [[0, 0], [1, 0]], [[0, 0], [1, 1]], [[0, 0], [1, 2]], [[0, 0], [2, 0]], [[0, 0], [2, 1]], [[0, 0], [2, 2]], [[0, 'A'], [0, 0]], [[0, 'A'], [0, 'A']], [[0, 'A'], [0, 2]], [[0, 'A'], [1, 0]], [[0, 'A'], [1, 1]], [[0, 'A'], [1, 2]], [[0, 'A'], [2, 0]], [[0, 'A'], [2, 1]], [[0, 'A'], [2, 2]], [[0, 2], [0, 0]], [[0, 2], [0, 'A']], [[0, 2], [0, 2]], [[0, 2], [1, 0]], [[0, 2], [1, 1]], [[0, 2], [1, 2]], [[0, 2], [2, 0]], [[0, 2], [2, 1]], [[0, 2], [2, 2]], [[1, 0], [0, 0]], [[1, 0], [0, 'A']], [[1, 0], [0, 2]], [[1, 0], [1, 0]], [[1, 0], [1, 1]], [[1, 0], [1, 2]], [[1, 0], [2, 0]], [[1, 0], [2, 1]], [[1, 0], [2, 2]], [[1, 1], [0, 0]], [[1, 1], [0, 'A']], [[1, 1], [0, 2]], [[1, 1], [1, 0]], [[1, 1], [1, 1]], [[1, 1], [1, 2]], [[1, 1], [2, 0]], [[1, 1], [2, 1]], [[1, 1], [2, 2]], [[1, 2], [0, 0]], [[1, 2], [0, 'A']], [[1, 2], [0, 2]], [[1, 2], [1, 0]], [[1, 2], [1, 1]], [[1, 2], [1, 2]], [[1, 2], [2, 0]], [[1, 2], [2, 1]], [[1, 2], [2, 2]], [[2, 0], [0, 0]], [[2, 0], [0, 'A']], [[2, 0], [0, 2]], [[2, 0], [1, 0]], [[2, 0], [1, 1]], [[2, 0], [1, 2]], [[2, 0], [2, 0]], [[2, 0], [2, 1]], [[2, 0], [2, 2]], [[2, 1], [0, 0]], [[2, 1], [0, 'A']], [[2, 1], [0, 2]], [[2, 1], [1, 0]], [[2, 1], [1, 1]], [[2, 1], [1, 2]], [[2, 1], [2, 0]], [[2, 1], [2, 1]], [[2, 1], [2, 2]], [[2, 2], [0, 0]], [[2, 2], [0, 'A']], [[2, 2], [0, 2]], [[2, 2], [1, 0]], [[2, 2], [1, 1]], [[2, 2], [1, 2]], [[2, 2], [2, 0]], [[2, 2], [2, 1]], [[2, 2], [2, 2]]]

What am I missing? I need that each list within a list be independent. I'm guessing that the reason is in the product of itertools but I don't totally understand why.

What would be the solution?

krm76
  • 357
  • 3
  • 13
  • Does this answer your question? [List changes unexpectedly after assignment. How do I clone or copy it to prevent this?](https://stackoverflow.com/questions/2612802/list-changes-unexpectedly-after-assignment-how-do-i-clone-or-copy-it-to-prevent) – 0x5453 Mar 03 '21 at 19:54
  • For a simple example of what is happening you can do this: a = [1, 2, 3]; b = a; b[1] = 'A'; print (b); print (a) – Will.Evo Mar 03 '21 at 19:56

3 Answers3

3

When you build the mecs_states list, you’re not creating new internal lists, you’re referencing the same lists you created in mec_capacity_states. As a smaller example:

import itertools

nums = [1, 2]

pairs = [list(p) for p in itertools.product(nums, repeat=2)]
# pairs = [[1, 1], [1, 2], [2, 1], [2, 2]]
a, b, c, d = pairs  # unpacking 'pairs' for the example later.

nested = [list(p) for p in itertools.product(pairs, repeat=2)]
# nested = [
#     [[1, 1], [1, 1]],
#     [[1, 1], [1, 2]],
#     [[1, 1], [2, 1]],
#     [[1, 1], [2, 2]],
# ...]

The thing about the inner elements of nested is that they are the same lists, by reference, as the elements of pairs:

nested[0][0] is a
# True
nested[0][1] is a
# True

nested[0][0][0] = 'x'
# a = ['x', 1]
# nested = [
#     [['x', 1], ['x', 1]],
#     [['x', 1], ['x', 2]],
#     [['x', 1], [2, 1]],
#     [['x', 1], [2, 2]],
# ...]

So when you modify nested[0][0][0], that is modifying the list that both nested[0][0] and a point to. Essentially, it’s a slightly more complex version of the problem people run into when they do [[]] * 10, and all ten lists are the same list in memory.

You could solve this by waiting until the very end, after you have constructed your collections, to turn the tuples that itertools.product gives you into lists. However, once you are getting into nested collections more than a couple levels deep, it is time to start thinking about whether a class or some other data structure might be more appropriate to encapsulate the behavior you want.

Paul Fisher
  • 9,618
  • 5
  • 37
  • 53
  • thanks for the explanation, I was not aware that itertools.product would just point to the same memory allocation. The truth is that is a great function to obtain all possible combinations between vectors, that's why I used it at first place. – krm76 Mar 03 '21 at 20:11
2

Just "clone" the mecs_states list item. See copy.deepcopy in the code

import itertools
import copy

NUM_MECS = 2
MAX_MEC_CAPACITY = 2
VNF_TYPES = 2

# Mec capacity to hold a vnf
served_vnfs = list(range(MAX_MEC_CAPACITY + 1))  # 0 does not  count as capacity
# All possible mec states as far as holded vnfs
mec_capacity_states = [list(s) for s in itertools.product(served_vnfs, repeat=VNF_TYPES)]
# All possible states with defined number of mecs
mecs_states = [copy.deepcopy(list(p)) for p in itertools.product(mec_capacity_states, repeat=NUM_MECS)]
print(mecs_states)
print(mecs_states[1], '-', mecs_states[1][1], '-', mecs_states[1][1][1])
mecs_states[1][1][1] = 'A'
print(mecs_states)

output

[[[0, 0], [0, 0]], [[0, 0], [0, 1]], [[0, 0], [0, 2]], [[0, 0], [1, 0]], [[0, 0], [1, 1]], [[0, 0], [1, 2]], [[0, 0], [2, 0]], [[0, 0], [2, 1]], [[0, 0], [2, 2]], [[0, 1], [0, 0]], [[0, 1], [0, 1]], [[0, 1], [0, 2]], [[0, 1], [1, 0]], [[0, 1], [1, 1]], [[0, 1], [1, 2]], [[0, 1], [2, 0]], [[0, 1], [2, 1]], [[0, 1], [2, 2]], [[0, 2], [0, 0]], [[0, 2], [0, 1]], [[0, 2], [0, 2]], [[0, 2], [1, 0]], [[0, 2], [1, 1]], [[0, 2], [1, 2]], [[0, 2], [2, 0]], [[0, 2], [2, 1]], [[0, 2], [2, 2]], [[1, 0], [0, 0]], [[1, 0], [0, 1]], [[1, 0], [0, 2]], [[1, 0], [1, 0]], [[1, 0], [1, 1]], [[1, 0], [1, 2]], [[1, 0], [2, 0]], [[1, 0], [2, 1]], [[1, 0], [2, 2]], [[1, 1], [0, 0]], [[1, 1], [0, 1]], [[1, 1], [0, 2]], [[1, 1], [1, 0]], [[1, 1], [1, 1]], [[1, 1], [1, 2]], [[1, 1], [2, 0]], [[1, 1], [2, 1]], [[1, 1], [2, 2]], [[1, 2], [0, 0]], [[1, 2], [0, 1]], [[1, 2], [0, 2]], [[1, 2], [1, 0]], [[1, 2], [1, 1]], [[1, 2], [1, 2]], [[1, 2], [2, 0]], [[1, 2], [2, 1]], [[1, 2], [2, 2]], [[2, 0], [0, 0]], [[2, 0], [0, 1]], [[2, 0], [0, 2]], [[2, 0], [1, 0]], [[2, 0], [1, 1]], [[2, 0], [1, 2]], [[2, 0], [2, 0]], [[2, 0], [2, 1]], [[2, 0], [2, 2]], [[2, 1], [0, 0]], [[2, 1], [0, 1]], [[2, 1], [0, 2]], [[2, 1], [1, 0]], [[2, 1], [1, 1]], [[2, 1], [1, 2]], [[2, 1], [2, 0]], [[2, 1], [2, 1]], [[2, 1], [2, 2]], [[2, 2], [0, 0]], [[2, 2], [0, 1]], [[2, 2], [0, 2]], [[2, 2], [1, 0]], [[2, 2], [1, 1]], [[2, 2], [1, 2]], [[2, 2], [2, 0]], [[2, 2], [2, 1]], [[2, 2], [2, 2]]]
[[0, 0], [0, 1]] - [0, 1] - 1
[[[0, 0], [0, 0]], [[0, 0], [0, 'A']], [[0, 0], [0, 2]], [[0, 0], [1, 0]], [[0, 0], [1, 1]], [[0, 0], [1, 2]], [[0, 0], [2, 0]], [[0, 0], [2, 1]], [[0, 0], [2, 2]], [[0, 1], [0, 0]], [[0, 1], [0, 1]], [[0, 1], [0, 2]], [[0, 1], [1, 0]], [[0, 1], [1, 1]], [[0, 1], [1, 2]], [[0, 1], [2, 0]], [[0, 1], [2, 1]], [[0, 1], [2, 2]], [[0, 2], [0, 0]], [[0, 2], [0, 1]], [[0, 2], [0, 2]], [[0, 2], [1, 0]], [[0, 2], [1, 1]], [[0, 2], [1, 2]], [[0, 2], [2, 0]], [[0, 2], [2, 1]], [[0, 2], [2, 2]], [[1, 0], [0, 0]], [[1, 0], [0, 1]], [[1, 0], [0, 2]], [[1, 0], [1, 0]], [[1, 0], [1, 1]], [[1, 0], [1, 2]], [[1, 0], [2, 0]], [[1, 0], [2, 1]], [[1, 0], [2, 2]], [[1, 1], [0, 0]], [[1, 1], [0, 1]], [[1, 1], [0, 2]], [[1, 1], [1, 0]], [[1, 1], [1, 1]], [[1, 1], [1, 2]], [[1, 1], [2, 0]], [[1, 1], [2, 1]], [[1, 1], [2, 2]], [[1, 2], [0, 0]], [[1, 2], [0, 1]], [[1, 2], [0, 2]], [[1, 2], [1, 0]], [[1, 2], [1, 1]], [[1, 2], [1, 2]], [[1, 2], [2, 0]], [[1, 2], [2, 1]], [[1, 2], [2, 2]], [[2, 0], [0, 0]], [[2, 0], [0, 1]], [[2, 0], [0, 2]], [[2, 0], [1, 0]], [[2, 0], [1, 1]], [[2, 0], [1, 2]], [[2, 0], [2, 0]], [[2, 0], [2, 1]], [[2, 0], [2, 2]], [[2, 1], [0, 0]], [[2, 1], [0, 1]], [[2, 1], [0, 2]], [[2, 1], [1, 0]], [[2, 1], [1, 1]], [[2, 1], [1, 2]], [[2, 1], [2, 0]], [[2, 1], [2, 1]], [[2, 1], [2, 2]], [[2, 2], [0, 0]], [[2, 2], [0, 1]], [[2, 2], [0, 2]], [[2, 2], [1, 0]], [[2, 2], [1, 1]], [[2, 2], [1, 2]], [[2, 2], [2, 0]], [[2, 2], [2, 1]], [[2, 2], [2, 2]]]
balderman
  • 22,927
  • 7
  • 34
  • 52
  • Your suggestion is not working properly, if you try to change mecs_states[0][0][0]='A' you'll see that it duplicates de modification – krm76 Mar 07 '21 at 22:40
0

Using json to serialize and deserialize.

import json

# Mec capacity to hold a vnf
served_vnfs = list(range(MAX_MEC_CAPACITY + 1))  # 0 does not  count as capacity
# All possible mec states as far as holded vnfs
mec_capacity_states = [list(s) for s in itertools.product(served_vnfs, repeat=VNF_TYPES)]
# All possible states with defined number of mecs
mecs_states = [copy.deepcopy(list(p)) for p in 
itertools.product(mec_capacity_states, repeat=NUM_MECS)]
mecs_states = json.loads(json.dumps(list(mecs_states)))

print(mecs_states)
mecs_states[0][0][0] = 'A'
print(mecs_states)

output

[[[0, 0], [0, 0]], [[0, 0], [0, 1]], [[0, 0], [0, 2]], [[0, 0], [1, 0]], [[0, 0], [1, 1]], [[0, 0], [1, 2]], [[0, 0], [2, 0]], [[0, 0], [2, 1]], [[0, 0], [2, 2]], [[0, 1], [0, 0]], [[0, 1], [0, 1]], [[0, 1], [0, 2]], [[0, 1], [1, 0]], [[0, 1], [1, 1]], [[0, 1], [1, 2]], [[0, 1], [2, 0]], [[0, 1], [2, 1]], [[0, 1], [2, 2]], [[0, 2], [0, 0]], [[0, 2], [0, 1]], [[0, 2], [0, 2]], [[0, 2], [1, 0]], [[0, 2], [1, 1]], [[0, 2], [1, 2]], [[0, 2], [2, 0]], [[0, 2], [2, 1]], [[0, 2], [2, 2]], [[1, 0], [0, 0]], [[1, 0], [0, 1]], [[1, 0], [0, 2]], [[1, 0], [1, 0]], [[1, 0], [1, 1]], [[1, 0], [1, 2]], [[1, 0], [2, 0]], [[1, 0], [2, 1]], [[1, 0], [2, 2]], [[1, 1], [0, 0]], [[1, 1], [0, 1]], [[1, 1], [0, 2]], [[1, 1], [1, 0]], [[1, 1], [1, 1]], [[1, 1], [1, 2]], [[1, 1], [2, 0]], [[1, 1], [2, 1]], [[1, 1], [2, 2]], [[1, 2], [0, 0]], [[1, 2], [0, 1]], [[1, 2], [0, 2]], [[1, 2], [1, 0]], [[1, 2], [1, 1]], [[1, 2], [1, 2]], [[1, 2], [2, 0]], [[1, 2], [2, 1]], [[1, 2], [2, 2]], [[2, 0], [0, 0]], [[2, 0], [0, 1]], [[2, 0], [0, 2]], [[2, 0], [1, 0]], [[2, 0], [1, 1]], [[2, 0], [1, 2]], [[2, 0], [2, 0]], [[2, 0], [2, 1]], [[2, 0], [2, 2]], [[2, 1], [0, 0]], [[2, 1], [0, 1]], [[2, 1], [0, 2]], [[2, 1], [1, 0]], [[2, 1], [1, 1]], [[2, 1], [1, 2]], [[2, 1], [2, 0]], [[2, 1], [2, 1]], [[2, 1], [2, 2]], [[2, 2], [0, 0]], [[2, 2], [0, 1]], [[2, 2], [0, 2]], [[2, 2], [1, 0]], [[2, 2], [1, 1]], [[2, 2], [1, 2]], [[2, 2], [2, 0]], [[2, 2], [2, 1]], [[2, 2], [2, 2]]]

[[['A', 0], [0, 0]], [[0, 0], [0, 1]], [[0, 0], [0, 2]], [[0, 0], [1, 0]], [[0, 0], [1, 1]], [[0, 0], [1, 2]], [[0, 0], [2, 0]], [[0, 0], [2, 1]], [[0, 0], [2, 2]], [[0, 1], [0, 0]], [[0, 1], [0, 1]], [[0, 1], [0, 2]], [[0, 1], [1, 0]], [[0, 1], [1, 1]], [[0, 1], [1, 2]], [[0, 1], [2, 0]], [[0, 1], [2, 1]], [[0, 1], [2, 2]], [[0, 2], [0, 0]], [[0, 2], [0, 1]], [[0, 2], [0, 2]], [[0, 2], [1, 0]], [[0, 2], [1, 1]], [[0, 2], [1, 2]], [[0, 2], [2, 0]], [[0, 2], [2, 1]], [[0, 2], [2, 2]], [[1, 0], [0, 0]], [[1, 0], [0, 1]], [[1, 0], [0, 2]], [[1, 0], [1, 0]], [[1, 0], [1, 1]], [[1, 0], [1, 2]], [[1, 0], [2, 0]], [[1, 0], [2, 1]], [[1, 0], [2, 2]], [[1, 1], [0, 0]], [[1, 1], [0, 1]], [[1, 1], [0, 2]], [[1, 1], [1, 0]], [[1, 1], [1, 1]], [[1, 1], [1, 2]], [[1, 1], [2, 0]], [[1, 1], [2, 1]], [[1, 1], [2, 2]], [[1, 2], [0, 0]], [[1, 2], [0, 1]], [[1, 2], [0, 2]], [[1, 2], [1, 0]], [[1, 2], [1, 1]], [[1, 2], [1, 2]], [[1, 2], [2, 0]], [[1, 2], [2, 1]], [[1, 2], [2, 2]], [[2, 0], [0, 0]], [[2, 0], [0, 1]], [[2, 0], [0, 2]], [[2, 0], [1, 0]], [[2, 0], [1, 1]], [[2, 0], [1, 2]], [[2, 0], [2, 0]], [[2, 0], [2, 1]], [[2, 0], [2, 2]], [[2, 1], [0, 0]], [[2, 1], [0, 1]], [[2, 1], [0, 2]], [[2, 1], [1, 0]], [[2, 1], [1, 1]], [[2, 1], [1, 2]], [[2, 1], [2, 0]], [[2, 1], [2, 1]], [[2, 1], [2, 2]], [[2, 2], [0, 0]], [[2, 2], [0, 1]], [[2, 2], [0, 2]], [[2, 2], [1, 0]], [[2, 2], [1, 1]], [[2, 2], [1, 2]], [[2, 2], [2, 0]], [[2, 2], [2, 1]], [[2, 2], [2, 2]]]
krm76
  • 357
  • 3
  • 13