4

I am trying to modify a list of two lists. For each of the two inside lists, I perform some operation and 'split' them into new lists.
Here is a simple example of what I'm trying to do:

[['a', 'b'], ['c', 'd']]  -->  [['a'], ['b'], ['c', 'd']]

Currently my algorithm passes ['a', 'b'] to a function that determines whether or not it should be split into [['a'], ['b']] (e.g. based on their correlations). The function returns [['a'], ['b']] which tells me that ['a', 'b'] should be split, or returns ['a', 'b'] (the original list) which indicates that it should not be split.

Currently I have something like this:

blist = [['a', 'b'], ['c', 'd']]   #big list
slist =  [['a'], ['b']]            #small list returned by function

nlist = [items for i in xrange(len(blist)) for items in (slist if i==0 else blist[i])]

This produces [['a'], ['b'], 'c', 'd'] as opposed to the desired output [['a'], ['b'], ['c', 'd']] which does not alter the second list in the original blist. I understand why this is happening--my second loop is also applied to blist[1] in this case, but I am not sure how to fix it as I do not understand list comprehension completely.

A 'pythonic' solution is preferred. Any feedback would be appreciated, thank you!

EDIT: Like the title suggests, I am trying to 'replace' ['a', 'b'] with ['a'], ['b']. So I would like the 'position' to be the same, having ['a'], ['b'] appear in the original list before ['c', 'd']

RESULTS Thank you Christian, Paul and schwobaseggl for your solutions! They all work :)

AsheKetchum
  • 1,098
  • 3
  • 14
  • 29
  • You are contradicting yourself. Are you passing items to a function or are you using a nested list comprehension? Anyway, if you don't understand list comprehensions, why don't you use simple for-loops until you do? I would. –  Feb 16 '17 at 15:24
  • Sorry, I meant that the function is used to help determine whether or not the list is splitted, the nested list comprehension performs the change/split – AsheKetchum Feb 16 '17 at 15:26
  • @hop I believe I understand simple for-loops. I would like to be more familiar with list comprehensions and have a better understanding. Just trying to get some practice. Also by 'I would' are you suggesting that only using simple for loops helps with understanding list comprehensions? I understand that they have similarities, but like in my example, the list comprehension seems to be a lot more compact with the statements in different orders. Could you elaborate more on the relationship between the two? – AsheKetchum Feb 16 '17 at 15:40

4 Answers4

2

Try

...  else [blist[i]])]

to create a list of lists.

Christian König
  • 3,437
  • 16
  • 28
1

You can use slice assignment:

>> l1 = [[1, 2], [3, 4]]
>>> l2 = [[1], [2]]
>>> l1[0:1] = l2
>>> l1
[[1], [2], [3, 4]]

This changes l1, so if you want to keep it make a copy before.

Another way that doesn't change l1 is addition:

>> l1 = [[1, 2], [3, 4]]
>>> l3 = l2 + l1[1:]
>>> l3
[[1], [2], [3, 4]]
Paul Panzer
  • 51,835
  • 3
  • 54
  • 99
  • in the case that `l2 = [1, 2]`, `l1[0:1] = l2[:]` returns l1 itself right? – AsheKetchum Feb 16 '17 at 15:31
  • 1
    no, that should put the elements of `l2` which are 1 and 2 in place of the element currently sitting at position zero of `l1` which is `[1, 2]`, in other words it loses the brackets around 1 and 2. – Paul Panzer Feb 16 '17 at 15:35
  • what does [:] actually do? Is it an iteration of elements? – AsheKetchum Feb 16 '17 at 15:42
  • 1
    It makes a so-called slice copy, it should be more or less equivalent to `.copy()`. Note that this is different in `numpy` where slicing doesn't create a copy but a view. – Paul Panzer Feb 16 '17 at 15:44
  • So the creation of the copy would require additional memory? – AsheKetchum Feb 16 '17 at 15:51
  • How does it compare in terms of memory consumption with `l1 = [items for i in xrange(len(l1)) for items in (l2 if i==0 else [l1[i]])]` Assuming that I am allowed to change l1 – AsheKetchum Feb 16 '17 at 15:53
  • 1
    You are not changing `l1` or rather the object bound to the name `l1` you are binding a new object (the product of your list comprehension) to `l1`. A list comprehension does produce a new list, so it needs to alllocate space for that. I don't really now, but I would guess list comprehensions are pretty efficient, that said if you are only changing one bit and copying all the rest over, then slice assignment migh be the better choice. Actually, the slice copy I'm using on the right hand side in the answer is not necessary. Sorry, I'll fix that. – Paul Panzer Feb 16 '17 at 16:01
  • 1
    I think the basic logic is `l1[0:1]` gives a copy of this part of `l1` as a list. So it should also work the other way round. if you assign to `l1[0:1]` it should accept a list, makee a copy and place the list at exactly the same spot. Note that this can change the length of the list. – Paul Panzer Feb 16 '17 at 16:08
  • Would the memory for the copy created by `l1[0:1]` be freed after the assignment? – AsheKetchum Feb 16 '17 at 16:16
  • 1
    I think if you use the syntax for assignment, no copy is made of this bit. Why should there be? The bit is not needed. However, since you are potentially changing the length of the list some comparatively expensive reorganisation may be necessary. I don't know the internals well enough to be more specific here. But these considerations really only are worthwhile if you are handling large lists maybe 100,000 elements or more or making lots of lists or looping over them lots of times. For garden variety lists you won't even notice. – Paul Panzer Feb 16 '17 at 16:23
  • Interesting! I am trying to recursively break down a list with about 300 variables. The entire data set is a pandas (300 x 80000 approximately) dataframe, so not quite as big as you mentioned. Luckily I can pass only the column names instead of the entire columns for my break down algorithm, which makes the size even smaller. I think I have a good idea of what I need to do now. Thank you so much for your time and kindness :) – AsheKetchum Feb 16 '17 at 16:32
1

You could alter your split function to return structurally adequate lists. Then you can use a comprehension:

def split_or_not(l):
  if condition: # split
    return [l[:1], l[1:]]
  return [l]  # wrap in extra list

# using map
nlist = [x for sub_l in map(split_or_not, blist) for x in sub_l]
# or nested comprehension
nlist = [x for sub_l in (split_or_not(l) for l in blist) for x in sub_l]
user2390182
  • 72,016
  • 6
  • 67
  • 89
  • That is a very good point! I upvoted but it doesn't show since I have less than 15 reputation haha. I will modify the return object structures. In addition, what happens when `[l]` is returned instead of `l`? Does it require any additional time to put `l` into a list? – AsheKetchum Feb 16 '17 at 15:59
  • 1
    The new list only holds a reference to the list, the overhead should be negligible! – user2390182 Feb 16 '17 at 16:16
  • If you use a `list.extend()` and a normal for-loop, you can reduce the complexity of the code considerably. Nested list comprehensions should be avoided (speaking of "pythonic"). –  Feb 16 '17 at 16:28
  • @hop Do you have a source for that last statement. To my knowledge, nested comprehensions are performant in comparison to other methods, see e.g. [http://stackoverflow.com/questions/952914/making-a-flat-list-out-of-list-of-lists-in-python](http://stackoverflow.com/questions/952914/making-a-flat-list-out-of-list-of-lists-in-python) – user2390182 Feb 16 '17 at 16:56
  • @schwobaseggl: what, like a court decision? if you want, you can quote me. readability trumps performance most of the time. –  Feb 16 '17 at 17:21
  • 1
    @hop While I (mostly) agree with readability > performance, I don't think any of the examples in this answer suffer from poor readability. I will admit that being used to it has sth. to do with it, but still more a matter of taste than Pythonicity ;) – user2390182 Feb 16 '17 at 17:27
  • @schwobaseggl: can you honestly say that my solution is as complex as yours? –  Feb 16 '17 at 17:32
  • @hop I understand schwobaseggl's solution but I can not say I understand yours since you refused to elaborate. I am not one to judge which is more complex since I do not know what happens internally for both solutions, but in terms of readability, schwobaseggl has definitely provided a better presentation of his solution. – AsheKetchum Feb 16 '17 at 17:39
0

Assuming you have the mentioned funtion that decides whether to split an item:

def munch(item):
    if item[0] == 'a': # split
        return [[item[0]], [item[1]]]
    return [item] # don't split

You can use it in s simple for-loop.

nlist = []
for item in blist:
    nlist.extend(munch(item))

"Pythonic" is whatever is easy to read and understand. Don't use list comprehensions just because you can.

  • could you provide a brief explanation of what munch does here? An example would be really helpful. Thank you! – AsheKetchum Feb 16 '17 at 15:59
  • @AsheKetchum: You can easily put the code into the Python command line and try it for yourself. –  Feb 16 '17 at 16:23