Merging a list of lists

Question

How do I merge a list of lists?

[['A', 'B', 'C'], ['D', 'E', 'F'], ['G', 'H', 'I']]

into

['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I']

Even better if I can add a value on the beginning and end of each item before merging the lists, like html tags.

i.e., the end result would be:

['<tr>A</tr>', '<tr>B</tr>', '<tr>C</tr>', '<tr>D</tr>', '<tr>E</tr>', '<tr>F</tr>', '<tr>G</tr>', '<tr>H</tr>', '<tr>I</tr>']

Is there anyway to add the tags before the lists are merged? I sorta simplified my problem. — Steven Matthews, Oct 25 '11 at 20:34
unsimplify your problem. I don't see how can we say how to add the tags without knowing what restrictions prevent you from using the obvious solution — Winston Ewert, Oct 25 '11 at 21:27

score 10 · Answer 1 · answered Oct 25 '11 at 20:36

Don't use sum(), it is slow for joining lists.

Instead a nested list comprehension will work:

>>> x = [['A', 'B', 'C'], ['D', 'E', 'F'], ['G', 'H', 'I']]
>>> [elem for sublist in x for elem in sublist]
['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I']
>>> ['<tr>' + elem + '</tr>' for elem in _]

The advice to use itertools.chain was also good.

Winston Ewert · Answer 2 · 2011-10-25T20:35:56.560

4

import itertools

print [('<tr>%s</tr>' % x) for x in itertools.chain.from_iterable(l)]

You can use sum, but I think that is kinda ugly because you have to pass the [] parameter. As Raymond points out, it will also be expensive. So don't use sum.

edited Oct 25 '11 at 20:35

answered Oct 25 '11 at 20:30

Winston Ewert

44,070
10
68
83

2

The itertools advice is good, but the sum() advice isn't. The latter is an O(n*n) operation :-( – Raymond Hettinger Oct 25 '11 at 20:33
@Raymond: Would it be possible to modify the `+` operator for lists to work in place if the left operand has reference count 1? Would this reduce the complexity of the `sum()` version to O(n)? – Sven Marnach Oct 25 '11 at 20:59
2

Sven, that might work though I don't think Guido would be interested in supporting it -- he already rejected ''.join() style logic for summing strings with sum(). – Raymond Hettinger Oct 25 '11 at 23:13
@SvenMarnach, actually the comments in the implemention of sum mention the possibility of doing inplace addition precisely to eliminate the quadratic behavior here. It states the the optimization is not actually done because it would end up modifying the second parameter to sum. – Winston Ewert Oct 26 '11 at 01:26
@Winston: I wouldn't do it in `sum()`, I'd do it in the `+` operator for lists, and only if the reference count of the list is 1. This would mean that `sum(list_of_lists, [])` is optimised -- we are allowed to modify `sum()`'s second parameter here -- while `a = []; sum(list_of_lists, a)` is not optimised -- we are not allowed to modify `a`. – Sven Marnach Oct 26 '11 at 10:45
@SvenMarnach, wouldn't work. Firstly, sum adds a reference to the second parameter so it would actually have two references on it when invoking PyNumber_Add. Secondly, for a function written in C, it may store an object with reference count in a local variable and then pass that argument to PyNumber_Add. But its not safe to reuse that object even though it only has a single reference. Python does have special cases for operator +, but they are implemented in the python opcode loop, and don't apply to functions written in C. – Winston Ewert Oct 26 '11 at 15:11
@Winston: The first point is not that relevant -- the implementation of `sum()` could easily be modified not to increase the reference count of the second parameter. Your second point is true, though it doesn't mean this "wouldn't work", it only means it would change the behaviour of the C interface (in a way that is very unlikely to affect a lot of code, but it doesn't seem to be worth it anyway.) – Sven Marnach Oct 26 '11 at 17:28
@SvenMarnach, removing the extra increment isn't trivial, although yes, it could be done. I'd expect that change to the C interface to break a lot of code. It would also be a deeply counter-intuitive bug to try and locate if it did happen. – Winston Ewert Oct 26 '11 at 17:54

score 2 · Answer 3 · edited May 23 '17 at 12:03

Use itertools.chain:

>>> import itertools
>>> list(itertools.chain(*mylist))
['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I']

Wrapping the elements in HTML can be done afterwards.

>>> ['<tr>' + x + '</tr>' for x in itertools.chain(*mylist)]
['<tr>A</tr>', '<tr>B</tr>', '<tr>C</tr>', '<tr>D</tr>', '<tr>E</tr>', '<tr>F</tr>',
'<tr>G</tr>', '<tr>H</tr>', '<tr>I</tr>']

Note that if you are trying to generate valid HTML you may also need to HTML escape some of the content in your strings.

score 2 · Accepted Answer · answered Oct 25 '11 at 20:29

2

To concatenate the lists, you can use sum

values = sum([['A', 'B', 'C'], ['D', 'E', 'F'], ['G', 'H', 'I']], [])

To add the HTML tags, you can use a list comprehension.

html_values = ['<tr>' + i + '</tr>' for i in values]

answered Oct 25 '11 at 20:29

Jack Edmonds

31,931
18
65
77

Merging a list of lists

4 Answers4

Linked