9

Lets say I have a list like this:

list_of_lists = [['how to apply'],['a function'],['to each list?']]

And I have a function let's say I want to apply the F function to each sublist of the F function can compute some score about two lists. How can apply this F function to each list of list_of_lists and return each score in a new list like this:

new_list = [score_1, score_2, score_3]

I tried with the map function the following:

map(F, list_of_lists).append(new_list)
thefourtheye
  • 233,700
  • 52
  • 457
  • 497
skwoi
  • 899
  • 3
  • 10
  • 14

5 Answers5

10

You can use the builtin map to do this.

So if the function you want to apply is len, you would do:

>>> list_of_lists = [['how to apply'],['a function'],['to each list?']]
>>> map(len, list_of_lists)
[1, 1, 1]

In Python3, the above returns a map iterator, so you will need an explicit list call:

>>> map(len, list_of_lists)
<map object at 0x7f1faf5da208>
>>> list(map(len, list_of_lists))
[1, 1, 1]

If you are looking to write some code for this which has to be compatible in both Python2 and Python3, list comprehensions are the way to go. Something like:

[apply_function(item) for item in list_of_lists]

will work in both Python 2 and 3 without any changes.

However, if your input list_of_lists is huge, using map in Python3 would make more sense because the iterator will be much faster.

Anshul Goyal
  • 73,278
  • 37
  • 149
  • 186
  • The question is for list of lists, for which the solution works. In case you are looking for a generic solution, ask a new question :) – Anshul Goyal Apr 12 '17 at 00:46
5

You can use a list comprehension, like this

[function_to_be_done(item) for item in list_of_lists]

For example,

>>> list_of_lists = [['how to apply'],['a function'],['to each list?']]
>>> [len(item) for item in list_of_lists]
[1, 1, 1]

Note: Though list comprehensions look like a way to apply a function to all the elements, its main purpose is to construct a new list. So, if you don't want to construct a new list, then just iterate with for loop and call the function.


Apart from that, you can use the map function in Python 2.7, to apply a function to all the elements and construct a list. For example,

>>> list_of_lists = [['how to apply'],['a function'],['to each list?']]
>>> map(len, list_of_lists)
[1, 1, 1]

But, map returns a map iterator object in Python 3.x. So, you need to explicitly convert that to a list, like this

>>> list_of_lists = [['how to apply'],['a function'],['to each list?']]
>>> map(len, list_of_lists)
<map object at 0x7f94026afd30>
>>> list(map(len, list_of_lists))
[1, 1, 1]

You might want to read about, what Guido thinks about map in this post.

Basically, map would more often demand you to create a new function (mostly people create a lambda function). But in many cases, list comprehension avoids that.

thefourtheye
  • 233,700
  • 52
  • 457
  • 497
5

Map is your friend! map takes a function and an iterable (list, for example) and applies the function on each element of the list.

map(len, [['how to apply'],['a function'],['to each list?']]) 

Output

[1, 1, 1]

If you wanted to do more granular calculation on elements of the sublist, you can nest the map:

map(lambda x: map(lambda y: y + 1, x), [[1], [1, 2], [1, 2, 3]])

Output

[[2], [2, 3], [2, 3, 4]]

Another possible approach (also from functional programming) are list comprehensions. List comprehension is a way of constructing a list from iterable in Python. The syntax is [element for element in iterable]. Any computation can be done on the element, so

[f(element) for element in iterable]

means that the resulting list will be a list of elements, where each element is the result of function f. Like map, list comprehension can be further nested, resulting in a nested element function application.

[element + 1 for element in el] for el in [[1], [1, 2], [1, 2, 3]]]

Output

[[2], [2, 3], [2, 3, 4]]
mpolednik
  • 1,013
  • 7
  • 15
4

How about

[ F(x) for x in list_of_lists ]

which will iterate over list_of_lists, call F with each sublist as an argument, then generate a list of the results.

If you want to use the sublists as all the arguments to F you could do it slightly differently as

[ F(*x) for x in list_of_lists ]
Eric Renouf
  • 13,950
  • 3
  • 45
  • 67
3

Something that works on arbitrarily nested lists, e.g. [[1,2],[[5]],[7,[8,[9,11]]]] :

def apply_f(a,f):
 if isinstance(a,list):
     return map(lambda t:apply_f(t,f), a)
 else:
     return f(a)

here is an example of running this:

>>> ll=[[1,2],[[5]],[7,[8,[9,11]]]]
>>> apply_f(ll,lambda t:t**2)
[[1, 4], [[25]], [49, [64, [81, 121]]]]

and here is how to do the same only on selected levels:

def apply_f(a,f,depth,levels):
    if isinstance(a,list):
        return map(lambda t:apply_f(t,f,depth+1,levels), a)
    else:
        if depth in levels:
            return f(a)
        else:
            return a

getting e.g.

>>> apply_f(ll,lambda t:t**2, 0, [2,4])
[[1, 4], [[5]], [49, [8, [81, 121]]]]

there are some optimisations to do here, by avoiding passing f and levels around (making the recursive function an inner function in a wrapper, so that it can use f and levels from the outer scope), but that's minor. (Note that this is Python 2, for Python 3 you need to replace map with something else).


For a more generic input, the following would do the trick:

def apply_f(a,f):
    try: 
       return(f(a))
    except:
       return map(lambda t:apply_f(t,f), a)

So now also the following sort of works:

>> apply_f([(1,2),[[5]],[7,(8,[9,11])]],lambda t:t**2)
[[1, 4], [[25]], [49, [64, [81, 121]]]]

(things get slightly rewritten, as map() always produces lists...)

Dima Pasechnik
  • 336
  • 4
  • 16
  • Can you generalize to map at a user-specified level or list of levels? – alancalvitti Jan 31 '19 at 15:35
  • yes, this is easy: you need to store the nestedness level of apply_f in its parameter, and increase it in recursive calls; and return f(a) or a itself, conditioned on the nestedness level. – Dima Pasechnik Feb 01 '19 at 09:45
  • here you are. In case, I am available for consulting work, and actually passed Google Foobar for the lols a couple of years ago :-) – Dima Pasechnik Feb 02 '19 at 01:12
  • Dima, thanks, will test. Im new to "pythonic" but can't wait to get back to a high level language vs this assembly-level sh*. What you just implemented is built in the system I like to use and it works generically for all expressions not just lists. van Rossum is a clown. – alancalvitti Feb 03 '19 at 03:29
  • You can make it more general by testing for Iterable instead In [5]: from collections import Iterable In [6]: isinstance([1,2],Iterable) Out[6]: True In [7]: isinstance((1,2),Iterable) Out[7]: True – Dima Pasechnik Feb 03 '19 at 09:40
  • WIth all respect, if you programmed in assembly you won't ever call Python an assembly. Try coding this in C (a glorified assembly) :-). I won't be surprised if the function you want is available in Common Lisp... – Dima Pasechnik Feb 03 '19 at 09:45
  • I added one more example, that lets one apply functions onto nested iterables. – Dima Pasechnik Feb 03 '19 at 10:05
  • (1) How do you extract raw lists in your output? I get nested `map` objects. (2) Can you modify to preserve the container type - in your example return `(1,4)` as `tuple` not `list`. (3) does it generalize to `dicts` as well? eg `apply_f([{'a':1,'b':2},[[5]],...}` – alancalvitti Feb 04 '19 at 16:43
  • In my forthcoming book on data I extend a chart published by Alan Kay (Scientific American, 1984) that compares the level of abstraction of various languages since the 1950s. Python is at a higher level than C, as C is higher than assembly and the latter is higher than machine code. But the language I've been using for data, Wolfram Language, makes working in Py feel like assembly, relatively speaking. – alancalvitti Feb 04 '19 at 16:46
  • yes, I used a bit of Mathematica in the past, but I've sworn off using it, along with Matlab, Maple, and Magma, since landing at some point in a department without a license for Matlab (something of a staple in my then research community), or to any of the other M's, in fact. Yes, I could have ordered a CD with Mathematica from Moscow, but I chose not to. I'd love to work in Haskell or something similar, just don't have an opportunity.... – Dima Pasechnik Feb 04 '19 at 19:23
  • You get nested map objects in Python 3. Run Python 2, you'll get what shown. Or wrap `map()` into `list()`, i.e. replace each `map()` with `list(map())`. – Dima Pasechnik Feb 04 '19 at 19:40
  • Well, I'm using Python 3 and cannot roll back to 2 at work. The problem is seems programmer would have to remember and match input types (`list`, `tuple` etc) to a nested combination of these to convert the nested maps. Also can you handle `dict`? And can this more generic type version be combined with user-specified levels (including a ":" spec to apply a function at all levels)? - All these features are built into WL. – alancalvitti Feb 04 '19 at 20:19
  • Haskell, while functional, doesn't have the large scale symbolic processing and pattern matching features of WL, which are very handy for data work. – alancalvitti Feb 04 '19 at 20:20
  • I merely explained how to solve a particular problem in Python. If you have a rather different problem, it's probably better to post another question. We merely scratched the surface of what Python can do for you here. (Needless to say, if you have to do pattern-matching, both Python and Haskell are more than adequate tools, and Haskell offers a lot of extra in form of type checking etc.) – Dima Pasechnik Feb 05 '19 at 10:36
  • Re `pattern-matching, both Python and Haskell are more than adequate tools`, I meant symbolic pattern matching of arbitrary code expressions. There's no such functionality in Python or Haskell. In WL, the code itself can be treated as input data to be transformed (metaprogramming). – alancalvitti Feb 05 '19 at 13:09
  • I think meta-programming is reserved for transforming code, rather than data, and in this both Python (decorators) and Haskell (just by design) are just fine. – Dima Pasechnik Feb 05 '19 at 14:41
  • as to mathematical expressions, check out e.g. http://doc.sagemath.org/html/en/prep/Calculus.html (sagemath is a Python library) – Dima Pasechnik Feb 05 '19 at 14:47
  • Yes I'm aware of Sage. But it's limited to Calculus. In WL you can symbolically pattern match / replace any code not just math. You can't do this in Py in general. Re meta-programming, the code represents data containers as well as functional operators acting on those, which is routine in WL, not limited to math or data. So in WL there is no real difference b/w code and data. Whereas in Py there is. – alancalvitti Feb 05 '19 at 15:08
  • I used to be somewhat familiar with Lisp macros, and before that had a student job involving a lot of code generation in MUMPS, and found them a bit too cryptic. Nowadays I think a degree of separation between code and data helps to maintain sanity :-) – Dima Pasechnik Feb 05 '19 at 15:31
  • Notable computer scientist Don Knuth still prefers assembly. A basic task like [dict recursive merge](https://stackoverflow.com/questions/7204805/dictionaries-of-dictionaries-merge/24088493#24088493), which clearly is painful in Py, is 1-line of code in WL. – alancalvitti Feb 05 '19 at 15:56
  • with https://pypi.org/project/deepmerge/ it should be 1 line in Python too, I guess. – Dima Pasechnik Feb 05 '19 at 19:05
  • I mean the function *definition* to perform the merge is 1-line in WL. – alancalvitti Feb 05 '19 at 19:34
  • by the way, Knuth prefers assembly for analysis of algorithms, rather than actual coding. E.g. he wrote TeX and other related utilities in WEB: https://en.wikipedia.org/wiki/WEB – Dima Pasechnik Feb 06 '19 at 09:24
  • Great, have fun in assembly-world. but If you are interested, I'll be doing live coding tutorials on functional data flow through WRI based on my book. – alancalvitti Feb 06 '19 at 14:18
  • I am sure if you post a challenge on 1-line implementation of your recursive merge in a proper PL forum, you'd be surprised to find that WL is not the only game in town, that it can be done without having to pay Wolfram tax (perhaps even in Haskell :-)) – Dima Pasechnik Feb 06 '19 at 15:09
  • well, that's what 20+ years of heavy Mathematica abuse can do to an individual. :-) – Dima Pasechnik Feb 06 '19 at 15:17
  • Quite the opposite: just about everything I looked up to see if there's an easy way, Stackoverflow says not doable. There's at least a 5:1 Py/WL LOC ratio for data. So even though it's free, you pay much more in dev time. – alancalvitti Feb 06 '19 at 15:40