It’s been a while since I’ve posted anything. I’ve got quite a few unfinished items like this. I detect that whole perfectionist/procrastination thing happening. To break out of it, I’m just publishing this in all its shittiness. That’ll show me!

What the hell is a list comprehension?

If you want to understand what that is, you will probably start at the docs.

https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions

The first example gives you

squares = list(map(lambda x: x**2, range(10)))

OK, I know what list is. We’re casting map(...) to a list.

What is map?

It’s a built in function. https://docs.python.org/3/library/functions.html#map

Return an iterator that applies function to every item of iterable, yielding the results.

Wait. It returns an iterator, not the actual result?

Let’s try it out. How do we write a function? https://docs.python.org/3/tutorial/controlflow.html#defining-functions

def is_odd(num):
    return num % 2 == 1

print(is_odd(1))
print(is_odd(2))

OK, we’ve got function now. Let’s plug it into our map and see what the result type is.

def is_odd(num):
    return num % 2 == 1

input_list = [0, 1, 2]
map_result = map(is_odd, input_list)
print(type(map_result))

# <class 'map'>

Sure enough, we got a map back, not the list.

So this is an iterator. What is that in Python terms? https://docs.python.org/3/tutorial/classes.html#iterators

The use of iterators pervades and unifies Python.

We should probably know these.

As expected, iterators allow you to step through a collection (or any iterable).

iterable = iter([0, 1, 2])
print(type(iterable))
# <class 'list_iterator'>

iterable = iter('Pendulum')
print(type(iterable))
# <class 'str_iterator'>

iterable = iter(range(10))
print(type(iterable))
# <class 'range_iterator'>

So those are a few iterables. Our map result, though, is a map.

Python uses duck typing, so maybe we can just look at our map instance and see if it has the iterable methods like __next__.

https://stackoverflow.com/questions/1952464/in-python-how-do-i-determine-if-an-object-is-iterable https://docs.python.org/3/library/stdtypes.html#iterator.next

We can call next, which calls the iterator’s __next__. https://docs.python.org/3/library/functions.html#next

Python defines several iterator objects to support iteration over general and specific sequence types, dictionaries, and other more specialized forms. The specific types are not important beyond their implementation of the iterator protocol.

They’re confirming what we thought. The type doesn’t matter. Just see if we can iterate through.

iterable = iter([0, 1, 2])
print(type(iterable))
print(next(iterable))
print(next(iterable))
print(next(iterable))

iterable = iter('Pendulum')
print(type(iterable))
print(next(iterable))
print(next(iterable))
print(next(iterable))

iterable = iter(range(10))
print(type(iterable))
print(next(iterable))
print(next(iterable))
print(next(iterable))


def is_odd(num):
    return num % 2 == 1

input_list = [0, 1, 2]
map_result = map(is_odd, input_list)
print(type(map_result))
print(next(map_result))
print(next(map_result))
print(next(map_result))

Hot damn, it is an iterable!

<class 'list_iterator'>
0
1
2
<class 'str_iterator'>
P
e
n
<class 'range_iterator'>
0
1
2
<class 'map'>
False
True
False

So let’s take this back to our original example.

squares = list(map(lambda x: x**2, range(10)))

list(...) is converting the iterable returned from map into a proper list.

In the example above, they’re using a lambda. What is that? https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions

Small anonymous functions can be created with the lambda keyword.

So our is_odd function could be done inline like this.

map_result = map(lambda x: x % 2 == 1, input_list)

It’s pretty silly, but can we assign a lambda to a variable?

is_odd = lambda x: x % 2 == 1

Yeah, we can. It defeats the purpose a bit, though.

We have enough to understand this now.

squares = list(map(lambda x: x**2, range(10)))

range(10)

Whoa. range is not a function, it’s a class.

https://docs.python.org/3/library/functions.html#func-range

Rather than being a function, range is actually an immutable sequence type

only stores the start, stop and step values, calculating individual items and subranges as needed).

https://www.pythoncentral.io/pythons-range-function-explained/

ranges support all the common sequence operations. https://docs.python.org/3/library/stdtypes.html#common-sequence-operations

This is why for x in some_range works.

OK, let’s explain it all.

squares = list(
        map(
            lambda x: x**2,
            range(10)
            )
        )

lambda x: x**2

  • an anonymous function
  • given x, returns x**2

range(10)

  • creates an instance of range
  • range is lightweight (only stores start, stop, and step)
  • This range will stop at 10.
  • ranges can be used with the common sequence operations for iteration

map(lambda, range)

  • creates a map, an iterable object with lambda applied to all items in range
  • map is not a function, but a class

list(map)

  • converts the map instance into a list instance

So now we know what this snippet does, we can understand the example. A list comprehension is a shorter way to write that same snippet.

# long way
squares = list(map(lambda x: x**2, range(10)))

# list comprehension way
squares = [x**2 for x in range(10)]

A list comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses.

brackets

[...]

expression

x**2

for clause

for x in range(10)

all the dirty details of expressions

https://docs.python.org/3/reference/expressions.html#expressions https://en.wikipedia.org/wiki/Expression_(computer_science)

Here’s another example.

[(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]

Don’t forget. You can read about this stuff in the REPL too.

help('map')

also useful

http://www.pythonforbeginners.com/basics/list-comprehensions-in-python