What is the difference between a generator and an array?

Question

Today the PHP team released the PHP 5.5.0 version, which includes support for generators. Reading the documentation, I noticed that it does exactly what it could do with an array.

PHP team generator example:

// Only PHP 5.5
function gen_one_to_three() {
    for ($i = 1; $i <= 3; $i++) {
        // Note that $i is preserved between yields.
        yield $i;
    }
}

$generator = gen_one_to_three();
foreach ($generator as $value) {
    echo "$value\n";
}

Result:

1
2
3

But I can do the same thing using arrays. And I can still keep compatible with earlier versions of PHP.

Take a look:

// Compatible with 4.4.9!
function gen_one_to_three() {
    $results = array();
    for ($i = 1; $i <= 3; $i++) {
        $results[] = $i;
    }

    return $results;
}

$generator = gen_one_to_three();
foreach ($generator as $value) {
    echo "$value\n";
}

So the question is: what is the purpose of the existence of this new feature? I got to play all examples of documentation without using the new feature, replacing it with array.

Can anyone give a good explanation and perhaps an example that is not necessarily impossible with older versions, but using generators can help in development?

Answer 1

The difference is in terms of efficiency. For example, many languages other than PHP include two range functions, range() and xrange(). This is a really good example of generators and why to use them. Let's build our own:

function range($start, $end) {
    $array = array();
    for ($i = $start; $i <= $end; $i++) {
        $array[] = $i;
    }
    return $array;
}

Now that's really straight forward. However, for large ranges, it takes a HUGE amount of memory. If we tried to run it with $start = 0 and $end = 100000000, we'd likely run out of memory!

But if we used a generator:

function xrange($start, $end) {
    for ($i = $start; $i <= $end; $i++) {
        yield $i;
    }
}

Now we use constant memory, yet still have an "array" (like structure) that we can iterate over (and use with other iterators) in the same space.

It doesn't replace an array, but it does provide an efficient way of avoiding to need the memory...

But it also provides savings in terms of the generation of items. Since each result is generated as-needed, you could delay execution (fetching or computing) each element until you needed it. So for example, if you needed to fetch an item from a database and do some complex processing around each row, you could delay that with a generator until you actually need that row:

function fetchFromDb($result) {
    while ($row = $result->fetchArray()) {
        $record = doSomeComplexProcessing($row);
        yield $record;
    }
}

So if you only needed the first 3 results, you'd only process the first three records.

For more info, I wrote a blog post on this exact subject.

Answer 2

Generators allow for lazy evaluation of complex statements. That way you save memory as you don't have to allocate everything at once.

Besides both being iterable they aren't nearly the same. An array is a data structure, a generator is not.

Answer 3

An array has to contain each value that you're looping over before you start looping; a generator creates each value "on the fly" as it is requested, so a lot less memory;

An array works with the values it contains, and has to be prepopulated with those values; a generator can create values according to special criteria to be used directly... e.g. a fibonnaci sequence, or letters from a non-A-Z alphabet (calculated by the UTF-8 numeric value) effectively allowing alphaRange('א','ת');

EDIT

function fibonacci($count) {
    $prev = 0;
    $current = 1;

    for ($i = 0; $i < $count; ++$i) {
        yield $prev;
        $next = $prev + $current;
        $prev = $current;
        $current = $next;
    }
}

foreach (fibonacci(48) as $i => $value) {
    echo $i , ' -> ' , $value, PHP_EOL;
}

EDIT

Just for fun, here's a generator that will return the Hebrew alphabet as UTF-8 characters

function hebrewAlphabet() {
    $utf8firstCharacter = 1488;
    $utf8lastCharacter = 1514;
    for ($character = $utf8firstCharacter; $character <= $utf8lastCharacter; ++$character) {
        yield html_entity_decode('&#'.$character.';', ENT_NOQUOTES, 'UTF-8');
    };
}

foreach(hebrewAlphabet() as $character) {
    echo $character, ' ';
}

Answer 4

As in Python:

When an iteration over a set of item starts using the for statement, the generator is run. Once the generator's function code reaches a "yield" statement, the generator yields its execution back to the for loop, returning a new value from the set. The generator function can generate as many values (possibly infinite) as it wants, yielding each one in its turn.

...Generators executes the yield statements one at a time, pausing in between to yield execution back to the main for loop.

-learnpython.org