Is possible iteration over undefined dereferenced array a bug or a feature?

Question

Following code crashes as expected, because $x does not hold Array reference:

$ perl -E 'use strict; use warnings; my $x; say @{$x}; say "OK";'

Can't use an undefined value as an ARRAY reference at -e line 1.

However this code works without any warning:

perl -E 'use strict; use warnings; my $x; say for @{$x}; say "OK";'

OK

Why? I have not found any documentation of this behavior in perldoc. Looks like for context is implying some DWIM-ness logic here (with dangerous consequences).

score 6 · Accepted Answer · answered Nov 17 '20 at 11:01

6

This is hopefully somewhat enlightening:

$ perl -E 'use strict; use warnings; my $x; say for @{$x}; say $x;'
ARRAY(0x561b92479420)

The $x variable is being autovivified by for.

Compare with:

$ perl -E 'use strict; use warnings; my $x; $x->[0]; say $x;'
Useless use of array element in void context at -e line 1.
ARRAY(0x561b92479420)

Certain reference operations on undefined scalars will cause autovivification to happen. for @{$var} is apparently one of them. There is a autovivication module on CPAN that allows you more control over when autovivification happens.

answered Nov 17 '20 at 11:01

tobyink

13,478
1
23
35

That is tricky. I always thought autovivification happens only during explicit traversal, like `$x{foo}->{bar}->{baz} = 123`. But when one think about it this is indeed consistent, `for` must traverse elems. Thanks for answer. – Pawel Pabian bbkr Nov 17 '20 at 12:58
I suspect `for` causes autovivifcation because it uses *aliasing* to access the elements of the array. `say @{$x}` doesn't need to create any aliases. – tobyink Nov 17 '20 at 17:02
@Pawel Pabian bbkr, Autovivification happens in an lvalue context, and `for` evaluates its list in lvalue context because you can do stuff like `for (@a) { $_ = uc($_); }` – ikegami Nov 17 '20 at 18:52

score 3 · Answer 2 · answered Nov 17 '20 at 11:47

You have this statement:

say for @{$x}

The value of $x is undef, so when you dereference it, Perl notices that you want to use $x as an array ref. It then creates the array for you ("autovivification"). That array is empty, so there's nothing that for can iterate over.

Autovivification is the feature that enables us to do tasks like counting in this loop. When a key for $first doesn't exist, Perl adds that key to the hash. But, there needs to be a second level hash where $second can be a key. Since the value of the $first key is undef, Perl autovivifies that second level hash so you can operate on it:

for ( ... ) {
    # source.example.com dest.example.org 1234
    my( $first, $second, $count ) = split;
    $hash->{$first}{$second} += $count;
    }

Image how hard this task would be if you had to know all the keys, hash sizes, and so on in advance, or had to extend or modify the data structure sizes yourself.

We have an extended example of this in Intermediate Perl, the book in the tutorial series where we talk about references. I also write about it in Understand autovivification.

@tobyink mentions the autovificiation pragma. It recognizes certain constructs and can warn or stop your program:

use v5.10;
use warnings;

no autovivification qw(store);

my $x;
for ( @$x ) {
    say "Got value: $_";
    }

say "End. x = $x";

Now the program stops:

Can't vivify reference at ...

But, all of this makes your program a bit slower as Perl has to do extra work to check all that.

There's also the StackOverflow answer How do I disable autovivification in Perl?, but I think the answers aren't that useful.

You should get used to autovivification and programming in an environment where it exists (along with all the other things that you can accidentally do to create bugs).

Thanks for detailed answer. I'm aware of autovivification (and love it). I was simply not aware that `for` also implies it. I've marked @tobyink response as answer because it pinpoints the issue, however your extended article on autovivification topic is also great. Thanks again. — Pawel Pabian bbkr, Nov 17 '20 at 12:48
@briandfoy, no `for` is being special. Compare the output of `perl -E'my $x; say @$x; say $x'` with `perl -E'my $x; 1 for @$x; say $x'`. They both dereference an undef value, but only the `for` autovivifies it. — tobyink, Nov 17 '20 at 16:57
Try it like this: `perl -E 'use strict; use warnings; my $x; say $#{$x}; say $x; say "OK";'` You have to do something with the undef value to get autovivification, and it looks like `say` (nor `print`) does that. — brian d foy, Nov 17 '20 at 19:26

Is possible iteration over undefined dereferenced array a bug or a feature?

2 Answers2