39

There's that (relatively) well known Perl axiom, "Only perl can parse Perl." I'm wondering, will that remain true for Perl 6?

Expanding the discussion... I thought of this question given the recent update of PyPy. Does Perl's unique parsability preclude it from similar efforts? Is there much value in a restricted, static view of Perl code (PPI?)? Can Perl 6 have a JIT compiler?*

* I'm not sure if these concepts are related. Are they?

jjmerelo
  • 22,578
  • 8
  • 40
  • 86
Mark Canlas
  • 9,385
  • 5
  • 41
  • 63
  • 6
    Voting to re-open this one. Perl6's parseability and its specification(s) are valid questions. – friedo May 06 '11 at 19:59
  • 7
    The axiom is "Only perl can parse Perl." Capital-P Perl is the language. Lowercase-p perl is the interpreter. – Chris Lutz May 06 '11 at 20:11
  • Apologies for the capitalization oversight. =) – Mark Canlas May 06 '11 at 20:26
  • MoarVM, the main VM targeted by Rakudo (the most used implementation of Perl 6) does JIT compilation as part of it's runtime optimization. https://www.moarvm.org/features.html – daotoad Aug 06 '18 at 23:12
  • To be a little pedantic the "axiom" isn't entirely true. As nicely put [here:](https://en.wikipedia.org/wiki/Perl#Implementation) *Because the Perl interpreter can simulate a Turing machine during its compile phase, it would need to decide the halting problem in order to complete parsing in every case.* So not even perl can always parse Perl. – jjj Jun 16 '19 at 11:49

5 Answers5

37

There is no perl6, and there are many Perl 6 compilers. Perl 6 has a grammar, although it's written in Perl 6, so as long as you can understand that, it tells you everything you need to know.

I just asked Larry this question, since I'm sitting across from him at lunch at the São Paulo Perl Workshop. He says it's now "Only Perl 6 can parse Perl 6", with capital letters on both Perls, which means something different than the original statement.

You don't need a specific program to do that though since Perl 6's goal is one standard and many implementations. There is no "perl6", although Larry had that aliased to his Rakudo for today's talk, even though he used a few different implementations for the examples.

brian d foy
  • 129,424
  • 31
  • 207
  • 592
10

Last I looked, Perl 6 inherits Perl 5's / character, which can mean "beginning of regex" when a term is expected, or "divide" when an operator is expected. Given that, and prototypes, and Perl 6 is at least in the same camp as Perl 5 for static tokenization. The only way to tokenize a Perl 6 program is to have a running Perl 6 interpreter in the Perl 6 compiler. Turtles all the way down, once again.

Randal Schwartz
  • 39,428
  • 4
  • 43
  • 70
  • But, doesn't JavaScript also have `/` character used as comment, division or RegExp? The difference being that JavaScript grammar isn't ambiguous. – Konrad Borowski Aug 04 '12 at 07:39
  • JavaScript doesn't have anything like Perl's prototypes, so every token has a well known affect on parser state and thus lexer state. Perl has to *run* the code of a BEGIN/use to properly load up the possible prototypes of things defined within. – Randal Schwartz Aug 21 '12 at 21:23
  • 5
    (late to the party, but whatever) this is not a problem in perl6, because / is an infix operator and you can never start a regex where a n infix operator is expected. However, if you define both prefix:> and postfix:>, you can have what used to be a regex /1+1/ parse as the sum of 1 and 1, for instance. However, the parser would know about this and correct itself accordingly. No code needs to be run at compile-time to make sure of this. – timotimo May 24 '13 at 14:55
9

The axiom "Only perl can parse Perl" where "perl" is the interpreter binary and "Perl" is the language largely stems from the fact that parsing rules can change while the file is being parsed. In Perl 5, this comes from prototyped subroutine declarations, from various pragmas, and from source filters.

In my opinion, this is only going to become more of a problem in Perl 6. In Perl 5, the number of places where parser rules could change are limited, whereas in Perl 6, they are wide and varied. In addition to everything Perl 5 has, Perl 6 allows you to define your own operators, and since this definition is done in Perl code, a Perl interpreter is needed to make sense of it.

As far as I know, no implementation supports it yet, but the Perl 6 spec also includes real language level macros, which can restructure Perl 6 code either textually or by manipulating the AST. Both of these require the existence of a Perl interpreter to perform their magic.

So in conclusion, I have a feeling that Perl 6 will make the axiom stronger than it is for Perl 5. (And will be even more of a nightmare for the authors of syntax highlighters :) ) Of course this is all to increase the expressive power of the language, so I am ok with the concession.

A corollary to the above is that unlike Perl 5, Perl 6 has a formal spec, so the axiom might have to change to "Only an interpreter implementing the Perl 6 spec can parse Perl 6", but that's being a bit pedantic.

Per the update:

I don't think that the above precludes the idea of a JIT compiler for Perl 6, since by definition, such a compiler would also have to contain a Perl 6 interpreter. As far as static compilation goes, it MIGHT be possible, but it would severely restrict the language's runtime power, since any construct involving eval would not work.

PPI is useful in Perl 5 land because the perl interpreter does not provide many rich and easy to use interfaces to its AST. In Perl 6, the level of introspection is FAR greater, so the interpreter itself may provide all of the necessary tools.

Eric Strom
  • 39,821
  • 2
  • 80
  • 152
  • 1
    The axiom "Only perl can parse Perl" refers to the inability to reliably parse Perl5 without being able to run Perl5 code. I don't see anything in your post about whether parsing Perl6 requires the ability to run Perl6 or not, so I don't see how this answers the question. – ikegami May 06 '11 at 23:32
  • @ikegami => ummm, take a look at the last sentence of the 2nd and 3rd paragraphs? Do you want me to prove that is so? I figured it was obvious. – Eric Strom May 06 '11 at 23:36
  • 4
    @Eric Strom, Yes, they talk of "the number of places where parser rules could change", the ability to define operators, macros, etc. None of those are relevant. For example, they all apply to C++, yet a C++ parser does not need to run arbitrary C++ code to compile a C++ program. – ikegami May 06 '11 at 23:40
  • 1
    @Eric Strom, Those things do show that a Perl6 parser is going to be *complex*, but that's not the question. – ikegami May 06 '11 at 23:42
  • 1
    @Eric Strom, In fact, I was specifically told (by moritz?) that the ability to define new operators does NOT require running any Perl code. – ikegami May 06 '11 at 23:44
  • The existence of operator overloading and macros in C++ are fully visible after a pass of a preprocessor (I assume you don't think that counts as requiring the interpreter???), whereas in Perl those decisions can be delayed until runtime: `if runtime_condition {define a macro} eval "code to which the macro could apply"` – Eric Strom May 06 '11 at 23:47
  • 4
    Here's an example of perl5 needed to parse Perl5: `BEGIN { if (rand()<.5) { *f = sub(){ 3 }; } else { *f = sub { 3 }; } } print f + 1`. Running arbitrary Perl code is required to determine if the product is equivalent to `print(f()+1)` or `print(f(+1))` – ikegami May 06 '11 at 23:55
  • @Eric Strom, "whereas in Perl those decisions can be delayed until runtime:". Can they in Perl6? That's the question, and it's not answered. Like I already said, they can't when it comes to defining new operators, but I don't know about other areas. – ikegami May 07 '11 at 00:04
  • 11
    @Eric Storm You make a strong argument, but I don't entirely buy it. The problem of perl5 only parsing Perl 5 is well known to the Perl 6 folks and was one of the design goals. Unlike Perl 5, Perl 6 language extensions are defined in terms of Perl 6 grammars which is a separate, smaller specification. As to the problem of operators, you can define new ops but not new *types* of ops. That is, the stock grammar can see that it has an infix operator without needing to know if it exists. Finally, this all goes away if ops can only be defined at compile time, something I'm checking. – Schwern May 07 '11 at 05:04
  • 5
    Yeah, you can't define a new operator at runtime that applies to already compiled code. So `if condition { eval "...define a new op or macro..." }` has no effect on parsing. In addition, you can't usefully define a new operator in a condition. `if condition { multi sub postfix:... }` doesn't have the desired effect. – Schwern May 07 '11 at 05:48
  • 10
    This is an incredibly ignorant answer upvoted by people who no idea what's going on. – brian d foy May 07 '11 at 09:34
6

Perl 6 is a specification and any program that follows that spec is Perl 6, just as is true for most other languages. There are a number of Perl 6 implementations in the works.

Bill Ruppert
  • 8,956
  • 7
  • 27
  • 44
4
Can Perl6 have a JIT compiler?

You imply that Perl5 can't JITted based on "only perl can parse Perl5", but that's not true. Even though a Perl5 program can compile differently each time it's run, that doesn't prevent it from JITted.

So, if Perl5 —one of the most loosely defined languages— can be JITted, why Perl6 couldn't be too.

ikegami
  • 367,544
  • 15
  • 269
  • 518