Need Help Understanding a Crude Benchmark: Regular PHP vs HHVM vs Zephir

Question

I performed this test using a simple factorial function (borrowed the logic from http://avelino.xxx/2014/03/golang-c-and-python-the-benchmark-time)

Regular PHP Code

function fact($n){ 
    if($n===0)return 1;
    return $n*fact($n-1);
}

function calc(){
    $t = 0;
    for($i=0; $i<100000; $i++){
        for($j=0; $j<8; $j++){
            $t += fact($j);
        }
    }
    return $t; 
}

$result = calc();
echo $result."\n";

PHP Using Zephir

$fact = new Utils\Fact();
$result = $fact->calc();
echo $result."\n";

Zephir Code

namespace Utils;

class Fact{
    public function fact(int n) -> int{
        if(n==0){
            return 1;
        }

        return n*this->fact(n - 1);
    }

    public function calc() -> int{
        int i,j,total;
        let total = 0;
        for i in range(0,99999){
            for j in range (0,7){
                let total = total + this->fact(j);
            }
        }
        return total;
    }
}

I executed these snippets using the time command in the following manner:

Regular PHP

time php -c /etc/php5/apache2/php.ini regular.php

Result

591400000
real 0m0.788s
user 0m0.736s
sys 0m0.026s

PHP Using Zephir Class

time php -c /etc/php5/apache2/php.ini zephyr.php

Result

591400000
real 0m1.529s
user 0m1.494s
sys 0m0.024s

HHVM

time hhvm regular.php

Result

591400000
real 0m0.883s
user 0m0.814s
sys 0m0.045s

Question:

As you can see from the results above, regular PHP code seems to have performed better than the one that uses a compiled Zephyr class as a PHP extension. This is what has me confused.

How can the scripted code end up being faster than the compiled one, especially when both employ the same logic? I think I'm missing something here and would be grateful if someone could help me understand this.

EDIT: Looks like others are facing a similar problem with Zephir: Zephir 2x slower

score 5 · Accepted Answer · answered Mar 07 '14 at 23:08

5

I don't know about zephyr but HHVM is a JIT so you pay extra startup cost so that future runs are faster. Try increasing your benchmark so it takes 1 minute or 10 minutes if you want to really see HHVM perform.

Also, in some situations we decide to not turn on the JIT in command line mode so force it on with -vEval.Jit=true

answered Mar 07 '14 at 23:08

Paul Tarjan

48,968
59
172
213

I just tested this out and you're right. When I increased the run time , HHVM did perform better (4 sec faster than regular PHP over a 1 minute 30 sec run). However, a similar test for the Zephir code came out almost 2 minutes slower! – Kul Mar 08 '14 at 05:34
2

HHVM should be 10x faster than zend. What does your config look like? Try running with `hhvm -vEval.Jit=1 foo.php` – Paul Tarjan Mar 09 '14 at 07:40
5

That did the trick. Zend took **73.3 seconds** while HHVM took **3.1 seconds**. (I put the call to the `calc` function in a loop for 100 times) – Kul Mar 09 '14 at 09:43
1

You are the inspiriation for https://github.com/facebook/hhvm/commit/190a12a7a5c8b79a16182107afad5ecc7971b303 – Paul Tarjan Mar 10 '14 at 18:19
1

Wow! This would certainly help first timers see the benefits of HHVM right away without any additional config :) – Kul Mar 11 '14 at 04:35
@Jay - so the benefit of HHVM is to gloat how fast it computes factorials compared to PHP or am I missing something? – N.B. Mar 12 '14 at 14:49
@N.B. the benefit for us is to run facebook.com 10x faster than php5 saving us a bajillion dollars per year. In open-sourcing it we hope it does the same for everyone else. – Paul Tarjan Mar 12 '14 at 17:59
1

@N.B. - I posted this in the hope that someone could help me understand things better and Paul helped me understand part of it. How is any of this the equivalent of gloating? – Kul Mar 12 '14 at 21:27

Need Help Understanding a Crude Benchmark: Regular PHP vs HHVM vs Zephir

Question:

1 Answers1