Symfony benchmark using HHVM

Yesterday Asm89 blogged about a custom version of symfony edited to run on Facebook’s HHVM. I’ve followed his great blog post (you can read it here) in how to setup HHVM to run the Symfony standard web application.

After that I’ve tested the performances compared to a nginx + php-fpm configuration and I wanted to share the results here.

N.B. This is a benchmark of the symfony standard application just to have a starting point on how much the difference can be YMMV

Test setup

I’ve run the tests on a vagrant VM on a i5 3750k with assigned 4 cores. The box was running Ubuntu 12.04 64-bit. I’ve installed both HHVM and nginx + php-fpm on the same box running on different ports. Nginx and php-fpm was tuned following this article and php-fpm with max_children = 10.

Tests

I’ve used apache bench to test the default symfony webapp demo page using the production environment, all tests used 2000 requests with 1, 10, 50, 100 concurrent clients.

This is what I’ve got, enjoy.

Server - concurrency	Requests per second	Time per request (ms)	50% req served within (ms)	90% req served within (ms)	99% req served within (ms)	100% req served within (ms)
Nginx - 1 user	25.48	39.245	39	40	41	52
Nginx (APC) - 1 user	93.74	10.668	10	11	14	17
Nginx 5.5 (OPcache) - 1 user	135.01	7.407	7	8	10	21
HHVM - 1 user	105.79	9.453	9	10	13	16
Nginx - 10 user	78.90	126.736	125	177	220	334
Nginx (APC) - 10 user	259.74	38.492	37	56	79	106
Nginx 5.5 (OPcache) - 10 user	399.23	25.048	24	32	42	66
HHVM - 10 user	362.79	27.564	24	47	81	144
Nginx - 50 user	73.74	678.035	665	869	1079	1249
Nginx (APC) - 50 user	239.62	208.661	190	275	371	586
Nginx 5.5 (OPcache) - 50 user	378.44	132.120	131	146	163	182
HHVM - 50 user	369.74	135.231	126	203	319	898
Nginx - 100 user	76.66	1304.518	1292	1481	1714	1980
Nginx (APC) - 50 user	278.30	359.326	357	384	408	455
Nginx 5.5 (OPcache) - 100 user	385.82	259.185	254	292	316	336
HHVM - 100 user	364.71	274.191	269	332	389	494

As you can see HHVM is about 4 times faster then php so as soon it gets stable and more usable you should really consider looking into it.

As I’ve already said, this is just Symfony standard application, you can get worse or better results using HHVM on our own project, but for such a big performance improvement, it should be worth the work :)

Update: I’ve added the tests with APC enabled, it increased a lot compared to the plain PHP version, but still HHVM is faster.

Update 2: I’ve added also tests with PHP 5.5 using the Zend OPcache enabled, it seems they perform even better than HHVM! I’ve used both PHP and HHVM out of the box, without any config changes, if you think there is another test that I haven’t done drop a comment and I’ll try to run it!

Complexity heads up!

This is just an hello world example, as pointed out on Twitter by @chregu when more work has to be done by PHP, HHVM performs better than PHP 5.5 + OPcache.

As a simple test I’ve tried to run this function:

<?php

function fib($n)
{
    if ($n <= 2)
        return 1;
    else
        return fib($n-1) + fib($n-2);
}

$n = 32;
printf("fib(%d) = %d\n", $n, fib($n, 2));

on both HHVM and PHP 5.5 + OPcache (using only 500 requests, too much to wait with just PHP), these are the results:

Server - concurrency	Requests per second	Time per request (ms)	50% req served within (ms)	90% req served within (ms)	99% req served within (ms)	100% req served within (ms)
Nginx 5.5 (OPcache) - 1 user	2.12	472.811	472	477	491	493
HHVM - 1 user	47.30	21.140	20	21	41	66
Nginx 5.5 (OPcache) - 10 user	7.39	1352.937	1277	1598	1755	1755
HHVM - 10 user	182.67	54.743	53	81	109	134

So HHVM performs way better even compared to PHP 5.5 + OPcache when you’ve to do some hard work (who said Doctrine hydration?). So basically as I said before, you should test your application with HHVM (if it’s compatbile) to know if it’s worth the switch.

I have to remind you that these are just simple, stupid, tests, I’ll try to keep the post updated when real-life examples are done.