How Lumen Is Benchmarked

I’ve received a few tweets asking me to demonstrate how Lumen is benchmarked against Silex and Slim.

First, I should state that all of these frameworks are faster than you will ever need out of the box. Keep in mind that even full-stack Laravel 5.0 powers sites that receive 15,000,000 unique visits per day.

The benchmarks themselves are actually really simple. I use a Homestead VM since that is something anyone can recreate. In my tests, I allocated 2GB of RAM to the machine. Of course, benchmark numbers are going to vary depending on the hardware of your host machine. I personally am running benchmarks on a 2012 Macbook Pro Retina 15″ with 16GB of RAM. So, don’t be surprised if your numbers are different than my numbers.

So, to get started, clone down a copy of each project. I used Slim 3, which is the develop branch of the Slim repository since it is close to release and utilizes the new PSR-7 standard for HTTP. It should be noted that Slim 3 benchmarks a bit slower than Slim 2. In my tests, Slim 2 and Lumen are fairly close, with Slim 2 averaging around 1700-1800 requests per second and Lumen averaging around 1800-1900 requests per second. Slim 3 is a little slower at around 1200-1300 requests per second.

So, once you have the projects cloned down and each of them simply returning “Hello World”, you should run “composer dump-autoload –optimize”. This will make a significant improvement in the performance of these frameworks during the test.

I used the simple “Apache Benchmark” program on Homestead which can be installed via “apt-get install apache2-utils”. Once you have installed Apache Benchmark, you can run the test like so:

ab -t 10 -c 10

This will instruct Apache Benchmark to run for 10 seconds with 10 concurrent requests happening.

You should also consider that benchmark numbers can vary depending on the software running on your computer. All three benchmarks in my test video run slower than advertised on the Lumen homepage probably because I am recording video, sound, etc. You will see some improvement in numbers if all other software is closed and your host machine is basically idle.

For the tests in my video the results were

Silex: 950 requests per second

Slim: 1250 requests per second

Lumen: 1700 requests per second

All of these frameworks are ridiculously fast.