Web Server Test Benchmark
This test benchmarked two identical Magento stores against one another to gauge performance and responsiveness on the FreeBSD operating system. Modules for each system were as follows:
- PHP 5.3, MySQL, Apache, Memcache
- PHP-FPM, Percona MySQL, Nginx, Memcache
Modules, Explained
PHP is an open-source, server-side scripting language designed for the development of dynamic webpages. It was one of the first such languages to be embedded into an HTML source document rather than called via an external file to process data. PHP code is interpreted by a web server equipped with a PHPO processor module, which generates the resulting webpage. “FPM” stands for FastCGI Process Manager, which constitutes a set of patches for the PHP source that reduce the overhead associated with interfacing the web server and interactive programs, allowing more webpage requests at once.
MySQL is an open-source relational database management system (RDBMS) that runs as a server, providing multi-user access to databases. Percona MySQL is an enhanced drop-in replacement for MySQL, enabling server queries to run faster and more consistently.
Apache is a web server software program providing a variety of MultiProcessing Modules (MPMs), allowing it to match the demands of a process-based, hybrid (process and thread), or event-hybrid infrastructure.
Nginx is a lightweight, high-performance web server designed to deliver large amounts of static content quickly and efficiently. It uses an asynchronous, event-driven model to provide more predictable performance under load.
Memcache is an in-memory, key-value store for small chunks of arbitrary data (strings or objects) pulled by database calls, API calls, or page rendering.
TESTING METHODOLOGY
Most comparisons were made using ab, Apache’s HTTP server benchmarking utility, with in-depth testing facilitated by a Python utility called Pylot. The test was conducted in a jailed UNIX environment, with two jails each containing a copy of the Magento store.
The following parameters were used:
- Run time: 120 seconds
- Virtual users: 200
- Ramp-up: 20 seconds (added to investigate performance from low to progressively higher loads)
- First request target: main site (homepage)
- Second request: product category page
Results
| Measurements | Apache | Nginx |
|---|---|---|
| Total Requests (response count) | 808 | 15069 |
| Successful Requests | 255 | 581 |
| Unsuccessful Requests (bad response) | 553 | 14488 |
| Transaction Response Time, Minimum | 1.8 sec | 0.007 sec |
| Transaction Response Time, Maximum | 115.118 sec | 62.648 sec |
| Transaction Response Time, Average | 15.87 sec | 1.085 sec |
Nginx:
[CHART]On the first ramp-up, Nginx was able to handle the heavy load, keeping response time for all 15,069 requests to 1.08 seconds.
[Chart]Response time increased steadily until it reached a plateau at 80 elapsed seconds, after which Nginx serviced all requests within 60 seconds.
Apache:
Before and after the first ramp-up, Apache handled the load (809 requests) and kept response time to an average of 15.867 seconds. After 90 elapsed seconds, response time jumped considerably.
[CHART]Response time remained at or around 20 seconds for most of the test, but some responses jumped up to 120 seconds.