Redis vs APCu 2018

Spread the love
(Last Updated On: June 10, 2018)

Redis VS APCu Object Cache performance test 2018: is it relevant anyway? Redis is fast, but APCu is faster. Unfortunately, APCu is being pushed on the side of the road as an Object cache, and here is why.

There are 3 categories of caching systems on the server side: Memory code caching, Memory object caching, and Disk file caching. Objects can be anything though, including files. By file I mean any file: the generated HTML code that makes a page, a CSS, an image…

They are in the 3rd layer on the map below:

Web Server Cache 9 levels

More details about other Caching systems here.

APCu is an in-memory key-value store for PHP. Keys are of type string and values can be any PHP variables. APCu only supports userland caching of variables. APCu is APC stripped of OpCode caching.

Without even testing, without even starting reading blogs and Stackexchanges about it, I knew APCu would perform better. Indeed, they are both memory caches, but APCu should definitely be faster for two reasons:

  1. APCu shares the same memory segments as PHP, for it’s an extension: apcu.so
  2. Redis has more overhead because it’s external, so you access it either via TCP, http(s) or unix socket.

Either way, Redis cannot be faster than APCu at delivering content because of that.

 

WordPress Plugins Available

Welcome to the grey zone. You cannot test APCu or Redis object cache without a third party plugin, and as of today May 2018, there are only 3 plugins left that implement both APCu and Redis:

powered cache icon 128x128 oq10Powered Cache

w3 total cache icon 128x128 oq10W3 Total Cache

WP LP Cache

not considered because it doesn’t implement Redis

There are more plugins (I counted 4) that offer APCu cache but they are outdated and not maintained for one year or more.

Since their implementation of object caching will vary from developer to developer, what you are really testing is their ability to code properly. And since they are developing for free (W3 has a premium version though), you don’t know how good is their code until you read it. This is why you cannot compare different plugins implementing different cache systems, because the implementation would be different.

Therefore, to test Redis VS APCu, only 2 plugins can be used, for which I must load test 3 times: control (no cache), then Redis , then APCu, with a cache flush before each test. That’s a total of 6 load tests. Is it worth it? I say no, because of the results I got from my tests with Redis Object Cache. APCu may or may not be better than Redis but it’s not relevant, for either one cannot compete against a good old static full page cache.

What we are certain of, is that both these plugins are third party cache implementations. Therefore, they should cache the same number of objects since WordPress core and its plugins haven’t changed between the tests.

Finally, as I said in the conclusions of the Redis Object Cache tests, there is a high CPU overhead induced by this type of caching system (from PHP), which would overcome any benefits. I will test load Redis VS APCu anyway but I know this is useless for a single instance without lots of database access.

 

WordPress Object Cache Load Tests

Server Configuration

  • Nginx, PHP-fpm, OS, APCu version (AWS EC2 t2.micro instance):

redisvsapcu2018

  • OS (AWS EC2 t2.micro instance): Ubuntu 16.04.4 LTS 4.4.0-1050-aws x86_64
  • vCPU: Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz (1 core only)
  • File system: SSD disks (314MB/s read speed for small files)

System monitoring: Netdata 1.9.1, collects every 2 seconds 1,792 metrics.

These tests will be run by loader.io, with 500 clients/mn, for 1mn, in this order:

  1. Cache disabled (control)
  2. Redis Cache enabled
  3. APCu cache enabled

That’s 500/60 = 8.3 new connections per seconds, for 60 seconds. I chose 500 connections/mn because my previous tests crashed with Object Cache at 1000 connections/mn, and I think 250/mn is not enough. Goal: having a server load between 50% and 80%.

For every tests,

  1. fastCGI is disabled (BYPASS)
  2. OPcache is enabled and not purged before test
  3. Redis/APCu is purged before test, and only the Object Cache method is checked
  4. CloudFlare is disabled

 

Powered Cache Load Test

powered cache banner 772x250 oq10

Plugin Configuration

Okay… Powered Cache is not really serious:

  1. First problem: you cannot chose which Redis database to use, it’s setup for #0 by default.
  2. Second, Redis is set up for TCP access. Impossible to configure unix socket or anything else.

redisvsapcu2018 1

 

Redis VS APCu Load Test Results with loader.io

Control Test Results:

redisvsapcu2018 2

Average response time = 640ms, 16 clients/s.

Redis Test Results:

redisvsapcu2018 3

Average response time = 660ms, 16 clients/s. It’s worse than without any cache.

APCu Test Results:

redisvsapcu2018 4

Continuous increase in connections count until it crashed at 95 clients or so. Response time caped at 10s/page. Thus, loader.io decided to interrupt the test. Not good.

 

Redis VS APCu Netdata Analysis

It’s not obvious, but Redis brings a slight CPU overhead, and APCu implementation is completely wrong as the CPU reached 100% usage, interrupting the test with timeouts ratio > 1:

redisvsapcu2018 5

Without cache or With Redis, network traffic is quite the same, to account for PHP-MySQL database communication, then for PHP-Redis communication. Much less network traffic with APCu as expected since it’s integrated to PHP, and also because the test didn’t work well:

redisvsapcu2018 6

IPv4 packets exchanged are higher with Redis than without cache, proof that Redis induces more network operations/s because it caches a lot more things than the database requests needed to serve the pages:

redisvsapcu2018 7

Overall, Redis clearly induces more CPU usage, for itself and because PHP needs to communicate with it:

redisvsapcu2018 8

Redis operations: ~1200/s

redisvsapcu2018 9

Redis bandwidth: ~37MB/s

redisvsapcu2018 10

PHP-fpm pool responded well with and without Redis, having the same number of PHP requests processed (~8.5/s). It completely stopped responding during APCu test:

redisvsapcu2018 11

Missing data for PHP request duration as well, with a peak at 3.25 seconds/request:

redisvsapcu2018 12

Nginx performed identically with/without Redis, with ~12 active connections and ~9 requests/s. For APCu test, since PHP stopped responding, connections started piling up to 130 and these timeouts were detected by loader.io which interrupted the test:

redisvsapcu2018 13

SQL requests were increased by a factor 3 for the test without cache, with around 700 queries/s. Not the same number of requests for Redis and APCu, proof that both cache methods are developed differently and do not cache the same objects:

redisvsapcu2018 14

This translated to an increased cache hit ratio of ~570 queries/s, proof that MySQL is correctly configured for this system. Few hits for Redis, and even less for APCu:

redisvsapcu2018 15

And finally, here is the reason of the crash: too many threads opened to the SQL database during APCu test. The cap is 16:

redisvsapcu2018 16

That’s because of PHP. Since the PHP requests didn’t end, more and more connections were opened until the cap was reached.

This plugin clearly has not implemented APCu cache correctly.

OPcache status after test with op-gui.php:

[row]
redisvsapcu2018 17 redisvsapcu2018 18

[/row]

APCu cache stats with apc.php:

redisvsapcu2018 19

redisvsapcu2018 20

770,000 hits for APCu cache, and the test didn’t finished. That’s already more than 10 times the number of Redis operations: 1,200 * 60s = 72,000, give or take. Both object cache implementations are NOT the same, they do not cache the same number of objects. This is why PHP stopped responding, too many transactions per second for my small EC2 instance.

This plugin is rigged, at least for APCu implementation.

 

W3 Total Cache Load Test

w3 total cache banner 772x250

Plugin Configuration

Good… W3 Total Cache seems to be more serious.

  1. You can chose which Redis database to use.
  2. Problem: You cannot configure Unix socket or anything else than TCP.

redisvsapcu2018 21

Interestingly enough, they still cover Xcache and eAccelerator, which are completely deprecated.

 

Redis VS APCu Load Test Results with loader.io

Control Test Results:

redisvsapcu2018 22

Average response time = 670ms, 16 clients/s. Roughly the same than with the other plugin. This is expected.

Redis Test Results:

redisvsapcu2018 23

Average response time = 640ms, 16 clients/s. Slightly better than the control.

APCu Test Results:

redisvsapcu2018 24

Average response time = 640ms, 16 clients/s. This is clearly better than Redis (by 100ms), as expected.

 

Redis VS APCu Netdata Analysis

Now, you can see what I’m talking about when I mentioned the difference in code implementation. This plugin clearly takes advantage of APCu and do the same caching job as with Redis. We can confirm the slight advantage for APCu over Redis with CPU usage: 80% (APCu) < 90% (Redis/no cache)

redisvsapcu2018 25

That’s because APCu is integrated in PHP as an extension and the difference in CPU usage is certainly due to the increased load in TCP networking for Redis as seen below:

redisvsapcu2018 26

Don’t be fooled by the fact that the same bandwidth is used during the first test without cache and with Redis, below is the proof that the 3 tests are different, with the number of IPv4 packets exchanged:

redisvsapcu2018 27

Overall, using Redis slightly increases CPU usage, for itself and for PHP that needs to communicate with it:

redisvsapcu2018 28

Redis operations: ~1200/s

redisvsapcu2018 29

Redis bandwidth: ~38MB/s

redisvsapcu2018 30

The PHP-fpm pool responded well, with the same number of requests processed: ~8.5/s

redisvsapcu2018 31

No problems of over duration for PHP requests:

redisvsapcu2018 32

Nginx performed identically in the 3 scenarios, with ~12 active connections and ~9 requests/s:

redisvsapcu2018 33

SQL requests were increased by a factor 3 for the test without cache, with around 700 queries/s. Same number of requests for Redis and APCu, proof that both cache methods do cache the same objects:

redisvsapcu2018 34

This translated to an increased cache hit ratio of ~570 queries/s (81% hit ratio), proof that MySQL is correctly configured for this system:

redisvsapcu2018 35

Roughly the same number of connections to MySQL database for all 3 tests:

redisvsapcu2018 36

OPcache status after test with op-gui.php:

[row]

redisvsapcu2018 37 redisvsapcu2018 38

[/row]

APCu cache stats with apc.php:

redisvsapcu2018 39

redisvsapcu2018 40

64,000 hits for APCu cache. That’s roughly the number of Redis operations: 1,200 * 60s = 72,000, give or take. Both object cache implementations are the same, they cache the same number of objects. Perfect score for W3 Total Cache.

 

Redis VS APCu Winner: It Depends

If we look only at CPU performance for Redis VS APCu, technically, the clear winner is indeed a local APCu server.

Just because one plugin didn’t implement APCu correctly doesn’t mean it’s more complex to develop. W3 Total Cache makes a perfect example of best practice in this matter. The load test proved that APCu is slightly faster at accessing cached objects than Redis, as expected.

Unfortunately, this plugin is the only one available to prove it. Therefore, having only one plugin available within a list of deprecated ones indicates a shift in PHP user focus: APCu is being slowly pushed down the side of the road, and therefore discarded as a standard object cache solution. That’s unfortunate but there is a good reason for that: it’s a local server cache. Its content cannot be replicated across instances or clusterized as Redis or Memcached.

Next best winner between Redis VS APCu, all things considered: Redis.

Redis is the current technology to install and maintain. It’s not meant to be a local cache server for static pages though. The slight CPU and network overhead induced for a classic blog overcomes the benefits, as shown in the load tests. It’s clearly a cache solution for heavy loaded, heavy connected application servers.

Team Cook

Self-taught IT guy, and Communist slayer for the past 40 years. Who controls the information, controls the future.

0 0 votes
Article Rating
Subscribe
Notify of
guest

7 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Frands
Frands
4 years ago

Thank you so much for this in depth comparison with good picture documentation.
I helped me A LOT to choose the right cache for Nextcloud.

Alan
Alan
Reply to  Frands
4 years ago

What did you choose for NextCloud?

G RAJU
3 years ago

I decided to install apcu for PHP 7 cauz it’s local server

Rad
Rad
3 years ago

This is idiotic – you did not run even one caching test successfully, so none of this information is accurate or useful… You essentially just compared several WP plugin’s caching implementations, but not the power of the caching technology at it’s core…

7
0
Would love your thoughts, please comment.x
()
x