Advanced MediaWiki Caching Strategies for High-Traffic Sites

Why caching is the lifeblood of a bustling wiki

Imagine a downtown coffee shop at 8 am on a Monday – the line snakes around the block, the barista is frantically grinding beans, and the espresso machine is humming nonstop. That’s what a popular MediaWiki installation feels like during a traffic surge. If you don’t have a solid caching plan, every page view ends up as a fresh, full‑blown request to the database, and before you know it the whole site crawls slower than a snail on a sticky note. The good news? Properly layered caching can turn that chaotic rush into a smooth, well‑oiled espresso machine, serving pages in a blink while the backend rests easy.

Layered caching – not just a buzzword

MediaWiki isn’t a one‑size‑fits‑all when it comes to caching. Think of it as a three‑tier cake: the bottom layer is the object cache, the middle is the parser cache, and the frosting on top is the HTTP reverse proxy. Each tier has its own purpose, its own knobs to turn, and its own set of quirks. Ignoring any one of them is like leaving the cherry off the cake – it still works, but it’s missing that final touch.

Object cache: Redis, Memcached, or the built‑in file store?

At the core of MediaWiki’s object cache sits $wgObjectCacheType. By default it falls back to the file store, which, let’s be honest, is about as fast as a horse‑drawn carriage in the age of hyperloops. Switching to Redis (type 2) or Memcached (type 1) can shave off milliseconds that, when multiplied by thousands of requests per second, translate into seconds saved per minute.

// Example: Enable Redis as the object cache
$wgObjectCacheType = 2; // 2 = Redis
$wgObjectCacheServers = [ [ 'host' => '127.0.0.1', 'port' => 6379 ] ];
$wgRedisServers = [ [ 'host' => '127.0.0.1', 'port' => 6379 ] ];
// Optional: set a prefix to avoid key clashes
$wgCachePrefix = 'mywiki';

Redis also lets you set an expiry policy per key, which can be handy for things like recent changes or user‑specific data that changes often. Remember, though, that each extra key you store consumes memory, and memory isn’t infinite – the devil’s in the details when you start pushing gigabytes of cache entries.

Parser cache: Stop re‑parsing the same page

Every time a user requests a page, MediaWiki normally parses the wikitext into HTML. That parsing step can be expensive, especially with heavy extensions like Scribunto or VisualEditor. The parser cache (controlled via $wgParserCacheType) stores the rendered HTML for a given revision, so subsequent requests can skip the heavy lifting.

// Enable the built‑in APCu parser cache (type 5)
$wgParserCacheType = 5;
$wgParserCacheEnableForLoggedInUsers = false; // saves memory

If you’re already using an object cache like Redis, you can point the parser cache there as well – just set $wgParserCacheType = CACHE_ANYTHING and let MediaWiki pick the best backend. A handy rule of thumb: for wikis with a lot of read‑only traffic (think documentation sites), crank the parser cache up to the max; for heavily edited wikis, you may want to keep it a tad lower to avoid serving stale content.

HTTP reverse proxy: Varnish, Nginx, or Cloudflare?

Layer three sits at the edge of your network – the HTTP reverse proxy. It catches the request before it even hits MediaWiki, checks if a cached copy of the page exists, and serves it straight away. Varnish is the classic choice for MediaWiki, but modern setups often lean on Nginx’s proxy_cache or even a CDN like Cloudflare for global distribution.

# Simple Nginx reverse‑proxy cache snippet
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=wiki_cache:10m max_size=5g inactive=60m use_temp_path=off;

server {
    listen 80;
    server_name wiki.example.com;

    location / {
        proxy_pass http://backend_upstream;
        proxy_set_header Host $host;
        proxy_cache wiki_cache;
        proxy_cache_valid 200 302 10m;
        proxy_cache_valid 404 1m;
        add_header X-Cache $upstream_cache_status;
    }
}

Notice the add_header X-Cache line? It’s a little debugging trick that shows you whether the request was a hit or a miss – handy when you’re fine‑tuning the cache rules. Also, remember to respect Cache-Control headers sent by MediaWiki; you don’t want to cache private user pages for the whole world.

Edge caching with a CDN – the cherry on top

If your audience is truly global, a CDN can bring the content physically closer to the reader. Cloudflare, Fastly, or Akamai can cache entire HTML pages, static assets, and even provide automatic TLS termination. The trick is to configure $wgCachePages and $wgCacheEpoch so that MediaWiki emits proper Cache‑Control headers, letting the CDN know when it’s safe to serve a cached copy.

// Let MediaWiki send a long cache lifetime for anonymous users
$wgCachePages = true;
$wgCacheEpoch = '2024-01-01T00:00:00Z'; // bump this when you need a global purge
$wgCachePages = true;
$wgCacheDirectory = "$IP/cache"; // for file‑based cache fallback

When you need to purge a page across the CDN, a simple API call to the CDN provider is usually enough – no need to touch the wiki core. Just keep a note of the Cache‑Tag header MediaWiki can emit; it makes bulk invalidation a breeze.

Monitoring, busting, and the art of graceful degradation

All that caching sounds great until something breaks. That’s why you need solid observability. Tools like Prometheus + Grafana can scrape metrics from MediaWiki (MediaWiki:CacheHitRatio, ParserCache:Hits, etc.) and from your reverse proxy (Varnish stats, Nginx cache hits). Set alerts for a sudden dip in hit ratio – it could mean a mis‑configured $wgCachePrefix after a server rename, or a stale memcached restart.

When it comes to busting caches, don’t go full‑blown “flush everything” unless you absolutely have to. Use purge links for individual pages, or run php maintenance/purgeCache.php with a list of page IDs. For Redis, the FLUSHDB command wipes the entire DB – be careful, that’s like emptying the pantry in the middle of dinner prep.

Pitfalls and gotchas – lessons from the front lines

• Stale parser cache on edits: If you turn off $wgParserCacheEnableForLoggedInUsers to save memory, logged‑in users may see older versions of pages after a quick edit. Test with a real user account before committing.
• Cache key collisions: Using the same Redis database for multiple wikis without a unique $wgCachePrefix leads to cross‑wiki contamination. The cache will silently serve the wrong page – a nightmare to debug.
• Object cache size limits: Memcached defaults to 64 MB per item. Storing huge data structures (like full revision histories) can silently fail. Keep an eye on memcached -vv logs.
• CDN edge‑case headers: Some CDNs ignore Vary: Cookie and will cache private pages for everyone. Double‑check your CDN’s documentation and add Cache‑Control: private on user‑specific pages.
• Reverse proxy bypass: If you accidentally set proxy_cache_bypass $http_cookie in Nginx, every request with any cookie (including harmless analytics cookies) will bypass the cache, killing performance.

Putting it all together – a sample configuration checklist

1. Choose an object cache backend (Redis recommended). Set $wgObjectCacheType and $wgObjectCacheServers.
2. Enable parser caching with $wgParserCacheType. Tune expiry based on edit frequency.
3. Deploy a reverse proxy (Varnish or Nginx). Configure proxy_cache rules and add cache‑status headers.
4. If you have a global audience, add a CDN. Ensure MediaWiki emits proper Cache‑Control and Cache‑Tag headers.
5. Instrument metrics: expose MediaWiki cache stats, monitor reverse proxy hit ratios.
6. Set up automated purge scripts for bulk actions (e.g., after a massive CSS update).
7. Document your $wgCachePrefix and any custom key namespaces.

Final thoughts

High‑traffic MediaWiki sites are a bit like high‑performance race cars – they look sleek, but under the hood there’s a whole orchestra of moving parts that need constant attention. Caching isn’t a one‑off switch; it’s a layered strategy that demands regular tuning, a dash of monitoring, and occasional “oh‑no‑what‑did‑I‑just‑break” moments. Yet, when you get it right, the difference is night‑and‑day: users get pages in a flash, servers stay cool, and the wiki team can finally sip that well‑earned coffee without worrying about a server overload.