Scaling a large XenForo forum: performance and architecture

A large XenForo board does not get slow because you forgot WebP. It gets slow because every guest homepage still runs PHP-FPM, because InnoDB’s buffer pool is smaller than the working set, because Redis and the page cache share one crowded instance, because a permission save multiplies users-or-groups by nodes and locks the tables, because search is still a MySQL full-text table, or because the job runner is so backed up that the next page view is doing yesterday’s work.

This is not the Core Web Vitals guide. That article already owns Gzip, debug and fullJs off, guest page cache as a CWV tool, WebP, the icon sprite, Cloudflare Auto Minify, and the first 60 minutes when members say the site died. Read it. Do those things. Then come back. This article is architecture: how the processes sit on the box, how cache contexts are split, how guests and members take different paths, how jobs and attachments behave as you grow, why a permission rebuild is an availability event, and when official XenForo Enhanced Search is the next lever rather than another VPS resize.

Official cache and job names below come from the public manuals (docs.xenforo.com/manual/config/cache and the configuration docs). Community operations (the U×N rebuild lock, InnoDB knobs, “do not invent core S3”) are labeled as such.

What “large” means here

Ignore registered-user vanity. Use load.

Signal	You are still “medium”	You are in this article
Concurrent PHP	A burst of 20 workers is news	Workers busy in ordinary evenings
Homepage guests	Mostly humans	Search, Discord unfurls, scrapers, plus humans
xf_post	Fits in RAM with room	Working set no longer fits the buffer pool
Search	MySQL full-text is fine	Search is the incident, or the table is the disk hog
Permission save	A few seconds	Minutes, timeouts, “the forum is down”
Jobs	Cron is empty	Rebuilds spill into page requests
Attachments	Local data/ on the same disk	Disk, inode, or backup window is the constraint

If you are still medium, do not build a two-box topology for sport. Turn on official cache, split the page context, update PHP, and go back to nodes and moderation. Architecture is what you add when a single well-tuned box is telling the truth.

1. The request path: nginx (or equivalent) and PHP-FPM

XenForo is PHP. A request that is not satisfied by a cache becomes: web server → PHP-FPM worker → MySQL (and maybe Redis, and maybe Elasticsearch). Scaling the application tier means enough workers, not too many workers, and not making them wait on disk.

A practical single-host shape that still looks like official XenForo:

• nginx (or Apache / LiteSpeed — pick one) terminates TLS, serves /data/ and static assets, and try_files into index.php when you use friendly URLs. Official nginx notes: protect internal_data, src, install/data, install/templates. Those directories are not public, even when you are “just testing scale.”
• PHP-FPM pool dedicated to the board. Official 2.3 floor is PHP 7.2, recommended 8.3 or 8.4 for opcode cache and execution time. OPcache must be on. pm = dynamic or ondemand with a pm.max_children you sized from RAM, not from a blog post about WordPress.
• One FPM pool per board. If you host two XenForo sites on one box, two pools. A rebuild or a traffic spike on board A should not take every worker board B needs.

Worker math is boring and mandatory. Each FPM child is a slice of RAM (measure memory_get_peak_usage on a logged-in thread view, do not guess). If the box has 8 GB, MySQL wants a real buffer pool, Redis wants a real dataset, and the OS wants a page cache, you might only have 2–3 GB for PHP. That is 20–40 children, not 200. Too many children: the box swaps, every request dies. Too few: the listen queue grows, TTFB explodes, you blame “XenForo.”

What this article will not do: publish a magic pm.max_children number, or tell you to put Varnish in front as if it were core. Community stacks (LiteSpeed Cache, nginx FastCGI cache, Cloudflare) are guest HTML offload. They are valid. They are also easy to stack on top of official $config['pageCache'] until nobody knows who owns a purge. One guest-cache owner. The CWV article said it. Architecture agrees.

Horizontal PHP (two app boxes, one database) is possible in the abstract. XenForo official docs do not hand you a clustering manual. If you go there, you are in custom operations: shared data/ and internal_data/ (or a decision about what lives where), sticky sessions or sessions in Redis, identical config.php, a deploy that does not split versions. Do that only after the single box is honestly maxed. Most “we need a cluster” tickets are an unsized buffer pool or a permission rebuild.

2. MySQL / MariaDB and InnoDB

Official 2.3 database line: MySQL 5.7 minimum, MySQL 8.0 or MariaDB recommended, Percona listed as compatible. Required PHP extension: mysqli. That is the supported floor. The scaling floor is InnoDB for the tables that take writes, a buffer pool that holds the hot pages, and a refusal to run DDL and permission rebuilds at peak.

Community practice, labeled as community because the official cache manual does not publish my.cnf:

Knob / habit	Why it shows up on large XF
InnoDB for almost everything	MyISAM table locks on write
Stock MyISAM leftovers on a default install	Community lists xf_search_index, xf_session, xf_session_admin, xf_session_install
xf_search_index on a small VPS	Leaving MyISAM can be rational until you move search off MySQL
innodb_buffer_pool_size	50–75% of RAM on a dedicated DB box, or table size + headroom if the DB is smaller than RAM
innodb_flush_log_at_trx_commit = 1	Durability. 2 or 0 only during a known write storm (import, upgrade), then put it back
innodb_flush_method = O_DIRECT	Avoid double buffering through the OS page cache
Query cache off	A busy forum is writes; the old query cache locks

Put MySQL on fast local disk. Remote MySQL on a cheap network is how every page waits. Official importer docs make the same point in another context: imports are fastest when both databases are local. Live traffic is not different.

Do not invent read replicas as a XenForo feature. Official manuals do not describe a built-in primary/replica adapter for forum reads. If you build one, you own lag (a reply that does not appear), session weirdness, and failover. That is a DBA project, not an ACP checkbox. Get the buffer pool and the search engine right first.

Watch SHOW PROCESSLIST during incidents. A large board’s typical villains are: a permission rebuild, a search query on xf_search_index, a member-stat or widget that scans too far, an add-on doing fetch() on a huge finder (official developer warning: do not do that; use jobs). Architecture is identifying which of those is allowed to run on the primary during the evening peak.

3. Redis as the official cache provider

XenForo 2.3 cache is Symfony Cache. Official public cache manual still names the config.php providers: ApcCache, Filesystem, Memcached, Redis, WinCache, XCache. Redis is not a third-party idea. It is a first-class provider.

A minimal official-shaped block (defaults in the manual: port 6379, optional password, database 0, persistent false):

$config['cache']['enabled'] = true;
$config['cache']['provider'] = 'Redis';
$config['cache']['config'] = [
    'host' => '127.0.0.1',
    'password' => 'password',
];

What that cache holds, in architectural terms: the data registry, compiled templates, CSS cache, permission combinations. Without it, 2.3 still runs and reads xf_data_registry and friends from MySQL on every page. On a large board that is a steady tax. Application cache in Redis is the first scaling lever that is still “stock XenForo.”

Sessions can live in cache ($config['cache']['sessions'] = true). Official warning: do not fill APC with sessions — if the store evicts, people cannot stay logged in. Redis has maxmemory policies too. If you set allkeys-lru on a tiny instance that also holds sessions and guest HTML, you will log people out to make room for a cached homepage. That is why the next section exists.

Community operations around Redis (not official ACP): install phpredis, confirm php -m, prefer a local socket or 127.0.0.1 so you are not adding a network hop, be careful with persistence if a heavy BGSAVE pauses the box. High-availability write-ups in the community mention Sentinel; they also say multi-master clustering is not the path those add-ons took. If you are not sure, run one Redis on the same box as PHP, with enough RAM, and monitor evictions.

Do not enable Redis for the first time during a permission rebuild. One variable.

4. A dedicated page-cache context

Guest page cache is official, and it is a different context from the application cache. Official 2.1+ rule: $config['pageCache']['enabled'] = true only works if you also define $config['cache']['context']['page'] with its own provider / instance. They recommend a separate cache instance so huge guest HTML does not evict sessions and the registry. A cached response sends X-XF-Cache-Status: HIT. Optional knobs the manual names: lifetime (default 300 seconds), recordSessionActivity (default true), routeMatches to limit which routes are cached.

$config['pageCache']['enabled'] = true;
$config['cache']['context']['page'] = [
    'provider' => 'Redis',
    'config' => [
        'host' => '127.0.0.1',
        'database' => 1, // conceptual: a separate instance or DB index you actually isolated
    ],
];

Treat that snippet as a shape, not a paste-and-forget. The architectural requirement is isolation: page HTML is large and popular; registry keys are small and sacred. Separate Redis databases are a weak isolation if you still share one maxmemory. Separate Redis instances (or at least a hard memory budget you understand) are the real split. Filesystem as the page-cache provider is legal and slower; it can still protect Redis from HTML blobs on a medium board.

Page cache is off by default because it eats RAM. On a large board it is usually on, because the alternative is PHP-FPM rendering the same guest homepage thousands of times. Measure with the response header. HIT on the second guest request is the test. MISS forever means the context is miswired. HIT for a logged-in moderator means you have a cookie / cache-key incident — stop and fix it before you scale traffic.

If you already offload guest HTML at nginx or LiteSpeed, you may leave XenForo page cache off. Two layers both believing they own / is an architecture bug. Pick the layer closest to the visitor that you can purge on thread create. Document the owner in the runbook.

5. Guest versus member: two sites, one codebase

A large XenForo is two applications that share templates.

Guests (including Googlebot, Discord unfurls, most “I clicked a link”) should almost never execute the full permission + widget + conversation chrome path. They get page cache or an edge cache. They see public nodes only. Their LCP problem is payload and TTFB; the CWV article owns the payload. This article owns the rule: guest traffic must not equal guest PHP.

Members bypass page cache. They need a warm application cache (registry, permissions, CSS), a fast InnoDB read of the thread, and widgets that do not issue a new heavy query on every hit. Widget cache lifetime exists for this. A “members online” block that hits the database every request is a choice. So is a custom widget that scans posts. The widgets guide is how you configure instances; at scale, treat each instance as a query budget.

Logged-in scale problems that are not “need more RAM”:

• What’s New and similar global queries as the default index for everyone. Fine for medium. On large, consider whether the index page route should be cheaper for some groups.
• Notices and promotions that run expensive criteria on every request. Criteria are powerful; they are not free.
• Add-ons that poll (live chat, “who is viewing this thread”). Each poll is a PHP-FPM child you could have spent on a real page.
• Admin browsing production with debug on. Official: debug “severely hurts performance” and prints internal queries. An admin session is not a reason to make the pool slower for members.

You will never make a logged-in thread view as cheap as a cached guest homepage. Stop trying. Make it a single-digit-millisecond cache hit for registry + a primary-key InnoDB read, and go home.

6. Jobs and cron: deferred work is part of the architecture

Official job runner: $config['jobMaxRunTime'] = 8 (seconds) before a job yields. Official developer rule: do not $finder->fetch() every post; batch through the job system. The same rule is how you run Tools rebuilds.

On a small board, cron is a footnote. On a large board, cron is a queue:

• Search index updates
• Sitemap generation (Setup → Options → XML sitemap generation)
• User group promotions (hourly, recently-active only — official)
• Mail if you chose the slower transports
• Attachment and image rebuilds (WebP conversion of history is a job, not a button you click at 20:00)
• Counters, cleanups, add-on tasks

If cron is dead or starved, deferred work starts riding the next visitor request. Members experience that as “the site is randomly stuck.” Check Tools → Cron entries and the job list before you resize the VPS. A pile of jobs is an architectural signal: you started three rebuilds, or mail is retrying, or an add-on is enqueueing faster than workers drain.

Operational rules that belong in the runbook:

• One heavy job at a time on production. Not “rebuild search” + “rebuild all attachments as WebP” + “rebuild caches” on the same evening.
• Prefer CLI for long rebuilds so you are not fighting the web-server time limit. The importer docs make the same CLI-versus-browser point; rebuilds are the same idea.
• Schedule permission-adjacent work (next section) in a freeze window, not when jobs are already behind.
• Watch mail. SMTP (each mail by XenForo) is officially the slower transport; PHP built-in mail is preferred in the email options docs. A large board that suddenly “feels slow” after a mass Contact users pass is often the mail job, not InnoDB.

$config['jobMaxRunTime'] is not a performance trophy. Raising it makes a single request hold a worker longer. Leave it sane unless you know why.

7. Attachment storage

Uploads live under data/ (public) and internal_data/ (private). Official backup and restore doctrine treats those two directories as part of the site: restore files + database at the same XenForo version, or avatars and attachments vanish. Empty data/ / internal_data/ after a “I only imported the SQL” move is a classic outage.

At scale, attachments become:

• Disk — years of screenshots. 2.3 WebP conversion (CWV article) shrinks new and, if you rebuild, old files. That rebuild is a job and a CPU event.
• Inodes — many small files. Filesystems care.
• Backups — a 400 GB data/ is why your nightly backup never finishes. That is an architecture ticket: exclude with a documented restore story, or snapshot the volume, or split the volume.
• PHP memory — $config['maxImageResizePixelCount'] defaults to 20,000,000; raising it lets larger images through and lets one upload spike a worker. Raise it for a photographer community, not for one complaint.

Do not invent official core S3. XenForo’s public manuals do not describe a built-in “put attachments on S3” checkbox as a core feature. Community add-ons and object-storage patterns exist; if you use one, name it in the runbook and test restore. The conceptual advice that is fair: treat attachment bytes as a volume you can snapshot, not as something MySQL should hold, and do not put internal_data on a public bucket.

Client-side resize (attachment manager, HTML5, official reason: less server CPU) is a scale feature disguised as UX. Turn it on before you shop for object storage.

Image proxy ($config['proxyUrlFormat']) fetches remote images so you do not hotlink. It is also a cache of other people’s bytes on your disk. On a large board with loose BB-code image permissions, the proxy store is a second attachment tree. Watch it.

8. The permission rebuild: U × N as an availability event

Community sources describe XenForo permission rebuild cost as multiplicative. Groups × nodes is the version the CWV article already quoted (a medium board, ~80 nodes and ~60 groups, two to four minutes; ~700 nodes, timeouts and table locks). Operators also talk about it as users × nodes when a change forces a wide rebuild of cached combinations. Either way, the architectural fact is the same: a permission save on a large tree is not an edit. It is a lock.

That is why the permissions starter told you not to create a group for every mood, and why flattening ghost nodes is a scale project. Every extra group and every extra node is a cell in a matrix the board must be willing to rebuild.

Runbook:

• Never save a primary-group change, a private-node flip, or a tree reparent at peak. Announce a window. Put a notice up. Staff only.
• Analyze permissions (Groups & permissions → Analyze permissions) on a sample user before you change the tree, so you are not “trying another Yes” in production.
• Prefer Inherit and fewer groups. Never is a sledgehammer; it also makes the next admin add another group to work around it.
• Promotions add secondary groups on an hourly cron for recently-active users. That is cheaper than a hand edit of ten thousand primaries. Empty promotion criteria never auto-award; disabling a promotion does not demote. Batch update is the whole-register tool — and it is also a write storm. Window it.
• If a rebuild is already running, do not start a second one, do not start XFES reindex, do not rebuild templates. Watch the process list. Wait.

This lock is the reason “we should just add a group” is an architecture decision on a large board. It is also not a reason to migrate to Discourse. It is a reason to keep the matrix small.

9. When XFES / search is the next lever

Stock XenForo search is MySQL full-text (xf_search_index in community engine notes). That is fine until it is not. Symptoms that search has become the lever:

• Search is the query in SHOW PROCESSLIST during slowness.
• Members stopped using search because it times out or misses.
• The search table is the thing you cannot fit in RAM, and you are considering moving it to InnoDB on a box that cannot afford it.
• You want similar-threads / better ranking than MySQL full-text will give you.

XenForo Enhanced Search (XFES) is an official add-on, not a random resource. It is a drop-in replacement for MySQL full-text, backed by Elasticsearch or OpenSearch. Official 2.3 materials in the CWV article: Enhanced Search wants ES/OS 7.2+. After install, you configure the connection and reindex. That reindex is a job. Window it. Do not treat XFES as a toggle you flip on the live primary during a launch.

Architecture notes:

• ES/OS is another process with its own heap. Putting it on the same 4 GB VPS as PHP, MySQL, and Redis is how you buy a new incident. Give it a box, or a box-sized slice, or wait until you can.
• XFES does not replace application cache, page cache, or InnoDB. It replaces search. If the homepage is slow for guests, XFES will not help.
• Third-party “Elasticsearch Essentials” style add-ons require XFES. Do not install the satellite first.
• If search is only slow because debug is on or because an add-on hooks every query, fix that. XFES is the next lever when MySQL search is honestly the bottleneck.

Official docs do not publish a post-count at which you must buy XFES. Use the process list, not folklore.

10. A growth ladder (so you do not skip rungs)

Do these in order. Skipping rungs is how you end up with Elasticsearch and a still-unlocked guest homepage.

1. Production hygiene. $config['debug'] = false, $config['development']['fullJs'] = false, Gzip as in the CWV article. PHP 8.3/8.4, OPcache. Cookie path correct so members are actually members.
2. Application cache on Redis (official provider). Confirm the board still updates after a style change + cache rebuild.
3. Dedicated page-cache context, or one external guest cache, not both. Confirm X-XF-Cache-Status and a logged-in bypass.
4. InnoDB buffer pool sized; query cache off; process list understood.
5. Widget and index diet for members. Cache lifetimes. Kill polling add-ons you cannot name a purpose for.
6. Job / cron health. One rebuild at a time. CLI for long work.
7. Attachment volume plan. WebP for new files, client-side resize, backup story for data/ + internal_data/.
8. Shrink the permission matrix. Fewer groups, fewer nodes. Rebuilds in a window.
9. XFES + ES/OS on their own resources when search is the incident.
10. Only then talk about a second app box, object-storage add-ons, or a DBA-owned replica. Official docs will not drive those for you.

If you are about to migrate because “XenForo does not scale,” check which rung you skipped. The stay-or-migrate article is the decision framework; this ladder is the work you do when the answer is stay.

Sample topologies (so the ladder has a floor plan)

Single box, honest large. 8–16 GB RAM. nginx + PHP-FPM + MySQL + Redis on the same machine. Redis application cache on instance A. Page-cache context on instance B or a separate database with a memory budget you can explain. Backups: database dump plus filesystem snapshot of data/ and internal_data/. This is where most “large” XenForo sites should live for years. If this box is slow, name the process (FPM wait, InnoDB disk, Redis evictions, permission rebuild) before you draw a second box.

App + database split. PHP-FPM and nginx on box 1. MySQL on box 2 with a buffer pool that actually fits the working set. Redis next to PHP (avoid a WAN hop on every registry read). data/ on local disk or a volume both app boxes can see if you later add a second app. Sessions in Redis if you will ever load-balance. Official docs will not draw this for you. You own clock skew, deploys, and “which box has the new internal_data.”

Search sidecar. Same as either of the above, plus Elasticsearch/OpenSearch on box 3 after XFES is purchased and a reindex window exists. Do not put ES on the 8 GB all-in-one “just to try.” Heap plus page cache plus InnoDB is how you invent a 3 a.m. OOM.

What these drawings are not. They are not Kubernetes. They are not “official XenForo cluster.” They are the minimum pictures you should be able to annotate with RAM numbers before you buy another invoice.

Attachment and backup runbook (architecture edition)

1. Measure data/ and internal_data/ separately. Avatars and attachments are usually the story; internal_data also holds things you must not put on a public CDN by accident.
2. Confirm a restore drill: files + DB at the same XF version, then fix Board URL and cookie path if the hostname changed (the mistakes article already owns that class of outage).
3. Nightly database backup must finish. If data/ makes the nightly window slip, snapshot the volume instead of pretending tar over 400 GB is a strategy.
4. After a host move, the first guest hit that shows broken avatars is an empty data/ directory, not a “CDN cache.” Check disk before you purge Cloudflare.
5. Image-proxy store is a second tree. If you allow arbitrary remote images, include it in the measurement.

XFES go / no-go

Turn the official search add-on on only when you can tick all of these:

• SHOW PROCESSLIST during slowness names search, not a permission rebuild or a dead Redis
• Members have actually complained that search misses or times out (not just that they never use it)
• You have a box or a sized slice for ES/OS 7.2+, not a leftover GB on the web node
• You have a reindex window and a person watching the job
• Application cache and guest page cache are already on and proven

Three ticks or fewer: you want a faster homepage or a smaller permission matrix, not Elasticsearch.

What not to invent

• Core S3 / “official object storage.” Not in the public manuals cited here. Add-on territory.
• Official read replicas. Not an ACP feature.
• A required third-party Redis add-on. Redis is an official provider in config.php. Community add-ons exist for stats and extras; they are not the reason cache works.
• A CWV checklist. Already written.
• A promise that LiteSpeed + Cloudflare + pageCache + FastCGI is “more scaled.” One guest owner.

When official docs omit a control, this article omitted it too.

Takeaways

• Large XenForo is a PHP-FPM + InnoDB + Redis system. Size workers from RAM, size the buffer pool from the working set, and put the official cache in Redis.
• Page cache is a separate context. Isolate it so guest HTML cannot evict sessions and the registry. Members must bypass it.
• Guests should not equal guest PHP. Members should hit a warm application cache and cheap primary-key reads, not a second homepage renderer.
• Jobs and cron are capacity. A dead queue makes the next page view do yesterday’s work. One heavy rebuild at a time.
• data/ and internal_data/ are the site. Attachments are a volume and a backup problem. Do not invent core S3.
• Permission rebuilds scale with the group × node (and cached-combination) matrix. Treat a save as an availability event.
• XFES is the official next lever when search is the incident, and Elasticsearch wants its own resources.
• Gzip, WebP, icon sprites, and Cloudflare hygiene live in the CWV article. Do not redo them here. Do not skip them either.

A board that scales is a board whose guests hit HTML, whose members hit Redis plus InnoDB, whose admins refuse to rebuild permissions at eight in the evening, and whose next invoice is Elasticsearch only after search has earned it.