How we configure huge pages in ClickHouse Managed Postgres

\->Scroll to top

<-Back

Blog
/
Engineering

Copy pageCopied!More actions

View as Markdown Open this page in Markdown
Open in ChatGPT Ask questions about this page
Open in Claude Ask questions about this page
Open in v0 Ask questions about this page

How we configure huge pages in ClickHouse Managed Postgres

image 512x512 1

Kaushik Iska

Jul 14, 2026 · 6 minutes read

What are huge pages, and why do they matter for Postgres? The OS hands out memory in 4KB pages, and the CPU keeps a small cache of virtual-to-physical translations, the TLB. A huge page is the same memory in much bigger units, 2MB or 1GB, so one page-table entry covers up to 262,144x more ground.

Postgres is unusually sensitive to page size. Its cache, shared\_buffers, is one big shared-memory segment, and every backend process maps it through its own page tables. At 4KB pages, a 100GB cache costs each backend about 200MB of page-table entries, so 100 connections burns 20GB of RAM on page tables alone; with 2MB huge pages, tens of MB. And a TLB holds a few thousand translations, which covers just a few MB of that cache at 4KB, so each 2MB page gives a slot 512x the reach, the hot working set stays in the TLB, and reads stop stalling on page-table walks.

In ClickHouse Managed Postgres every server runs its buffer cache on huge pages. Getting that to hold reliably takes three pieces: reserving the pages early, refusing to run without them, and sizing shared\_buffers so the whole shared memory segment lands on the reserved pool exactly.

A quarter of the machine's memory is reserved as huge pages, mirroring the `shared_buffers` \= 25% of RAM rule in the base configuration. The reservation runs before Postgres ever starts, and it flushes caches and compacts memory first so the kernel can still find contiguous 2MB blocks:

1echo 'vm.nr_hugepages = ' > /etc/sysctl.d/10-hugepages.conf
2sync
3echo 3 > /proc/sys/vm/drop_caches
4echo 1 > /proc/sys/vm/compact_memory
5sysctl --system

Copy command

The allocation is then verified against `HugePages_Total` in `/proc/meminfo`, and provisioning fails if the kernel comes up short. Huge pages can only be reserved reliably while memory is unfragmented; on a machine that has been under load, the same request routinely fails.

Postgres is pinned to `huge_pages = 'on'`, so it either gets its pages or refuses to start. The default, `'try'`, silently falls back to 4KB pages with nothing in the logs, and every property described above quietly disappears. A refusal at startup is diagnosable; a silent fallback is discovered months later, usually by a profiler.

There is a runtime check, too. The postmaster's `/proc/<pid>/status` records how much of its address space sits on hugetlb pages; on the demo box below it reads:

1$ grep HugetlbPages /proc//status
2HugetlbPages:   700416 kB

Copy command

`HugetlbPages: 0` on a server that should be using huge pages means the fallback happened.

Postgres's shared memory segment is bigger than shared\_buffers: it also carries buffer descriptors, WAL buffers, and lock tables. On the demo box below, `shared_buffers = 32GB` produces a 32.75GB segment, about 750MB of overhead. If shared\_buffers alone filled the reserved pool, the full segment would not fit, and `'on'` would refuse to boot. The sizing works in two passes. First point shared\_buffers at the entire pool, then ask the binary what the full segment would actually cost, without starting the server:

1postgres -D  -C shared_memory_size

Copy command

Subtract the difference back out of shared\_buffers, round down to whole 8KB blocks, and write the result:

1# conf.d/002-hugepages.conf
2huge_pages = 'on'
3huge_page_size = 0        # use the system default, 2MB
4shared_buffers = kB

Copy command

The overhead is measured rather than assumed because it shifts with version and settings, and measuring at the larger size errs a few pages under the reservation. The reserved pool and the rest of memory are then accounted separately: huge pages do not count toward the kernel's commit limit, so memory overcommit is tuned against the remaining 75% of RAM.

We put the claim about page tables on a single EC2 box: an `r7i.4xlarge` (16 vCPU, 128GB), Postgres 16 with `shared_buffers = 32GB`, and a pgbench dataset whose 15.6GB accounts table is loaded fully into the cache with `pg_prewarm`. Each test connection then runs one full sequential scan of that table, which makes its backend touch every cached page and build its complete page-table mapping, and holds the connection open. We sampled the kernel's `PageTables` counter from `/proc/meminfo` at 25, 50, 100, and 200 connections, once with 4KB pages and once with 2MB huge pages. Same box, same data, same access pattern; the only variable is the page size.

With 4KB pages, page tables grow by 31.1MB per connection, matching the arithmetic (15.6GB of touched cache / 4KB pages x 8 bytes per entry = 31.2MB), and reach 6.1GB at 200 connections, kernel memory spent on bookkeeping for 15.6GB of data. With 2MB huge pages the same 200 connections cost 111MB, about 0.5MB per connection.

| Connections | PageTables, 4KB | PageTables, 2MB huge pages | | ----------- | --------------- | -------------------------- | | 0 | 48 MB | 5 MB | | 25 | 789 MB | 18 MB | | 50 | 1.53 GB | 31 MB | | 100 | 3.06 GB | 58 MB | | 200 | 6.12 GB | 111 MB |

The `huge_pages = 'on'` behavior is equally visible. With no pool reserved, Postgres refuses to start and says why:

1FATAL:  could not map anonymous shared memory: Cannot allocate memory
2HINT:  This error usually means that PostgreSQL's request for a shared
3memory segment exceeded available memory, swap space, or huge pages.

Copy command

Throughput moves too, though less dramatically than the memory: select-only pgbench at 100 clients ran 373,083 TPS on 4KB pages and 418,087 TPS on 2MB huge pages, a 12% difference on an otherwise identical box. The structural change is in the memory: page tables that scale linearly with connections versus page tables that stay flat.

Huge pages turn shared\_buffers from a per-connection page-table tax into a flat, translation-friendly cache. Reserving the pool before fragmentation, running `huge_pages = 'on'` so a broken setup fails at boot instead of degrading silently, and sizing the segment to the reservation with the numbers the postgres binary itself reports are what make the configuration hold in production rather than only on a fresh box.

Try Postgres managed by ClickHouse

ClickHouse + Postgres has become the unified data stack for applications that scale. With Managed Postgres now available in ClickHouse Cloud, this stack is a day-1 decision.

---

Share this post