How tests work

This page describes how the numbers on this site are produced, so you can judge them and reproduce them.

The test box

A KVM virtual machine: 4 vCPU on an AMD Ryzen 5 PRO 3600 host (KVM).
8 GB RAM for the idle measurements on the RAM board. Some functional tests use a smaller machine, and those guides state the size they used.
Ubuntu 24.04, kernel 6.8, from Canonical's official cloud image, verified against its published checksum.
Docker Engine 29.9.2, Compose 5.6.1, installed from Docker's official apt repository, with default daemon settings unless a guide says otherwise. Later runs pull images through Google's public Docker Hub mirror (mirror.gcr.io), because Docker Hub rate-limited this server; that changes where an image is downloaded from, not how it runs.

Every run starts from a throwaway copy of the same base disk and the machine is destroyed afterwards, so one test cannot affect the next.

The host is a shared server that runs other work, and several test machines sometimes ran on it at the same time. Memory readings are not much affected. Timings are, so they are repeated and read best as comparisons within one guide, not as absolutes. The host's load is saved with each run.

What each number means

RAM at idle: the sum of the memory reported by docker stats across a stack's containers. It is the median of 12 samples, taken after a settling period of at least two minutes. Docker's figure excludes reclaimable page cache. The state is a fresh install with no users or content, and some apps are still at their setup wizard.
CPU at idle: the same samples, added across containers. 100% equals one vCPU. Short bursts between samples will not appear.
Images on disk: the size reported by docker compose images. Docker 29's containerd image store includes compressed layers, so this is larger than the unpacked size most sites quote.
Ready after: seconds from docker compose up -d to the first valid response or readiness check, with images already downloaded. Some apps keep doing setup work after this point.

Guide-specific tests, such as backups, load tests and failure drills, repeat timed steps three times and report the median and range unless the guide says otherwise.

What gets recorded

For each test: the script that ran, tool and image versions, the date, the host load and the raw output. Tables on the site are built from those saved files when the site is generated, and compose files shown on a page are read from the same files that ran. A number in a table therefore cannot drift from its source.

Test passwords are throwaway values such as lab-only-not-a-secret. If you copy a compose file, replace them.

Limits you should keep in mind

It is a virtual machine. Disk speed, network speed and failure behavior differ from real hardware. Failed-disk tests use virtual disks, which do not fail the way real drives do.
No wattage, no heat, no noise. None of these can be measured from here, and no claims are made about them.
No GPU. Transcoding and AI results are CPU-only.
Short runs. Tests last minutes to hours. Slow memory growth, database bloat and long-term reliability are out of reach.
Isolated network. The box has no public ports, so nothing has been tested against real internet traffic, and mail deliverability cannot be tested at all.
Dated results. Versions change quickly. Each page shows what was tested and when, so treat older results accordingly.

Last updated: 2026-11-01