Skip to main content

Bench Command

The bench command is a high-performance HTTP load generator built directly into Rumour. It allows you to benchmark a single request .toml file or a whole workflow directory by sending concurrent requests and gathering detailed latency percentiles.

This command is ideal for load testing, identifying bottlenecks, and preventing performance regressions before code changes are deployed.

Basic Usage​

rumour bench <PATH> [OPTIONS]
  • <PATH> — Path to the request .toml file or the workflow directory you want to benchmark.

Options​

OptionShortDefaultDescription
--iterations-n100The total number of requests to execute.
--concurrency-c10The maximum number of concurrent requests to execute in parallel.
--env-file-e✗Path to a workspace environment file (e.g. workspace.env.toml) to use.
--variable-V✗Key-value overrides to customize variables (e.g. -V base_url=http://localhost:3000). Can be passed multiple times.

How It Works Under the Hood​

When you execute rumour bench:

  1. Target Detection: If the path is a file, the engine benchmarks that single request. If the path is a directory, the engine benchmarks the whole workflow directory.
  2. Asynchronous Workers: Rumour uses an asynchronous semaphore configured with your --concurrency value to limit the concurrent requests executing in parallel.
  3. Execution Mode: Every request run is set to Verbosity::Quiet, ensuring that individual request details, logs, or reports are suppressed to avoid slowing down the load generation with console I/O.
  4. Latency Measurement: The engine records the exact duration of each request in milliseconds, sorts the resulting durations, and computes statistical aggregates.
  5. Histogram Bucketing: A 10-bucket histogram distribution is calculated dynamically using the minimum and maximum latencies.

Examples​

Example 1 — Basic Benchmark​

Benchmark a single endpoint with default settings (100 total requests, 10 concurrent):

rumour bench ./extract_pipeline/01_login.toml

Output:

--- BENCHMARK START ---
Target: ./extract_pipeline/01_login.toml
Iterations: 100
Concurrency: 10

--- STATISTICS ---
RPS:                 32.45 req/s
Average:            304.50 ms
Min:                 120 ms
Max:                 892 ms
P50:                 280 ms
P90:                 410 ms
P99:                 795 ms

PASS Benchmark completed successfully.

Example 2 — High Concurrency Load Test​

Generate higher load by running 1,000 iterations with a concurrency of 50:

rumour bench ./extract_pipeline/01_login.toml -n 1000 -c 50

Output:

--- BENCHMARK START ---
Target: ./extract_pipeline/01_login.toml
Iterations: 1000
Concurrency: 50

--- STATISTICS ---
RPS:                 85.12 req/s
Average:            580.20 ms
Min:                 115 ms
Max:                2140 ms
P50:                 490 ms
P90:                 980 ms
P99:                1850 ms

PASS Benchmark completed successfully.

Example 3 — Benchmark with Environment and Variables​

Use a custom environment file and override specific template variables on the fly:

rumour bench ./extract_pipeline/01_login.toml \
  -e ./development.env.toml \
  -n 500 \
  -c 20 \
  -V base_url="https://httpbin.org" \
  -V timeout=5000

Output:

--- BENCHMARK START ---
Target: ./extract_pipeline/01_login.toml
Iterations: 500
Concurrency: 20

--- STATISTICS ---
RPS:                 45.89 req/s
Average:            420.10 ms
Min:                 150 ms
Max:                1250 ms
P50:                 380 ms
P90:                 620 ms
P99:                1100 ms

PASS Benchmark completed successfully.

Interpreting the Output​

The --- STATISTICS --- block reports the following metrics:

MetricFormatDescription
RPSreq/sRequests Per Second: The overall throughput of the benchmark (Total Requests / Total Elapsed Time). Higher is better.
AveragemsThe arithmetic mean response time of all executed requests.
MinmsThe fastest single response time recorded.
MaxmsThe slowest single response time recorded.
P50ms50th Percentile (Median): 50% of the requests completed in this time or faster.
P90ms90th Percentile: 90% of the requests completed in this time or faster. Excellent indicator of typical user experience.
P99ms99th Percentile: 99% of the requests completed in this time or faster. Crucial for identifying edge-case tail latencies.

Important Differences from rumour run​

warning

The bench command is designed for isolated stateless performance evaluation. As a result:

  • No Variable Propagation: Unlike rumour run, variables extracted in a benchmark run are not written to the global environment or propagated to subsequent runs. Each request starts with a clean slate.
  • Parallel Isolation: Requests are fired concurrently without regard to dependencies. Do not benchmark a downstream node that relies on a dynamically generated variable from an upstream node (like {{token}}) unless you provide that variable statically using the -V override.
  • Quiet Verbosity: Individual request responses and assertions are not logged. If an endpoint returns an error code, it will still count as an execution but won't print failure logs to the console unless the entire benchmark run fails.