Performance Validation
Welcome to the Performance Validation documentation!
Rumour provides detailed performance profiling to help you understand and optimize API response times, as well as the ability to enforce strict Service Level Agreements (SLAs) through the duration assertion.
Performance SLA Assertion (duration)
To enforce a maximum acceptable response time for an endpoint, use the duration field inside the [assert] block. The value is specified in milliseconds. If the total request time (including DNS lookup, TLS handshake, and content download) exceeds this limit, the test fails.
Example: Passing SLA
Let's assert that fetching a user must take less than 5000 milliseconds.
duration_demo.toml
name = "duration_demo"
[request]
method = "GET"
url = "{{base_url}}/users/1"
[assert]
status = 200
duration = 5000
Output (Success)
When the server responds within the time limit (e.g., 4ms), the assertion passes:
rumour run ./duration_demo.toml -v
GET http://localhost:4000/api/v2/users/1
URL: http://localhost:4000/api/v2/users/1
✓ SUCCESS: /home/bugsfounder/workspace/testing/assertion_examples/duration_demo.toml (2ms)
✓ ./duration_demo.toml → PASS (1 Pass, 0 Fail, 0 Skip)
╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 1 │
│ Successful: 1 │
│ Failed: 0 │
│ Skipped: 0 │
│ Success Rate: 100.0% │
│ Total Time: 4ms │
╰──────────────────────────────────────────────────────────────────────────╯
✓ Successful Requests:
- /home/bugsfounder/workspace/testing/assertion_examples/duration_demo.toml [200] [4ms]
Let's break down what just happened:
- The Race Against the Clock: Rumour fired off the request and started a stopwatch. Because the entire network lifecycle finished in just
4ms—well under your generous5000mslimit—Rumour happily flags it as a✓ SUCCESS. - The Final Report: The execution report neatly logs the total time spent executing the suite and marks the request as passing. Everything looks clean and fast!
Example: Failing SLA
If we artificially enforce a completely unrealistic SLA (e.g., 0 milliseconds), the test will fail and report the timing violation.
duration_demo_fail.toml
name = "duration_demo_fail"
[request]
method = "GET"
url = "{{base_url}}/users/1"
[assert]
status = 200
duration = 0
Output (Failure)
✗ FAILED: /home/bugsfounder/workspace/testing/assertion_examples/duration_demo_fail.toml (1ms) - Performance SLA: Duration 1ms exceeds limit of 0ms
✗ ./duration_demo_fail.toml → FAIL (0 Pass, 1 Fail, 0 Skip)
╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 1 │
│ Successful: 0 │
│ Failed: 1 │
│ Skipped: 0 │
│ Success Rate: 0.0% │
│ Total Time: 1509ms │
╰──────────────────────────────────────────────────────────────────────────╯
✗ Failed Requests:
- /home/bugsfounder/workspace/testing/assertion_examples/duration_demo_fail.toml: [200]
Reason: Performance SLA: Duration 1ms exceeds limit of 0ms
Let's break down what just happened:
- The SLA Breach: We intentionally set the SLA limit to an impossible
0ms. Even though the server was lightning fast (1ms), it technically missed our incredibly tight deadline! Rumour is strict, so it instantly marks this as an✗ FAILEDexecution. - The Post-Mortem: Check out the
✗ Failed Requestssection at the bottom. TheReason:perfectly spells out the math:Duration 1ms exceeds limit of 0ms. No guessing games needed!
Detailed Timing Profiling
When an SLA is breached or you simply want to optimize performance, you can use the --profile flag combined with -v (verbose) to break down the total execution time into processing and transfer segments.
Profiling Output Example
rumour run ./duration_demo.toml -v --profile
GET http://localhost:4000/api/v2/users/1
URL: http://localhost:4000/api/v2/users/1
✓ SUCCESS: /home/bugsfounder/workspace/testing/assertion_examples/perf_demo.toml (1ms)
✓ ./perf_demo.toml → PASS (1 Pass, 0 Fail, 0 Skip)
╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 1 │
│ Successful: 1 │
│ Failed: 0 │
│ Skipped: 0 │
│ Success Rate: 100.0% │
│ Total Time: 2ms │
╰──────────────────────────────────────────────────────────────────────────╯
✓ Successful Requests:
- /home/bugsfounder/workspace/testing/assertion_examples/perf_demo.toml [200] [2ms] [Proc: 1ms, Xfer: 0ms]
Identifying Network Bottlenecks
Use the profiling tags ([Proc: 1ms, Xfer: 0ms]) appended to the end of each verbose line to pinpoint latency sources:
- Proc (Processing Time): The time the server took to process the request and generate the response (Time to First Byte).
- Xfer (Transfer Time): The time it took to download the response payload from the server over the network.
If Proc is consistently high, the backend database or server logic needs optimization.
If Xfer is consistently high, the payload is likely too large. Consider enabling server compression (gzip, br) or implementing pagination.
Need deeper analysis? If you need extreme, sub-millisecond precision to diagnose exactly where latency is occurring at the socket layer (DNS vs TCP vs TLS), use the dedicated rumour profile command!
Best Practices
- Set realistic SLAs: Account for network jitter. A strict 50ms SLA on a cloud service might trigger false failures.
- Benchmark locally and remotely: Your TTFB on
localhostwill be completely different from requests hitting an external staging server. - Combine with Benchmarking: For statistically significant timings, run performance tests with the
rumour benchcommand.