Self-Healing (-H)
Rumour's soft self-healing mode automatically attempts to recover from transient failures. When a node fails, the healer analyzes the server's error response body and selects the best recovery strategy from its built-in playbook — all without any manual intervention.
Enabling Soft Healing
CLI Flag
rumour run ./requests/ -H
# or
rumour run ./requests/ --heal
Collection Config File
# api.config.toml
[config]
heal = true
Recovery Strategies
The healer selects one of four strategies automatically based on the failure type:
| Strategy | Triggered By | Action |
|---|---|---|
| Persona Switch | 401 Unauthorized, response contains unauthorized, credentials, token expired, forbidden | Finds a login/auth node in the graph, re-runs it, injects the fresh accessToken into runtime scope, retries |
| Smart Mutation | 400 Bad Request, response contains null, required, validation, invalid, or missing | Strips all null JSON fields from the request body, retries the cleaned payload |
| State Promotion | 403 Forbidden, response contains state, status, approved, pending, transition, or workflow | Finds a promoter node (approve_*.toml, submit_*.toml, publish_*.toml) in the graph, runs it, then retries |
| Simple Retry | Any other transient error — network timeout, 503, unrecognized failures | Retries the request once unchanged |
Healing Depth Limit
To prevent infinite loops, healing is limited to a maximum of 5 recursive attempts per node. If this depth is reached, the node is marked as failed and execution continues.
Strategy 1: Smart Mutation
Trigger: A request body contains null fields, causing a 400 Validation Error. The healer strips all null fields and retries.
File Layout
config_examples/08_heal_system/
├── workspace.env.toml
└── update_profile.toml
workspace.env.toml
base_url = "http://localhost:4000/api/v2"
update_profile.toml
name = "update_profile"
[request]
method = "POST"
url = "{{base_url}}/auth/profile"
[headers]
Content-Type = "application/json"
[body]
type = "json"
raw = '{"name":"Test User","age":null,"bio":null}'
[assert]
status = 200
The server rejects bodies with explicit null values. With -H active, the healer detects the validation error, strips the null fields from the body automatically, and retries with the cleaned payload.
Run
rumour run ./config_examples/08_heal_system/update_profile.toml -H -v
Output
POST http://localhost:4000/api/v2/auth/profile
Header: Content-Type: application/json
URL: http://localhost:4000/api/v2/auth/profile
Body (json): {"name":"Test User","age":null,"bio":null}
✗ FAILED: .../update_profile.toml (1ms) - HTTP Status: Expected status 200, got 400
↻ RETRYING: .../update_profile.toml (Attempt 1/2) [Wait 500ms]
POST http://localhost:4000/api/v2/auth/profile
Body (json): {"name":"Test User","age":null,"bio":null}
✗ FAILED: .../update_profile.toml (0ms) - HTTP Status: Expected status 200, got 400
↻ RETRYING: .../update_profile.toml (Attempt 2/2) [Wait 1000ms]
POST http://localhost:4000/api/v2/auth/profile
Body (json): {"name":"Test User","age":null,"bio":null}
✗ FAILED: .../update_profile.toml (1ms) - HTTP Status: Expected status 200, got 400
POST http://localhost:4000/api/v2/auth/profile
Body (json): {"name":"Test User"}
✓ SUCCESS: .../update_profile.toml (1ms)
✨ .../update_profile.toml → HEALED (Smart Mutation)
✓ update_profile.toml → PASS (1 Pass, 0 Fail, 0 Skip)
╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 1 │
│ Successful: 1 │
│ Failed: 0 │
│ Skipped: 0 │
│ Success Rate: 100.0% │
│ Total Time: 1513ms │
│ Self-Healed: 1 │
╰──────────────────────────────────────────────────────────────────────────╯
🔧 Self-Healing Recovery Log:
→ Node: .../update_profile.toml
Action: Smart Mutation: Omitting null fields
Outcome: Success
✓ Successful Requests:
- update_profile.toml [200] [1513ms]
What happened: The original body {"name":"Test User","age":null,"bio":null} failed with 400. After exhausting the standard retries, the healer stripped the null fields producing {"name":"Test User"}, and the retry succeeded with 200.
Smart Mutation only removes fields whose value is the JSON literal null. It does not change field values, add fields, or modify non-null data.
Strategy 2: Persona Switch
Trigger: A node fails with authorization or privilege errors (such as 403 Forbidden or 401 Unauthorized responses indicating insufficient permissions or role mismatches, e.g. "Admin access required").
How it Works Under the Hood
When a request fails due to permission errors, the healer attempts to resolve the failure by switching to a more privileged identity:
- Verify Node Identity Needs: The healer checks if the failing node requires higher privileges (such as admin access) by analyzing whether the
node_id(filename) or route contains"admin". - Locate Target Login Node:
- If the node requires admin permissions, it searches the workflow graph for a login/auth node containing
"admin"(e.g.user_admin_login.toml). - Otherwise, it searches for a standard
loginorauthnode.
- If the node requires admin permissions, it searches the workflow graph for a login/auth node containing
- Bootstrap and Extract:
- The healer executes the identified login node to obtain the elevated token.
- It extracts the token from the response (looking first for
accessToken, then fallbackadminaccessToken).
- Context Switch: The healer overrides the
accessTokenvariable in the active runtime context with the new token and immediately retries the original forbidden request.
This strategy executes dynamically at runtime when privilege limits are reached. In order to allow both user and admin login workflows to exist in the same workflow graph without pre-flight dependency resolution conflicts, we structure the admin login file to extract adminaccessToken and come alphabetically after login.toml to prevent shared state ordering conflicts.
File Layout
config_examples/08_heal_system/
├── workspace.env.toml
├── login.toml ← logs in a standard user, sets `accessToken`
├── get_admin_data.toml ← requests admin-only route, using `{{accessToken}}`
└── user_admin_login.toml ← logs in admin, extracts `adminaccessToken`
login.toml
name = "login"
[request]
method = "POST"
url = "{{base_url}}/auth/login"
[headers]
Content-Type = "application/json"
[body]
type = "json"
raw = '{"email":"user@test.com","password":"pass123"}'
[assert]
status = 200
[extract]
accessToken = "token"
get_admin_data.toml
name = "get_admin_data"
[request]
method = "GET"
url = "{{base_url}}/auth/admin-only"
[headers]
Authorization = "Bearer {{accessToken}}"
[assert]
status = 200
user_admin_login.toml
name = "user_admin_login"
[request]
method = "POST"
url = "{{base_url}}/auth/login"
[headers]
Content-Type = "application/json"
[body]
type = "json"
raw = '{"email":"admin@test.com","password":"adminpass123"}'
[assert]
status = 200
[extract]
adminaccessToken = "token"
persona_switch.suite.toml
# Persona Switch Suite Demo
[suite]
name = "Persona Switch Demo"
ordered = true
requests = [
"login.toml",
"get_admin_data.toml",
"user_admin_login.toml"
]
Run
rumour run ./config_examples/08_heal_system/persona_switch.suite.toml -H -v
Output
⠠ Executing: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/login.toml
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/login.toml (2ms)
⠠ Executing: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml ← [login]
GET http://localhost:4000/api/v2/auth/admin-only
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTU5OTMsImV4cCI6MTc4MDY1OTU5M30...
URL: http://localhost:4000/api/v2/auth/admin-only
✗ FAILED: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml (0ms) - HTTP Status: Expected status 200, got 403
↻ RETRYING: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml (Attempt 1/2) [Wait 500ms]
⠠ Executing: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml ← [login]
GET http://localhost:4000/api/v2/auth/admin-only
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImFkbWluQHRlc3QuY29tIiwicm9sZSI6ImFkbWluIiwiaWQiOjIsImlhdCI6MTc4MDY1NjE0NiwiZXhwIjoxNzgwNjU5NzQ2fQ...
URL: http://localhost:4000/api/v2/auth/admin-only
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml (0ms)
✨ /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml → HEALED (Persona Switch)
⠠ Executing: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/user_admin_login.toml ← [get_admin_data, login]
POST http://localhost:4000/api/v2/auth/login
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/user_admin_login.toml (0ms)
✓ /home/bugsfounder/workspace/testing/config_examples/08_heal_system/persona_switch.suite.toml → PASS (3 Pass, 0 Fail, 0 Skip)
╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 3 │
│ Successful: 3 │
│ Failed: 0 │
│ Skipped: 0 │
│ Success Rate: 100.0% │
│ Total Time: 1513ms │
│ Self-Healed: 1 │
╰──────────────────────────────────────────────────────────────────────────╯
🔧 Self-Healing Recovery Log:
→ Node: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml
Action: Identity Bootstrap: Running login node /home/bugsfounder/workspace/testing/config_examples/08_heal_system/user_admin_login.toml
Outcome: Attempting login...
→ Node: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml
Action: Persona Switch: Switching to Bootstrapped context
Outcome: Success
What happened: get_admin_data.toml initially failed with a 403 Forbidden because it was executed under the standard user's accessToken established by login.toml. Recognizing the failure was administrative-related (due to route or node names matching "admin"), the healer identified the administrative credentials login file (user_admin_login.toml), executed it, extracted the adminaccessToken, and promoted it to the active accessToken variable context, letting the original request retry successfully.
Strategy 3: State Promotion
Trigger: A request fails (typically with 403 Forbidden or a similar lifecycle error), and the server response body contains state-transition keywords.
How it Works Under the Hood
When a request fails, the healer follows a precise 4-step sequence to locate and run a "promoter" request:
- Scan Response for Keywords: The healer inspects the response body to classify the required transition action:
- Contains
"approved","approv", or"accept"$\rightarrow$ Action type:"approve" - Contains
"pending","submit", or"review"$\rightarrow$ Action type:"submit" - Contains
"published","public", or"live"$\rightarrow$ Action type:"publish"
- Contains
- Identify Resource Name: The healer parses the failing node's URL to identify the resource it operates on (e.g., in
/auth/premium-features, the resource ispremium-features). - Locate Promoter Node: The healer scans the current loaded workflow graph for a request node that:
- Operates on the same resource (uses
premium-featuresin its path). - Uses the
POSTmethod (since status promotions are mutative actions). - Has a filename containing the target action keyword (e.g.,
approve,submit, orpublish).
- Operates on the same resource (uses
- Execute Promotion: The healer pauses the failing node, executes the promoter node first, and if that succeeds, retries the original failing request.
Example Scenario
A client attempts to call GET /api/v2/auth/premium-features to fetch premium details. However, their account is still in a pending state, so the server rejects the request with a 403 Forbidden status and the following JSON:
{
"error": "State Conflict",
"message": "Account for user@test.com is pending. It must be approved before accessing premium features."
}
Since the response message contains approved, the healer determines the action type is "approve". It searches the graph for a POST request to premium-features whose filename contains approve (which matches approve_account.toml). It runs approve_account.toml to transition the state to approved, then successfully retries the original GET request.
File Layout
config_examples/08_heal_system/
├── workspace.env.toml
├── login.toml ← Authenticates and sets {{accessToken}}
├── approve_account.toml ← The promoter node (POST to transition state)
├── get_premium.toml ← The failing node (GET, requires approved state)
└── state_promotion.suite.toml ← Suite running get_premium.toml
approve_account.toml
name = "approve_account"
[request]
method = "POST"
url = "{{base_url}}/auth/premium-features"
[headers]
Authorization = "Bearer {{accessToken}}"
[assert]
status = 200
get_premium.toml
name = "get_premium"
[request]
method = "GET"
url = "{{base_url}}/auth/premium-features"
[headers]
Authorization = "Bearer {{accessToken}}"
[assert]
status = 200
state_promotion.suite.toml
[suite]
name = "State Promotion Demo"
ordered = true
requests = [
"login.toml",
"get_premium.toml",
"approve_account.toml"
]
Run
rumour run ./config_examples/08_heal_system/state_promotion.suite.toml -H -v
Output
POST http://localhost:4000/api/v2/auth/login
Header: Content-Type: application/json
URL: http://localhost:4000/api/v2/auth/login
Body (json): {"email":"user@test.com","password":"pass123"}
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/login.toml (26ms)
GET http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✗ FAILED: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (3ms) - HTTP Status: Expected status 200, got 403
↻ RETRYING: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (Attempt 1/2) [Wait 500ms]
GET http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✗ FAILED: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (2ms) - HTTP Status: Expected status 200, got 403
↻ RETRYING: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (Attempt 2/2) [Wait 1000ms]
GET http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✗ FAILED: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (2ms) - HTTP Status: Expected status 200, got 403
POST http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/approve_account.toml (1ms)
GET http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (1ms)
✨ /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml → HEALED (State Promotion)
POST http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/approve_account.toml (1ms)
✓ ./config_examples/08_heal_system/state_promotion.suite.toml → PASS (3 Pass, 0 Fail, 0 Skip)
╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 3 │
│ Successful: 3 │
│ Failed: 0 │
│ Skipped: 0 │
│ Success Rate: 100.0% │
│ Total Time: 1555ms │
│ Self-Healed: 1 │
╰──────────────────────────────────────────────────────────────────────────╯
🔧 Self-Healing Recovery Log:
→ Node: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml
Action: State Promotion: Running approve (/home/bugsfounder/workspace/testing/config_examples/08_heal_system/approve_account.toml)
Outcome: Success
✓ Successful Requests:
- /home/bugsfounder/workspace/testing/config_examples/08_heal_system/login.toml [200] [28ms]
- /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml [200] [1523ms]
- /home/bugsfounder/workspace/testing/config_examples/08_heal_system/approve_account.toml [200] [3ms]
Actionable Recommendations:
→ Run with --json to export this report for your CI/CD pipeline.
Naming and Location Rules:
- The promoter file name must contain
approve,submit, orpublishdepending on the action keyword. - The promoter node must be loaded into the active workflow graph (either by being part of the same directory being run, or explicitly listed in the suite).
Strategy 4: Simple Retry
Trigger: Any failure that does not match the other three strategies — network timeouts, 503 Service Unavailable, connection resets, or any error the healer cannot classify. The request is re-executed once, unchanged.
This is the catch-all fallback. No node lookups, token refreshes, or payload mutations occur.
✗ FAILED: .../get_post.toml (0ms) - Connection refused
↻ RETRYING: .../get_post.toml (Attempt 1/2) [Wait 500ms]
GET http://localhost:4000/api/v2/posts/1
✓ SUCCESS: .../get_post.toml (2ms)
Interactive Healing Mode (-i)
When combined with -i, you manually choose the recovery strategy on each failure instead of letting the healer decide automatically:
rumour run ./requests/ -H -i
On failure, the CLI presents a menu:
⚠ Failure in node: get_premium.toml
Error: HTTP Status: Expected status 200, got 403
How would you like to proceed?
[1] Persona Switch (Try another user identity)
[2] Resource Reconstruction (Try creating missing dependencies)
[3] Smart Mutation (Try auto-fixing payload issues)
[4] Retry (Execute this node again)
[5] Skip (Mark as failed and continue workflow)
[6] Abort (Stop entire workflow immediately)
Select an option [1-6]:
Interactive mode forces sequential execution — it cannot be combined with -p (parallel).
Interaction with Auto-Cleanup (-C)
When combining Soft Self-Healing (-H) with Automated Resource Cleanup (-C), keep the following behaviors in mind:
- Successful Creations Are Captured: If a resource-creating request (e.g. a
POST) fails but is recovered and successfully retried via Soft Healing, the created resource ID is captured and registered for cleanup. - Healing Inactive During Cleanup: The auto-cleanup phase executes after the main workflow completes. Healing strategies are disabled during the cleanup phase. If a cleanup
DELETErequest fails (e.g. token expired during cleanup), it will be logged in the recovery report but no healing or retries will be attempted.
Limitations
- Cannot heal logic errors — wrong data, incorrect assertions, or invalid TOML.
- Cannot heal business rule violations — the server must accept the request after the mutation.
- Persona Switch requires a
login/authnode in the same workflow graph, and the failing node must use{{accessToken}}(not a hardcoded token string) in its header. - State Promotion requires a promoter node named with
approve,submit, orpublishin the same graph. - Smart Mutation only strips JSON
nullfields — it does not change field values or restructure the payload. - Healing increases execution time proportionally to the length of the recovery chain.
Combining with Other Flags
# Heal + verbose (see full recovery log in output)
rumour run ./tests/ -H -v
# Heal + resume
rumour run ./tests/ -H -r
# Heal + hard heal (both soft and hard strategies active simultaneously)
rumour run ./tests/ -H -X
# Heal + parallel (healing applies per-node inside worker threads)
rumour run ./tests/ -H -p