Skip to main content

Self-Healing (-H)

Rumour's soft self-healing mode automatically attempts to recover from transient failures. When a node fails, the healer analyzes the server's error response body and selects the best recovery strategy from its built-in playbook — all without any manual intervention.

Enabling Soft Healing

CLI Flag

rumour run ./requests/ -H
# or
rumour run ./requests/ --heal

Collection Config File

# api.config.toml
[config]
heal = true

Recovery Strategies

The healer selects one of four strategies automatically based on the failure type:

StrategyTriggered ByAction
Persona Switch401 Unauthorized, response contains unauthorized, credentials, token expired, forbiddenFinds a login/auth node in the graph, re-runs it, injects the fresh accessToken into runtime scope, retries
Smart Mutation400 Bad Request, response contains null, required, validation, invalid, or missingStrips all null JSON fields from the request body, retries the cleaned payload
State Promotion403 Forbidden, response contains state, status, approved, pending, transition, or workflowFinds a promoter node (approve_*.toml, submit_*.toml, publish_*.toml) in the graph, runs it, then retries
Simple RetryAny other transient error — network timeout, 503, unrecognized failuresRetries the request once unchanged

Healing Depth Limit

To prevent infinite loops, healing is limited to a maximum of 5 recursive attempts per node. If this depth is reached, the node is marked as failed and execution continues.

Strategy 1: Smart Mutation

Trigger: A request body contains null fields, causing a 400 Validation Error. The healer strips all null fields and retries.

File Layout

config_examples/08_heal_system/
├── workspace.env.toml
└── update_profile.toml

workspace.env.toml

base_url = "http://localhost:4000/api/v2"

update_profile.toml

name = "update_profile"

[request]
method = "POST"
url = "{{base_url}}/auth/profile"

[headers]
Content-Type = "application/json"

[body]
type = "json"
raw = '{"name":"Test User","age":null,"bio":null}'

[assert]
status = 200

The server rejects bodies with explicit null values. With -H active, the healer detects the validation error, strips the null fields from the body automatically, and retries with the cleaned payload.

Run

rumour run ./config_examples/08_heal_system/update_profile.toml -H -v

Output

POST http://localhost:4000/api/v2/auth/profile
Header: Content-Type: application/json
URL: http://localhost:4000/api/v2/auth/profile
Body (json): {"name":"Test User","age":null,"bio":null}
✗ FAILED: .../update_profile.toml (1ms) - HTTP Status: Expected status 200, got 400
↻ RETRYING: .../update_profile.toml (Attempt 1/2) [Wait 500ms]
POST http://localhost:4000/api/v2/auth/profile
Body (json): {"name":"Test User","age":null,"bio":null}
✗ FAILED: .../update_profile.toml (0ms) - HTTP Status: Expected status 200, got 400
↻ RETRYING: .../update_profile.toml (Attempt 2/2) [Wait 1000ms]
POST http://localhost:4000/api/v2/auth/profile
Body (json): {"name":"Test User","age":null,"bio":null}
✗ FAILED: .../update_profile.toml (1ms) - HTTP Status: Expected status 200, got 400
POST http://localhost:4000/api/v2/auth/profile
Body (json): {"name":"Test User"}
✓ SUCCESS: .../update_profile.toml (1ms)
.../update_profile.toml → HEALED (Smart Mutation)
✓ update_profile.toml → PASS (1 Pass, 0 Fail, 0 Skip)
╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 1
│ Successful: 1
│ Failed: 0
│ Skipped: 0
│ Success Rate: 100.0% │
│ Total Time: 1513ms │
│ Self-Healed: 1
╰──────────────────────────────────────────────────────────────────────────╯

🔧 Self-Healing Recovery Log:
→ Node: .../update_profile.toml
Action: Smart Mutation: Omitting null fields
Outcome: Success

✓ Successful Requests:
- update_profile.toml [200] [1513ms]

What happened: The original body {"name":"Test User","age":null,"bio":null} failed with 400. After exhausting the standard retries, the healer stripped the null fields producing {"name":"Test User"}, and the retry succeeded with 200.

note

Smart Mutation only removes fields whose value is the JSON literal null. It does not change field values, add fields, or modify non-null data.

Strategy 2: Persona Switch

Trigger: A node fails with authorization or privilege errors (such as 403 Forbidden or 401 Unauthorized responses indicating insufficient permissions or role mismatches, e.g. "Admin access required").

How it Works Under the Hood

When a request fails due to permission errors, the healer attempts to resolve the failure by switching to a more privileged identity:

  1. Verify Node Identity Needs: The healer checks if the failing node requires higher privileges (such as admin access) by analyzing whether the node_id (filename) or route contains "admin".
  2. Locate Target Login Node:
    • If the node requires admin permissions, it searches the workflow graph for a login/auth node containing "admin" (e.g. user_admin_login.toml).
    • Otherwise, it searches for a standard login or auth node.
  3. Bootstrap and Extract:
    • The healer executes the identified login node to obtain the elevated token.
    • It extracts the token from the response (looking first for accessToken, then fallback adminaccessToken).
  4. Context Switch: The healer overrides the accessToken variable in the active runtime context with the new token and immediately retries the original forbidden request.
important

This strategy executes dynamically at runtime when privilege limits are reached. In order to allow both user and admin login workflows to exist in the same workflow graph without pre-flight dependency resolution conflicts, we structure the admin login file to extract adminaccessToken and come alphabetically after login.toml to prevent shared state ordering conflicts.

File Layout

config_examples/08_heal_system/
├── workspace.env.toml
├── login.toml ← logs in a standard user, sets `accessToken`
├── get_admin_data.toml ← requests admin-only route, using `{{accessToken}}`
└── user_admin_login.toml ← logs in admin, extracts `adminaccessToken`

login.toml

name = "login"

[request]
method = "POST"
url = "{{base_url}}/auth/login"

[headers]
Content-Type = "application/json"

[body]
type = "json"
raw = '{"email":"user@test.com","password":"pass123"}'

[assert]
status = 200

[extract]
accessToken = "token"

get_admin_data.toml

name = "get_admin_data"

[request]
method = "GET"
url = "{{base_url}}/auth/admin-only"

[headers]
Authorization = "Bearer {{accessToken}}"

[assert]
status = 200

user_admin_login.toml

name = "user_admin_login"

[request]
method = "POST"
url = "{{base_url}}/auth/login"

[headers]
Content-Type = "application/json"

[body]
type = "json"
raw = '{"email":"admin@test.com","password":"adminpass123"}'

[assert]
status = 200

[extract]
adminaccessToken = "token"

persona_switch.suite.toml

# Persona Switch Suite Demo
[suite]
name = "Persona Switch Demo"
ordered = true

requests = [
"login.toml",
"get_admin_data.toml",
"user_admin_login.toml"
]

Run

rumour run ./config_examples/08_heal_system/persona_switch.suite.toml -H -v

Output

⠠ Executing: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/login.toml
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/login.toml (2ms)

⠠ Executing: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml ← [login]
GET http://localhost:4000/api/v2/auth/admin-only
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTU5OTMsImV4cCI6MTc4MDY1OTU5M30...
URL: http://localhost:4000/api/v2/auth/admin-only
✗ FAILED: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml (0ms) - HTTP Status: Expected status 200, got 403

↻ RETRYING: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml (Attempt 1/2) [Wait 500ms]

⠠ Executing: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml ← [login]
GET http://localhost:4000/api/v2/auth/admin-only
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6ImFkbWluQHRlc3QuY29tIiwicm9sZSI6ImFkbWluIiwiaWQiOjIsImlhdCI6MTc4MDY1NjE0NiwiZXhwIjoxNzgwNjU5NzQ2fQ...
URL: http://localhost:4000/api/v2/auth/admin-only
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml (0ms)
✨ /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml → HEALED (Persona Switch)

⠠ Executing: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/user_admin_login.toml ← [get_admin_data, login]
POST http://localhost:4000/api/v2/auth/login
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/user_admin_login.toml (0ms)

✓ /home/bugsfounder/workspace/testing/config_examples/08_heal_system/persona_switch.suite.toml → PASS (3 Pass, 0 Fail, 0 Skip)

╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 3
│ Successful: 3
│ Failed: 0
│ Skipped: 0
│ Success Rate: 100.0% │
│ Total Time: 1513ms │
│ Self-Healed: 1
╰──────────────────────────────────────────────────────────────────────────╯

🔧 Self-Healing Recovery Log:
→ Node: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml
Action: Identity Bootstrap: Running login node /home/bugsfounder/workspace/testing/config_examples/08_heal_system/user_admin_login.toml
Outcome: Attempting login...
→ Node: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_admin_data.toml
Action: Persona Switch: Switching to Bootstrapped context
Outcome: Success

What happened: get_admin_data.toml initially failed with a 403 Forbidden because it was executed under the standard user's accessToken established by login.toml. Recognizing the failure was administrative-related (due to route or node names matching "admin"), the healer identified the administrative credentials login file (user_admin_login.toml), executed it, extracted the adminaccessToken, and promoted it to the active accessToken variable context, letting the original request retry successfully.

Strategy 3: State Promotion

Trigger: A request fails (typically with 403 Forbidden or a similar lifecycle error), and the server response body contains state-transition keywords.

How it Works Under the Hood

When a request fails, the healer follows a precise 4-step sequence to locate and run a "promoter" request:

  1. Scan Response for Keywords: The healer inspects the response body to classify the required transition action:
    • Contains "approved", "approv", or "accept" $\rightarrow$ Action type: "approve"
    • Contains "pending", "submit", or "review" $\rightarrow$ Action type: "submit"
    • Contains "published", "public", or "live" $\rightarrow$ Action type: "publish"
  2. Identify Resource Name: The healer parses the failing node's URL to identify the resource it operates on (e.g., in /auth/premium-features, the resource is premium-features).
  3. Locate Promoter Node: The healer scans the current loaded workflow graph for a request node that:
    • Operates on the same resource (uses premium-features in its path).
    • Uses the POST method (since status promotions are mutative actions).
    • Has a filename containing the target action keyword (e.g., approve, submit, or publish).
  4. Execute Promotion: The healer pauses the failing node, executes the promoter node first, and if that succeeds, retries the original failing request.

Example Scenario

A client attempts to call GET /api/v2/auth/premium-features to fetch premium details. However, their account is still in a pending state, so the server rejects the request with a 403 Forbidden status and the following JSON:

{
"error": "State Conflict",
"message": "Account for user@test.com is pending. It must be approved before accessing premium features."
}

Since the response message contains approved, the healer determines the action type is "approve". It searches the graph for a POST request to premium-features whose filename contains approve (which matches approve_account.toml). It runs approve_account.toml to transition the state to approved, then successfully retries the original GET request.

File Layout

config_examples/08_heal_system/
├── workspace.env.toml
├── login.toml ← Authenticates and sets {{accessToken}}
├── approve_account.toml ← The promoter node (POST to transition state)
├── get_premium.toml ← The failing node (GET, requires approved state)
└── state_promotion.suite.toml ← Suite running get_premium.toml

approve_account.toml

name = "approve_account"

[request]
method = "POST"
url = "{{base_url}}/auth/premium-features"

[headers]
Authorization = "Bearer {{accessToken}}"

[assert]
status = 200

get_premium.toml

name = "get_premium"

[request]
method = "GET"
url = "{{base_url}}/auth/premium-features"

[headers]
Authorization = "Bearer {{accessToken}}"

[assert]
status = 200

state_promotion.suite.toml

[suite]
name = "State Promotion Demo"
ordered = true

requests = [
"login.toml",
"get_premium.toml",
"approve_account.toml"
]

Run

rumour run ./config_examples/08_heal_system/state_promotion.suite.toml -H -v

Output

POST http://localhost:4000/api/v2/auth/login
Header: Content-Type: application/json
URL: http://localhost:4000/api/v2/auth/login
Body (json): {"email":"user@test.com","password":"pass123"}
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/login.toml (26ms)
GET http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✗ FAILED: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (3ms) - HTTP Status: Expected status 200, got 403
↻ RETRYING: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (Attempt 1/2) [Wait 500ms]
GET http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✗ FAILED: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (2ms) - HTTP Status: Expected status 200, got 403
↻ RETRYING: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (Attempt 2/2) [Wait 1000ms]
GET http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✗ FAILED: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (2ms) - HTTP Status: Expected status 200, got 403
POST http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/approve_account.toml (1ms)
GET http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml (1ms)
✨ /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml → HEALED (State Promotion)
POST http://localhost:4000/api/v2/auth/premium-features
Header: Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InVzZXJAdGVzdC5jb20iLCJyb2xlIjoidXNlciIsImlkIjoxLCJpYXQiOjE3ODA2NTY3ODEsImV4cCI6MTc4MDY2MDM4MX0.SahmhyHcAqcQxZbNJq1wVC3VJ16rUfBVInFRsDEU4Nk
URL: http://localhost:4000/api/v2/auth/premium-features
✓ SUCCESS: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/approve_account.toml (1ms)
✓ ./config_examples/08_heal_system/state_promotion.suite.toml → PASS (3 Pass, 0 Fail, 0 Skip)
╭──────────────────────────────────────────────────────────────────────────╮
│ RUMOUR EXECUTION REPORT │
├──────────────────────────────────────────────────────────────────────────┤
│ Total Requests: 3
│ Successful: 3
│ Failed: 0
│ Skipped: 0
│ Success Rate: 100.0% │
│ Total Time: 1555ms │
│ Self-Healed: 1
╰──────────────────────────────────────────────────────────────────────────╯

🔧 Self-Healing Recovery Log:
→ Node: /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml
Action: State Promotion: Running approve (/home/bugsfounder/workspace/testing/config_examples/08_heal_system/approve_account.toml)
Outcome: Success

✓ Successful Requests:
- /home/bugsfounder/workspace/testing/config_examples/08_heal_system/login.toml [200] [28ms]
- /home/bugsfounder/workspace/testing/config_examples/08_heal_system/get_premium.toml [200] [1523ms]
- /home/bugsfounder/workspace/testing/config_examples/08_heal_system/approve_account.toml [200] [3ms]

Actionable Recommendations:
→ Run with --json to export this report for your CI/CD pipeline.
note

Naming and Location Rules:

  • The promoter file name must contain approve, submit, or publish depending on the action keyword.
  • The promoter node must be loaded into the active workflow graph (either by being part of the same directory being run, or explicitly listed in the suite).

Strategy 4: Simple Retry

Trigger: Any failure that does not match the other three strategies — network timeouts, 503 Service Unavailable, connection resets, or any error the healer cannot classify. The request is re-executed once, unchanged.

This is the catch-all fallback. No node lookups, token refreshes, or payload mutations occur.

✗ FAILED: .../get_post.toml (0ms) - Connection refused
↻ RETRYING: .../get_post.toml (Attempt 1/2) [Wait 500ms]
GET http://localhost:4000/api/v2/posts/1
✓ SUCCESS: .../get_post.toml (2ms)

Interactive Healing Mode (-i)

When combined with -i, you manually choose the recovery strategy on each failure instead of letting the healer decide automatically:

rumour run ./requests/ -H -i

On failure, the CLI presents a menu:

⚠ Failure in node: get_premium.toml
Error: HTTP Status: Expected status 200, got 403

How would you like to proceed?
[1] Persona Switch (Try another user identity)
[2] Resource Reconstruction (Try creating missing dependencies)
[3] Smart Mutation (Try auto-fixing payload issues)
[4] Retry (Execute this node again)
[5] Skip (Mark as failed and continue workflow)
[6] Abort (Stop entire workflow immediately)

Select an option [1-6]:
note

Interactive mode forces sequential execution — it cannot be combined with -p (parallel).

Interaction with Auto-Cleanup (-C)

When combining Soft Self-Healing (-H) with Automated Resource Cleanup (-C), keep the following behaviors in mind:

  1. Successful Creations Are Captured: If a resource-creating request (e.g. a POST) fails but is recovered and successfully retried via Soft Healing, the created resource ID is captured and registered for cleanup.
  2. Healing Inactive During Cleanup: The auto-cleanup phase executes after the main workflow completes. Healing strategies are disabled during the cleanup phase. If a cleanup DELETE request fails (e.g. token expired during cleanup), it will be logged in the recovery report but no healing or retries will be attempted.

Limitations

  • Cannot heal logic errors — wrong data, incorrect assertions, or invalid TOML.
  • Cannot heal business rule violations — the server must accept the request after the mutation.
  • Persona Switch requires a login/auth node in the same workflow graph, and the failing node must use {{accessToken}} (not a hardcoded token string) in its header.
  • State Promotion requires a promoter node named with approve, submit, or publish in the same graph.
  • Smart Mutation only strips JSON null fields — it does not change field values or restructure the payload.
  • Healing increases execution time proportionally to the length of the recovery chain.

Combining with Other Flags

# Heal + verbose (see full recovery log in output)
rumour run ./tests/ -H -v

# Heal + resume
rumour run ./tests/ -H -r

# Heal + hard heal (both soft and hard strategies active simultaneously)
rumour run ./tests/ -H -X

# Heal + parallel (healing applies per-node inside worker threads)
rumour run ./tests/ -H -p