Skip to content

Commit

Permalink
Implement warning-level validation test failures
Browse files Browse the repository at this point in the history
Sometimes we don't want to fail a battery of validation tests just
because a device implementation misses a small detail (example: cost
of OSPF default route). You can use 'level: warning' on such tests,
indicating that if the test fails it's not catastrophic, but neither
is it good.
  • Loading branch information
ipspace committed Sep 3, 2024
1 parent 48eb36b commit d5596af
Show file tree
Hide file tree
Showing 5 changed files with 103 additions and 70 deletions.
9 changes: 9 additions & 0 deletions docs/netlab/validate.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,15 @@ options:
-e, --error-only Display only validation errors (on stderr)
```

The **netlab validate** command returns the overall test results in its exit code:

| Exit code | Meaning |
|----------:|---------|
| 0 | All tests passed |
| 1 | At least one test failed |
| 2 | `netlab validate` did not find a single usable test to execute |
| 3 | Some of the tests generated warnings |

## Example

**netlab validate** command was executed on the [Advertise IPv4 Prefixes to BGP Neighbors](https://bgplabs.net/basic/3-originate/) lab before the user configured BGP prefix origination:
Expand Down
1 change: 1 addition & 0 deletions docs/topology/validate.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Each test has a name (dictionary key) and description (dictionary value) -- anot
* **wait** (integer, optional) -- Time to wait (when specified as the only action in the test) or retry (when used together with other actions). The first wait/retry timeout is measured from when the lab was started; subsequent times are measured from the previous test containing the **wait** parameter.
* **wait_str** (string, optional) -- Message to print before starting the wait.
* **stop_on_error** (bool, optional) -- When set to `True`, the validation tests stop if the current test fails on at least one of the devices.
* **level** (string, optional) -- When set to `warning,` the test failure does not indicate that the whole testing sequence has failed but generates a warning message.

[^TEX]: Objects, non-empty strings, lists, or dictionaries, integers not equal to zero or `True`.

Expand Down
Loading

0 comments on commit d5596af

Please sign in to comment.