Scenario

Scenarios are modules used by the plan to figure out which types of tests it build.

They contain several test signatures, each of which can be individually skipped in-case you only need certain parts of the scenario.

Scenario types

The following scenarios are available, everything except Baseline is optional

Baseline	The basic success stories of your API (required)
Validation	Challenges the request and response definitions
Client failures	Challenges the expected failure paths of the API

See the configuration guide to see how to choose scenarios.

Stability and roadmap

Scenarios are versioned so that we can offer a balance between stable tests, and new features. See the roadmap for information a scenario's stability.

Each suite has the ability to use the "experimental", "stable" or persistent scenario versions.

Build
Experimental	Scenario will use a potentially unstable experimental release
Stable	Tests will always use latest stable test suite
(version)	Tests are locked into a specific stable test suite

To be regarded as stable, a scenario must meet the following requirements (to the best of our knowledge): - Tests do not fail if the specification has been correctly defined - At least 100,000 tests have been performed without a reported incident

Once a scenario is stable, the tests will be locked in. If new issues are discovered, or features are added, they will be tested on the experimental branch, and eventually tagged as a new stable version.

We cannot guarantee that scenarios on an experimental branch will incorrectly fail, but you may appreciate the ability to try it out for yourself.

If a stable version contains an issue, the issue will not be fixed in a future version.