Step

Steps are either actions or assertions that are built from all relevant test signatures.

Every step is performed and can either pass or fail. Some tests will exit on failure, however some tests may provide multiple failures from several steps.

Failed steps will always provide a failure reason which can be used to debug any issues your API is facing.

Examples

Some assertion examples that can appear in a test results,

Assertion	---
(Can) see similar or improved response time	The test expects that the response time has not increased significantly between tests
(Can) see response code is client error	The test expects the API to respond with 4xx (client failure)
(Can) see response code is successful	The test expects the API to response with 2xx (successful)
(Can) See response code is documented	The test compares your API response's status code against documented status codes.
(Can) see content type is documented	The test compares your API response's content type against the document content types.
(Can) see payload is documented*	The test compares your API response's payload against the documented payloads. (Currently JSON only)