Result

A test result roughly corresponds to a single test against your API. Billing is based on these test results.

Some test signatures may result in more than one request against your API, which are represented as a single test.

Test results are generated in realtime when a report is running.

Result information

Each result contains information used to query test results. The information can be broken down into Request information, Response information, and Additional information...

Requests use HTTP/2 if the target server supports it.

Request information,

Server	The target server of the request	eg. https://api.apilicio.us/v1
Request Method	The HTTP verb used by the test	eg. GET, POST,...
Request Path	The relative path of the request	eg. /v1/suites
Operation	request method + path	eg. POST /v1/suites
Query templates	request method + URI template	eg. POST /v1/suites{?foo}{&bar}

Response information,

Response code	The response code of the test	eg. 200, 404,...
Response	The response payload and headers (No aggregation). Payloads exceeding 50kB are never stored	eg. { "health": "ok" }
Timings	See Timings

Additional information,

Failure reason	Failures/Warnings that applied to failed tests
Result status	See Status types
Report	See Report
Scenario	See Scenario
Signature	See Signature

Timings

Each report has timings as reported by cURL.

App Connect	The time it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed.
Connect	The time it took from the start until the TCP connect to the remote host was completed.
Name lookup	The time it took from the start until the name resolving was completed.
Pre-Transfer	The time it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
Redirect	The time it took for all redirection steps including name lookup, connect, pre-transfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections.
Start Transfer	The time it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
Total	The total time of the request

Additionally, speed is also stored.

Status types

Results can have one of the following statuses,

Success	All actions and assertions were performed successfully
Failed	Some actions or assertions failed
Skipped	The plan included a test that the test runner decided not to run in runtime
Error	Apilicious failed to perform test. Regard this as a bug with Apilicious, but most are caused by invalid schemas.