Automating suites
What you'll learn
- How to automate test reports
- How to setup and handle notifications generated by each test report
Several options are available to fit test suites in your automated workflow,
| Github's webhooks | Ingress | Configure any Github webhook to start a test report |
| Scheduling | Ingress | Start reports hourly, daily, weekly, monthly |
| Test completion webhooks | Egress | When a report has completed, POSTs a report object to a URL |
| Email notifications | Egress | Configurable email notifications sent to users |
| Slack webhooks | Egress | When a report completes receive a message using an incoming webhook |
| API Integration | Ingress+Egress | Use our RESTful API to completely customise your integration |
| JUnit XML | Egress | Suites are available in JUnit XML, a popular pipeline software format |
Github's webhooks
GitHub webhooks can trigger test reports using any event in your deployment process, including push events, deployment events and pull requests.
To enable a webhook, the owner must connect to their Github account then choose the Github repository in the suite configuration.
From the project's Github directory, go to Settings > Webhooks > Add webhook, set the Payload URL to "https://api.apilicious.com/incoming/github", and choosing which events you would like to start each report.
The secret is optional, suites that share the same repository and secret will all be triggered.
How to configure a git webhook to start a test report at every push event
Scheduling
A schedule will start a test report at a regular interval.
The intervals available are hourly, daily, weekly, monthly, and every ten minutes (always on).
A test schedule will affect the estimation on the billing page, which can be used to optimise your billing package
Setting a schedule will ensure tests are run regularly, which is useful for monitoring
Test completion webhooks
A webhook will send a POST request to a public URL when a report has completed.
The payload contains a Report object.
POST https://example.com/my-webhook
Signature: SIGNATURE
{
"event": "report.completed",
"report": { /* Report Object - See API Documentation */ }
}
The signature is a SHA256 encoded value of the payload, using the secret as a key. Use it to verify Apilicious sent the webhook.
The webhook will retry up to three times if the response fails.
Email notifications
Each account can receive an email for each report, or weekly/monthly summary emails. Notifications can be configured from the account page
| Each report | An email is sent after each report is completed |
| Weekly summary | A summary email is sent every Sunday |
| Monthly summary | A summary email is sent on the first day of every month |
An email notification can be sent for each report
Slack incoming webhooks
Post a summary message to an incoming webhook every time a report is completed. Read Slack's webhook guide to learn how to set one up.
A short message is sent to the configured Slack channel
API integration
Build Apilicious into highly specialised workflows by simply communicating with the API yourself.
Some useful API endpoints,
| POST /suites | Create a new suite |
| POST /suites/{suite}/clone | Clone an existing suite |
| GET /suites/{suite} | Retrieve information about a single suite |
| PATCH /suites/{suite} | Update a suite's configuration |
| GET /suites/{suite}/start | Starts a suite, generates a report |
| GET /suites/{suite}/junit | Retrieve JUnit formatted results for suite |
| GET /suites/{suite}/results | Retrieve JSON formatted results for suite |
| GET /reports/{report} | Retrieve information about a specific report |
Authorizing requests
For API requests, you will need to setup an oAuth client with client credentials, all users can do this. The oAuth client always acts as the user who created it.
When you first create an oAuth client, you will be given a secret (once). Remember to copy it into your application's configuration
Some accounts can create a client that allows an Auth Code grant flow so that several users can securely use the same oAuth client.
JUnit
JUnit XML Format is a well-established test report format.
Apilicious does not run on JUnit under-the-hood, but you can use the API to retrieve a JUnit XML-formatted document and integrated into your pipeline.
Tools such as jenkins can integrate the results within your existing pipeline.
