Storing API Test Results to a Database with Eventn and Runscope

A common requirement for Runscope users is to save and analyze test results for alerting, building custom dashboards, and other analytical purposes. One way to do that is to pipe all your test results to an Amazon S3 bucket. But what if you want more flexibility in how you store the test results? Or if you want to filter and process the test result data before storing it? 

Eventn is an HTTP microservices platform specifically designed for data processing and analytical workloads. You can use it to launch microservices, and each one you create provides a Node.js JavaScript function runtime that executes when an HTTP request is made.

In this example, we will demonstrate how to call an Eventn microservice after each test completes and provide an example of saving the results to a database. We will also report on the data using SQL.

Eventn Overview

Each Eventn microservice provides two JavaScript functions: an HTTP POST function for data submission and an HTTP GET function for data reporting and analytics. These execute when triggered by their respective HTTP methods. The platform currently supports Node.js V7.6.0 which allows for all ES6 capabilities as well as newer features such as async/await and also supports a set of whitelisted npm modules.

It also provides database connectivity from each microservice, which allows users to easily save test results to a database of their choosing (support for Postgres, MySQL, or SQL Server at the time of writing), or alternatively, save results to an Eventn hosted MySQL instance (a free 250Mb allocation is provided).

With the full power of modern JavaScript in each microservice, it is simple to perform ETL and SQL reporting on stored test results.

Integrating Eventn Microservices and Runscope Webhooks

Connecting Runscope to an Eventn service is a breeze using a Runscope Webhook. We simply set up a webhook to call the Eventn POST function of a microservice on test completion. To start with we will look at saving the data to Eventn’s built-in MySQL store and then look at how to store the data to a remote database.

To get started, first we need to create a free account at https://eventn.com and create a new microservice by clicking on“Create Service” or the plus button:

Once created, take note of the service URL and authentication token from the “Manage” tab:

Next, navigate to your Runscope account and select the “Connected Services” link from the icon in the top right-hand corner:

And select “Webhooks” from the list of available integrations:

Note: The advanced webhook functionality within Runscope is not enabled by default, so contact Runscope support to get this enabled.

Next, we need to configure the webhook by entering the POST URL and authentication token from the Eventn service. To set the authentication token, an Authorization header must be set with the token prefixed with Bearer. For example: Bearer xyz. You can see an example in the following screenshot:

Click “Save Changes”, navigate to one of your API tests, and edit the test environment settings. Select "Integrations" on the left-hand side menu and check the toggle to “on” next to our newly configured Webhook integration:

The test is now configured to make a POST request to the Eventn microservice after each completion. By default, it's going to send all test result data.

Now, run the test and then navigate back to your Eventn account. Select the “Edit” tab, then the “GET” subtab to view the JavaScript function that executes when a GET request is made. By default, the  Eventn GET function simply counts the number of records within a table. Click the “GET” button as shown below to run the function and a count of 1 will be returned if the test data has been saved:

Note the Eventn store() interface returns a Knex.js instance, which provides a powerful SQL query builder. For example, to update the onGet() function to show all stored records we could replace:

return context.stores.default()
.table()
.count();

With:

return context.stores.default()
.table()
.select();

Or perhaps, we could just return the most recent record:

return context.stores.default()
.table().select()
.limit(1)
.orderBy('ts_created', 'desc');

Given we have the full power of SQL, it is trivial to create more complex reports that can be consumed by any other down-stream services e.g. for visualization or alerting. Here is another example showing a breakdown of failed vs. passed tests:

function onGet(context) {

    return context.stores.default()
            .raw(`SELECT ts_created as Date,
                  DATE_FORMAT(ts_created,"%d %b %y") as Date_Human,
                  sum(case when data-> "$.result" = "pass" then 1 else 0 end) as Pass,
                  sum(case when data-> "$.result" = "fail" then 1 else 0 end) as Fail,
                  COUNT(*) AS Total 
                  FROM SV_P3MNNL9LJ
                  group by DAY(ts_created)`);

}

module.exports = onGet;

This would return a result such as:

[
  {
    "Date": "2017-04-01T09:45:08.000Z",
    "Date_Human": "01 Apr 17",
    "Pass": 4,
    "Fail": 2,
    "Total": 6
  },
  {
    "Date": "2017-04-02T01:45:20.000Z",
    "Date_Human": "02 Apr 17",
    "Pass": 203,
    "Fail": 0,
    "Total": 203
  }
....
]

And we could use another tool, such as Tableau or Excel, to create a visualization chart:

External Databases

The previous example showed how to utilize Eventn’s built-in MySQL storage however it is also possible to save the results to an external database of your choice. This is done by creating a store function. Store functions contain the database connection details and can be called from within a microservice. See the Eventn User Guide for details on creating a store function.

Once the store function is created, simply call it in the same way as the default() function and pass in a table name. In the following example, we created a store function called “mydb” and we are accessing a table named “results”:

return context.stores.mydb("results").count();

Tip: If you wish to test with a locally hosted database, ngrok provides a great way to expose a local database via a public address.

Note: It is best practice to create a dedicated DB user for such usage with restricted permissions e.g. just INSERT. Setting a restricted access port is also advised.

Conclusion

The above examples show how easy it is to save Runscope monitoring test results to a database. Eventn microservices make it easy to process the data and provide RESTful analytical services for other integrations.

If you need any help setting up your Runscope webhook integration, please reach out to us. If you need help with the Eventn platform, please reach out to Eventn's support.

Happy hacking!

---

If you're new to API monitoring and testing, learn more about Runscope's cloud-based API testing solution and sign up for your free trial account today.

---