Today, we're really excited to talk about the release of the BlazeMeter Continuous Testing platform, and the integration of Runscope's API Monitoring capabilities as part of this new offering.

We have two great blog posts talking more about this new platform, why we're excited about it, and how we believe it'll help bring value and transform teams looking to shift-left and adopt modern testing practices:

• Reimagining Software Testing for Agile and DevOps by Sushil Kumar (Head of Broadcom's Continuous Testing Business)

• Introducing The BlazeMeter Continuous Testing Platform by Stephen Feloney (Head of Products for Broadcom's Continuous Testing Business)

For Runscope users, in addition to the API Monitoring features there's also some new exciting capabilities that you'll be able to use including:

• Performance Testing: run massively scalable, open source-based performance tests against all of your apps, from web and mobile apps to microservices and APIs. Validate performance at every software delivery stage with advanced analytics, collaboration and more. Upload and configure your scripts with an intuitive UI leveraging your favorite open source tools, Apache JMeter™, Selenium, Gatling, Grinder, Locust and more. 

• GUI Functional Testing: create GUI Functional Tests (either through your existing Selenium scripts or scriptless with BlazeMeter’s Chrome recorder) to test your application UX, then run those tests under load to understand how your front end performs with heavy traffic.

• Mock Services: create mock services to run tests, removing dependencies on external or internal resources that may not be easily available. The Mock Services feature accommodates popular open source and developer friendly tools including WireMock and CodeSV, so that developers can choose the tools that best suit their needs. Scripts can be reused and shared by different teams for optimum collaboration and minimum bottlenecks.

BlazeMeter is a powerful new platform, and the world’s first unified Continuous Testing platform for enterprises. We're excited about this new release, and to be part of a single product that helps teams automate and create tests at every stage of the development lifecycle.

To learn more about the new platform, sign up for our upcoming webinar or request a demo.

Just as we can choose a bad dish in a restaurant, we can sometimes use the wrong HTTP method, which can often be blamed on poor/unclear menus (documentation). Even though they should, the endpoint documentation often does not adequately detail what methods are supported, or they use the wrong verb entirely.

In this context, tools should also be used with caution. Take curl for example. If a user issues a GET request with the data option (-d), but forgets to specify -XGET, then it will automatically default to a POST request and include the `Content-Type:application/x-www -form-urlencoded` header.

In all cases, it’s important to carefully read the documentation and not fall victim to making incorrect assumptions – especially when working across different tools and APIs. For example, the Runscope® API uses a POST method when creating API test environments or steps, and PUT to modify them. On the other hand, the Stripe® API uses POST to both create and update objects. While both are perfectly valid, the goal of an API provider is to remain consistent and keep documentation clear and up to date.

Just like failing to add salt to a dish causes a diner to wince, forgetting a single “s” can cause some surprises during API testing. Many APIs will only support HTTPS, while others may support a combo of HTTP for some endpoints and not others. Even if an API has the apparent flexibility to support both, there can still be problems.

For example, some APIs will conveniently redirect HTTP requests to their HTTPS counterpart; however, not all frameworks are configured to follow a 302-status code.

An example is the Node.js® ‘request’ module, which will follow a GET redirect by default, but has to be explicitly configured to follow non-GET responses (PUSH, POST, etc) as redirects. It is also common for an API provider to stop supporting HTTP. Good providers will notify you well in advance via their websites, dev communities, and social media. An example was the Instant Payment Notification microsite of PayPal.®

It’s important to stay up-to-date and integrate important update notices into your own developer communication channels. On the API provider side, it’s important that HTTPS strategies are in place to ensure the secure, private, and reliable connections users, customers and partners expect from your APIs. The process for getting certificates might have previously been used as an excuse for not moving to HTTPS, however, solutions like Let’s Encrypt® – an open certificate authority – have helped lessen the pain.

A good waiter will find out what’s wrong with your order and correct the problem with minimal fuss. Similarly, a good API error message will help developers quickly determine coding issues and what’s needed to fix them.

HTTP provides 70+ status codes that can be used, but a pragmatic approach suggests implementing at least three status codes (200, 300, 500) and augmenting these with status codes that are consistent and have actionable meaning across multiple APIs. Another best practice is to use concise error messages, with links guiding developers back to documentation.

As an API consumer, assumptions can again cause problems. While a status code of 200 suggests everything is ok, it’s not always so clear cut. For example, Facebook’s® Graph API can return a status code 200, however, the actual error message is included in the response data.

It’s important therefore to be mindful when working with APIs, making sure to carefully read documentation and responses as you learn how they work.

A good API error message will help developers quickly determine coding issues and what’s needed to fix them.

Trying to get a same-day reservation at a popular restaurant can be nearly impossible, especially during peak dinner hours. In the same way, some APIs require ‘authorization’ and you must be careful to not try to use ‘authentication’ instead, where word similarities can cause headaches. For example, OAuth 2 implementations usually require including an ‘Authorization’ header, but it’s easy to type ‘Authentication’ instead. It’s a common problem, so if a request keeps failing make sure you’re using the correct word. Authorization header construction also has a few traps for the unwary. OAuth 2 tokens must be prepended with a “Bearer” to work, and with respect to Authentication, common mistakes include:

• Forgetting the ‘Basic’ prefix (including a space)

• Not encoding ‘username: password’ or forgetting the colon

• Falling foul to quirkiness – e.g. where an API provider only requires a username (often the API key), yet it’s still necessary to place a colon after it (even if there’s no password)

Ordering your favorite dish at a new restaurant can often result in an unexpected taste and ingredients. Failing to acknowledge the quirks and details of how APIs and tools behave differently can also cause unexpected results. This is especially true in the case of HTTP redirects.

Redirects can be tricky and can also be triggered by small details. Like we mentioned before, HTTP API endpoints might redirect to their HTTPS counterparts. Another common occurrence is endpoints with trailing slashes, such as https://yourapihere.com/users/ being redirected to https://yourapihere.com/users or vice-versa. Often times, after an API call goes through a redirect, the HTTP method for that request will default to GET, no matter what the original request method was.

This behavior can vary between APIs and can also vary between tools. Again, using curl to illustrate, specifying -L to follow HTTP redirects together with an -X option (which is actually superfluous) will result in that method also being used on the redirected-to requests, which may not be what the server asks for at all. As Internet protocols guru and inventor of Curl® Daniel Stenberg points out in his unnecessary use of a curl-X blog, dropping the -X makes curl adhere to what the server asks for. If you want to alter what method to use on a redirect, curl has options to cover this -- named “--post301,--post302” and “--post303.”

Once again, staying ahead by reading and keeping up-to-date with documentation is absolutely critical, especially when using open source tools that are updated regularly.

When at the restaurant, we can never assume that the kitchen will be aware of your dietary requirements mentioned when you placed your order. It’s the same with APIs, especially when negotiating the type of information sent or received between client and servers via Accept and Content-Type headers. While some APIs will conveniently accept requests that don’t contain any of these headers (defaulting to good old JSON and XML), others will be stricter and return a 403 error when you’re not being specific. This issue can cause confusion when testing using different tools. Curl,® for example, together with tools such as Postman, will automatically include an ‘Accept’ header for any MIME type: ‘*/*’ with every request, while Runscope doesn’t add a default Accept header. The net-net could mean getting different results when testing the same end-point, so be prepared to cover all bases.

Some APIs will conveniently accept requests that don’t contain any Accept and Content-Type headers, while others are stricter.

So, you thought you ordered mac and cheese but the server places a bowl of nachos in front of you. It’s the same story with APIs. Like cases where an API returned an HTML error page instead of the JSON you expected, causing your code to self-combust.

As mentioned above, this is often due to not specifying ‘Accept’ header with the request and the API not being sure what response format your code is expecting. On the provider side, many frameworks and web servers default to HTML, so if you’re creating an API that has no good reason for dishing up HTML, always make sure to check the default error response.

Other complicating factors might have nothing to do with your API, but be the result of software sitting in front (e.g. load balancers). If for example, an nginx instance encounters a request time-out, it can return an HTML error page by default before your API ever gets the chance to determine what’s cooking.

Have you ever tried to order from a restaurant when you and the waiter don't speak the same language? You might face similar confusion when trying to debug SSL errors since returned messages can be cryptic or don’t provide indications on how to fix the problem.  Many technologies and tools handle SSL differently, however, two of the most common SSL errors (invalid certificates and incomplete certificate chains) are very easy to remedy. 

When running into invalid certificate problems, the first action is to make sure that your API testing tool supports the certificate authority (CA) that created the SSL certificate. At Runscope®, we use the Mozilla® CA Included Certificate List, but it’s easy to find what your tool supports with a quick Google search.

If the server’s SSL certificate provider is on the list, the next step is to debug the issue with the SSLLabs SSL Server Test - a free tool that performs a deep analysis of any public SSL web server. In addition to providing hostname grading, warnings and trust issues, SSLLabs SSL Server Test will also uncover incomplete certificate chain issues. This issue can cause confusion when testing with different tools. Curl,® for example, together with tools such as Postman, will automatically include an ‘Accept’ header for any MIME type: ‘*/*’ with every request, while Runscope doesn’t add a default Accept header. The net-net could mean getting different results when testing the same end-point, so be prepared to cover all bases. SSL Server Test will also uncover incomplete certificate chain issues.

SSL certificate chains can be handled differently across different browsers and tools. If a browser or a tool is trying to connect to a server it will try to check if its certificate was issued by a trusted CA. If it's not, then it will check whether the intermediate certificate was instead issued by a trusted CA, and so on. Those intermediate certificates can be bundled into one file, or they might be links that the browser or tool has to follow. Additionally, some browsers or tools won't follow those links, causing an SSL error.

If you have control of the server, fixing this issue requires bundling all intermediate certificates into a single file and updating your server with it. You can usually find instructions on how to do that with a quick Google search of " bundle certificates".

Like an old cheese board, API data problems can be rich, varied, and smelly. This can include:

• Content-length mismatched to actual content length causing connections to be prematurely closed and invalid data returned/parsed

• Unexpected gzip content because of missing Content-Type in the response header

• Unstructured response data or different structures from similar endpoints

• Putting an auth token in the wrong place (header -v- URL -v- payload)

Unfortunately, these and many others can cause developer frustration, with lengthy attempts to validate data using suspect coding hacks and poor workarounds.

Tools play an important role in resolving data issues – ranging from the simple to the hair-pulling variety. This can include simple assertions for JSON and XML, header and response body validation, but extending to support more complex API use-cases, such as chained-requests and dynamic data insertion (e.g. IDs and timestamps into requests).

No waiter worth their salt would crack a joke after they tell you that the kitchen has a problem with your order. Same in development, right?

Wrong! Back in 1998, a group of developers jokingly proposed the Hyper Text Coffee Pot Control Protocol and error 418 I’m a teapot (which is not a recognized standard server error type). However, perhaps because of its mega-meme status, many development teams have implemented “I’m a Teapot” in their applications and APIs, often using the code as a catch-all for bugs that are difficult to classify. However, well-intentioned, jokes can fall flat. For example, the May 2018 worldwide failure of NPM JavaScript package manager, which greeted developers with “ERR! 418 I’m a teapot” when they attempted to install or update a JavaScript Node.js package.

Humor is a great thing, but it shouldn’t compromise professionalism. Teams should be judicious in their use of error codes, avoiding jokes which may not be relevant to a frustrated user. In cases where more detail is needed, this should be provided with sub-status error codes and messages. Twilio® is a great example of best practice in this regard, going the extra mile with clear sub-status codes, messages, and links to more information.