Fourth in a series focusing on using JMeter for functional testing of an API.
So far we have relied solely on JMeter’s built in ability to tell if our API calls have passed or failed based on the HTTP response code we receive from the server. So a Signin response returning a 200 code would display as a pass (or success) and a 401 code would display as a failure.
This will often not be enough to validate the response from the server was in fact correct. The server could return a good response code but send bad data back which could cause problems down the line. In this tutorial we’ll enhance our Library module for Signin to validate that we are getting a token field in the response. It will be a very simple introduction to some more complicated validation we will do in a later module.
Before we begin, some notes:
- If you skipped the But Wait! addition to the third tutorial (which is totally okay), download this test plan to use with this tutorial. Also download this csv file which contains our user login info for our updated test plan. The csv file must be in the same folder as your test plan.
- I also am making the jump to JMeter 4.0 for this tutorial as certain plugins needed to do JSON validation in 3.0 have been replaced with build in features in JMeter 4.0 and will make things easier. You should remember to install the Parameterized Controller Plugin again if you started with JMeter 3.0.
Functional Testing 04: Everyone Needs a Little Validation
- First modify our credential files to include a login that will result in the test server sending back a bad response. Open crendetials.csv and add the last line at the bottom so it looks like this:
- Open LearnJmeter_03-1.jmx in JMeter and run the test plan. Your Results Tree should look like this:
Notice the last Signin test for Loki, whose response code is 200 and yet the response body does not contain a token. It instead contains a message. This is not correct as we would expect one of two results; either (1) a 200 code and a token or (2) a 401 code with an Unauthorized message. In the case above it looks like a bug in our signin code and we would expect this test to fail, so let’s add some validation to make sure it does.
- Expand Sampler Library > SignIn
- Right-click on “POST Signin” and select Add > Assertions > JSON Assertion
- We will use this JSON assertion to make sure that the response from the server contains a “token” field. In the JSON assertion:
Set Assert JSON Path exists to “$.token“.
In this case “$” refers to the root of the JSON response from the server and and “.token” refers to a token field on the root of the response.
- Now run the test plan again and we should see the last signin attempt fail.
You can see that the call fails because it is in red and the JSON assertion is telling you that it could not find “$.token” in the response from the server. This is good! We expected the test to pass, however while the server replied with a good 200 code, we didn’t get the data we expected – so the test failed.
This is really the most basic of validation that you can do. Notice that we don’t do any validation on the token value itself… yet. In a future tutorial we will do many more and more complex validations.