load testing vegeta

Load Testing with Vegeta

Reviewed on 24 February 2023 • Published on 20 May 2019

Vegeta
Load
web-application
target-file
Ubuntu
performance
testing
install
vegata

Vegeta load testing

Load testing allows testing the behavior of an application under real conditions by simulating concurrent user requests. This can help determine the behavior of an application when multiple users access it simultaneously. This tutorial shows how to use Vegeta, an open-source application written in the Go programming language, to run load testings on an application.

The following problems can be identified with the help of load testing, before moving an application into a production environment:

Response time checking for each request
Performance of the overall system under different loads
Database performances under heavy loads
Verification if load balancing is working as designed
Network latency between the client and the application
Performance limitations due the technical specifications of the Instances used

Tip:

We recommend you follow this tutorial using a Production-Optimized Instance.

Security & Identity (IAM):

You may need certain IAM permissions to carry out some actions described on this page. This means:

you are the Owner of the Scaleway Organization in which the actions will be carried out, or
you are an IAM user of the Organization, with a policy granting you the necessary permission sets

Requirements:

You have an account and are logged into the Scaleway console
You have configured your SSH Key
You have created an Instance running on Ubnutu Linux

Downloading and installing Vegeta on Ubuntu Linux

Connect to your Instance via SSH.

Download the latest binary version to install Vegata on your Ubuntu Linux machine:

wget https://github.com/tsenart/vegeta/releases/download/v12.8.4/vegeta_12.8.4_linux_amd64.tar.gz

Unpack the binary file:

tar xfz vegeta-12.8.4-linux-amd64.tar.gz

Move the binary file into /usr/bin to make it available system-wide. This step actually installs the Vegata application on your Ubuntu system:
```
mv vegeta /usr/bin/vegeta
```

Running a load test with Vegeta

To run a load test during 120 seconds, run the following command. It allows you to test the performance using Vegeta:

echo "GET http://<application_url>/" | vegeta attack -duration=120s | tee results.bin | vegeta report

The command above will return a report like the following:

Requests      [total, rate]            6000, 50.01
Duration      [total, attack, wait]    2m0.072491358s, 1m59.980003112s, 92.488246ms
Latencies     [mean, 50, 95, 99, max]  76.235976ms, 72.996349ms, 109.192641ms, 133.960662ms, 187.907358ms
Bytes In      [total, mean]            65862000, 10977.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:6000
Error Set:

Requests shows the total number of requests sent during the test and the rate of the requests
Duration shows the total duration of the test, the attack period simulating load on the application and the wait time
Latencies shows the mean latency, 50th, 95th an 99th percentiles, respectively, of the latencies of all requests in an attack as well as the maximum latency recognized.
Bytes In and Bytes Out shows:
- The total number of bytes sent (out) or received (in) with the request or response bodies.
- The mean number of bytes sent (out) or received (in) with the request or response bodies.
Success shows the percentiles of successful requests sent to the application
Status Codes provides a counter of the HTTP response codes received and their occurrence. A 0 status codes mean a request failed to be sent.

As visible in the example above, 100 % of the requests sent to the application were successful.

Following another result:

Requests      [total, rate]            6000, 50.01
Duration      [total, attack, wait]    2m9.500461177s, 1m59.980081991s, 9.520379186s
Latencies     [mean, 50, 95, 99, max]  843.903272ms, 77.954318ms, 3.744635321s, 25.164627897s, 30.000611567s
Bytes In      [total, mean]            62842863, 10473.81
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  95.38%
Status Codes  [code:count]             0:45  200:5723  500:33  502:13  503:186
Error Set:
500 Internal Server Error
502 Bad Gateway
Get http://<application_url>/: EOF
503 Service Unavailable
Get http://<application_url>/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

As visible in the output above, only 95.38% of the requests sent to the application were successful. In the Status Code section, it is visible that 5723 requests were successful (HTTP Code 200), 186 requests received an HTTP 503 error, and 13 requests received an HTTP 502 error.

These results show that the application has trouble handling the load and needs optimization.

Setting up a target file

Vegeta also supports multiple targets to attack an application with multiple endpoints simultaneously. This allows you to test the performance of your application (e.g. an API) with Vegeta:

Create a target file and open it in a text editor:
```
nano target.list
```

Put the different endpoints into it:

GET http://<application_url>/list/user/1
GET http://<application_url>/list/user/2
GET http://<application_url>/list/user/3

It is also possible to define POST or PUT commands into the file.

POST http://<application_url>/create/newuser/
Content-Type: application/json
@/path/to/newuser.json

The content of the file /path/to/newuser.json contains the body of the request:

{
  "name": "Peter";
  "lastname": "Smith";
  "email": "psmith@example.com"
}

Run Vegeta, the -duration flag specifies the duration of the attack, the -rate flag defines the number of requests per time unit (the default value is set to 50/1s):
```
vegeta attack -duration=5s -rate=5 -targets=target.list
```

Generating a report

Every load test executed to collect data that can be saved in an output file using the -output parameter.

The data collected during each load test is stored in an output file.

Run an attack in Vegeta. The default filename of the output file is results.bin. It can be specified to any name with the -output flag:
```
vegeta attack -duration=120s -rate=100 -targets=target.list -output=attack-5.bin
```
Generate the HTML report and plot with the vegeta plot command. The -title flag allows specifying a title for the plot.
```
vegeta plot -title=Attack%20Results attack-5.bin > results.html
```
Open the HTML file in a web browser to view the plot:

Alternatively, it is also possible to generate the report as JSON output:

vegeta report -type=json results.bin

Returning a JSON output as follows:

{
  "latencies":{
      "total":5018316104,
      "mean":418193,
      "50th":393268,
      "95th":523664,
      "99th":737809,
      "max":13479790
  },
  "bytes_in":{
      "total":7344000,
      "mean":612
  },
  "bytes_out":{
      "total":0,
      "mean":0
  },
  "earliest":"2019-05-23T15:14:25.216411448+02:00",
  "latest":"2019-05-23T15:16:25.206316941+02:00",
  "end":"2019-05-23T15:16:25.206710805+02:00",
  "duration":119989905493,
  "wait":393864,
  "requests":12000,
  "rate":100.00841279685865,
  "success":1,
  "status_codes":{
      "200":12000
  },
  "errors":[
  ]
}

To view the report directly in the terminal, run:

vegeta report results.bin

Which provides an report in the format seen in the first steps:

Requests      [total, rate]            12000, 100.01
Duration      [total, attack, wait]    1m59.990299357s, 1m59.989905493s, 393.864µs
Latencies     [mean, 50, 95, 99, max]  418.193µs, 393.268µs, 523.664µs, 737.809µs, 13.47979ms
Bytes In      [total, mean]            7344000, 612.00
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:12000
Error Set:

For more information about Vegeta, refer to the official documentation.