Load Testing with Vegeta
- Vegeta
- Load
- web-application
- target-file
- Ubuntu
- performance
- testing
- install
- vegata
Vegeta load testing
Load testing allows testing the behavior of an application under real conditions by simulating concurrent user requests. This can help determine the behavior of an application when multiple users access it simultaneously. This tutorial shows how to use Vegeta, an open-source application written in the Go programming language, to run load testings on an application.
The following problems can be identified with the help of load testing, before moving an application into a production environment:
- Response time checking for each request
- Performance of the overall system under different loads
- Database performances under heavy loads
- Verification if load balancing is working as designed
- Network latency between the client and the application
- Performance limitations due the technical specifications of the Instances used
We recommend you follow this tutorial using a Production-Optimized Instance.
You may need certain IAM permissions to carry out some actions described on this page. This means:
- you are the Owner of the Scaleway Organization in which the actions will be carried out, or
- you are an IAM user of the Organization, with a policy granting you the necessary permission sets
- You have an account and are logged into the Scaleway console
- You have configured your SSH Key
- You have created an Instance running on Ubnutu Linux
Downloading and installing Vegeta on Ubuntu Linux
- Connect to your Instance via SSH.
- Download the latest binary version to install Vegata on your Ubuntu Linux machine:
wget https://github.com/tsenart/vegeta/releases/download/v12.8.4/vegeta_12.8.4_linux_amd64.tar.gz
- Unpack the binary file:
tar xfz vegeta-12.8.4-linux-amd64.tar.gz
- Move the binary file into
/usr/bin
to make it available system-wide. This step actually installs the Vegata application on your Ubuntu system:mv vegeta /usr/bin/vegeta
Running a load test with Vegeta
To run a load test during 120 seconds, run the following command. It allows you to test the performance using Vegeta:
echo "GET http://<application_url>/" | vegeta attack -duration=120s | tee results.bin | vegeta report
The command above will return a report like the following:
Requests [total, rate] 6000, 50.01Duration [total, attack, wait] 2m0.072491358s, 1m59.980003112s, 92.488246msLatencies [mean, 50, 95, 99, max] 76.235976ms, 72.996349ms, 109.192641ms, 133.960662ms, 187.907358msBytes In [total, mean] 65862000, 10977.00Bytes Out [total, mean] 0, 0.00Success [ratio] 100.00%Status Codes [code:count] 200:6000Error Set:
Requests
shows the total number of requests sent during the test and the rate of the requestsDuration
shows the total duration of the test, the attack period simulating load on the application and the wait timeLatencies
shows the mean latency, 50th, 95th an 99th percentiles, respectively, of the latencies of all requests in an attack as well as the maximum latency recognized.Bytes In
andBytes Out
shows:- The
total
number of bytes sent (out) or received (in) with the request or response bodies. - The
mean
number of bytes sent (out) or received (in) with the request or response bodies.
- The
Success
shows the percentiles of successful requests sent to the applicationStatus Codes
provides a counter of the HTTP response codes received and their occurrence. A0
status codes mean a request failed to be sent.
As visible in the example above, 100 % of the requests sent to the application were successful.
Following another result:
Requests [total, rate] 6000, 50.01Duration [total, attack, wait] 2m9.500461177s, 1m59.980081991s, 9.520379186sLatencies [mean, 50, 95, 99, max] 843.903272ms, 77.954318ms, 3.744635321s, 25.164627897s, 30.000611567sBytes In [total, mean] 62842863, 10473.81Bytes Out [total, mean] 0, 0.00Success [ratio] 95.38%Status Codes [code:count] 0:45 200:5723 500:33 502:13 503:186Error Set:500 Internal Server Error502 Bad GatewayGet http://<application_url>/: EOF503 Service UnavailableGet http://<application_url>/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
As visible in the output above, only 95.38% of the requests sent to the application were successful. In the Status Code
section, it is visible that 5723 requests were successful (HTTP Code 200), 186 requests received an HTTP 503 error, and 13 requests received an HTTP 502 error.
These results show that the application has trouble handling the load and needs optimization.
Setting up a target file
Vegeta also supports multiple targets to attack an application with multiple endpoints simultaneously. This allows you to test the performance of your application (e.g. an API) with Vegeta:
-
Create a target file and open it in a text editor:
nano target.list -
Put the different endpoints into it:
GET http://<application_url>/list/user/1GET http://<application_url>/list/user/2GET http://<application_url>/list/user/3It is also possible to define
POST
orPUT
commands into the file.POST http://<application_url>/create/newuser/Content-Type: application/json@/path/to/newuser.jsonThe content of the file
/path/to/newuser.json
contains the body of the request:{"name": "Peter";"lastname": "Smith";"email": "psmith@example.com"} -
Run Vegeta, the
-duration
flag specifies the duration of the attack, the-rate
flag defines the number of requests per time unit (the default value is set to 50/1s):vegeta attack -duration=5s -rate=5 -targets=target.list
Generating a report
Every load test executed to collect data that can be saved in an output file using the -output parameter.
The data collected during each load test is stored in an output file.
-
Run an attack in Vegeta. The default filename of the output file is
results.bin
. It can be specified to any name with the-output
flag:vegeta attack -duration=120s -rate=100 -targets=target.list -output=attack-5.bin -
Generate the HTML report and plot with the
vegeta plot
command. The-title
flag allows specifying a title for the plot.vegeta plot -title=Attack%20Results attack-5.bin > results.html -
Open the HTML file in a web browser to view the plot:
-
Alternatively, it is also possible to generate the report as JSON output:
vegeta report -type=json results.binReturning a JSON output as follows:
{"latencies":{"total":5018316104,"mean":418193,"50th":393268,"95th":523664,"99th":737809,"max":13479790},"bytes_in":{"total":7344000,"mean":612},"bytes_out":{"total":0,"mean":0},"earliest":"2019-05-23T15:14:25.216411448+02:00","latest":"2019-05-23T15:16:25.206316941+02:00","end":"2019-05-23T15:16:25.206710805+02:00","duration":119989905493,"wait":393864,"requests":12000,"rate":100.00841279685865,"success":1,"status_codes":{"200":12000},"errors":[]} -
To view the report directly in the terminal, run:
vegeta report results.binWhich provides an report in the format seen in the first steps:
Requests [total, rate] 12000, 100.01Duration [total, attack, wait] 1m59.990299357s, 1m59.989905493s, 393.864µsLatencies [mean, 50, 95, 99, max] 418.193µs, 393.268µs, 523.664µs, 737.809µs, 13.47979msBytes In [total, mean] 7344000, 612.00Bytes Out [total, mean] 0, 0.00Success [ratio] 100.00%Status Codes [code:count] 200:12000Error Set:
For more information about Vegeta, refer to the official documentation.