wrk is the HTTP benchmarking tool which combined with Lua scripts can be the bazooka in your benchmarking arsenal. This article shows an advanced example of using Lua scripts with wrk. I will also show you how to debug wrk.

Introduction

Recently I published an article on Digital Ocean which is an introduction to a great HTTP benchmarking tool called wrk. It describes the basic concepts and shows usage examples. I recommend you read it before continuing with this post.

And after you are back we will focus on a more advanced example.

Requirements

For some time now I use Docker instead of virtual machines when possible.

I would like to share the Docker awesomeness with you, so please make sure you have the following tools installed:

JSON File Example

The idea is very simple. We have an JSON file with request details and wrk will use this information in its benchmark. wrk doesn’t have this feature build in but we will use a Lua script to tell it what to do.

The JSON file might look like this:

[
  {
    "path": "/path-1",
    "body": "some content",
    "method": "GET",
    "headers": {
      "X-Custom-Header-1": "test 1",
      "X-Custom-Header-2": "test 2"
    }
  },
  {
    "path": "/path-2",
    "body": "some content",
    "method": "POST",
    "headers": {
      "X-Custom-Header-1": "test 3",
      "X-Custom-Header-2": "test 4"
    }
  }
]

As you can see each request has different properties.

Save it in the data directory as data/requests.json.

Now we need a proper lua script.

I prepared one below, save it as scripts/multi-request-json.lua:

-- Module instantiation
local cjson = require "cjson"
local cjson2 = cjson.new()
local cjson_safe = require "cjson.safe"

-- Initialize the pseudo random number generator
-- Resource: http://lua-users.org/wiki/MathLibraryTutorial
math.randomseed(os.time())
math.random(); math.random(); math.random()

-- Shuffle array
-- Returns a randomly shuffled array
function shuffle(paths)
  local j, k
  local n = #paths

  for i = 1, n do
    j, k = math.random(n), math.random(n)
    paths[j], paths[k] = paths[k], paths[j]
  end

  return paths
end

-- Load URL paths from the file
function load_request_objects_from_file(file)
  local data = {}
  local content

  -- Check if the file exists
  -- Resource: http://stackoverflow.com/a/4991602/325852
  local f=io.open(file,"r")
  if f~=nil then
    content = f:read("*all")

    io.close(f)
  else
    -- Return the empty array
    return lines
  end

  -- Translate Lua value to/from JSON
  data = cjson.decode(content)


  return shuffle(data)
end

-- Load URL requests from file
requests = load_request_objects_from_file("/data/requests.json")

-- Check if at least one path was found in the file
if #requests <= 0 then
  print("multiplerequests: No requests found.")
  os.exit()
end

print("multiplerequests: Found " .. #requests .. " requests")

-- Initialize the requests array iterator
counter = 1

request = function()
  -- Get the next requests array element
  local request_object = requests[counter]

  -- Increment the counter
  counter = counter + 1

  -- If the counter is longer than the requests array length then reset it
  if counter > #requests then
    counter = 1
  end

  -- Return the request object with the current URL path
  return wrk.format(request_object.method, request_object.path, request_object.headers, request_object.body)
end

This script is similar to the example presented in the Digital Ocean artilce. It is extended with an shuffle functionality and instead of loading URL paths line by line we laod a JSON file. Parsing JSON requires a JSON library, which is loaded at the very top. This library transforms the request details from a JSON string into an Lua array.

The only thing that stops us from testing this example is the missing JSON library itself - we need to install it. To save time for the installation and setup, I prepared a Docker image, which is available here. If you are interested in how the library is installed simply take a look at the Dockerfile - it’s well documented.

Let’s summarise: you need to save the JSON file as data/requests.json and the Lua script as scripts/multi-request-json.lua.

We will run wrk in a Docker container. This diagram will give you a good overview of how this is done:

container overview for the wrk JSON example

Run the benchmark with:

docker run --rm \
           -v `pwd`/scripts:/scripts \
           -v `pwd`/data:/data \
           czerasz/wrk-json wrk -c1 -t1 -d5s -s /scripts/multi-request-json.lua http://$APPLICATION_IP:$APPLICATION_PORT

This command will pull the czerasz/wrk-json image from the public Docker registry. Then it executes the wrk command (inside the Docker container) which additionally takes the Lua script. The Lua script opens the /data/requests.json file, parses it and feeds wrk with the data. Everything is possible because we shared the appropriate direcotries with the Docker container by using the -v option.

The returned output might look like this:

multiplerequests: Found 2 requests
multiplerequests: Found 2 requests
Running 5s test @ http://10.135.232.163:3000
  1 threads and 1 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.32ms    1.41ms  18.91ms   91.81%
    Req/Sec     0.89k   304.69     1.47k    64.00%
  4447 requests in 5.00s, 0.85MB read
Requests/sec:    888.67
Transfer/sec:    174.44KB

Adding weights to this example could be also very easy: just duplicate the request objects or adjust the Lua script to recognize a weight property.

I hope that you understand now that “sky is the limit” with wrk and Lua.

Debugging WRK

The ability to debug a technology properly is worth more than knowing the technology itself.

To debug wrk I use a special Node.js application and a custom Lua script. The whole environment (based on Docker) is available on Github.

If you want to use it, then we need to install the required software. We just need:

  • git - to download the project
  • docker-compose - to start the Docker container environment

I will help you setup them on Ubuntu.

Install git simply by using Ubuntu’s package manager:

apt-get install -y git

Docker Compose can be installed with this commands:

sudo curl -L https://github.com/docker/compose/releases/download/1.3.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

If you have any trouble or want to install docker-compose on another system please refer to this Docker Compose installation guide.

Now clone the project:

git clone https://github.com/czerasz/wrk-debugging-environment.git

Enter it:

cd wrk-debugging-environment

And spin up the required containers with:

docker-compose run --rm wrk bash

The command above will create two containers - one for the application and one for wrk.

An overview is presented below:

container overview for the wrk debugging scenario

The command will also automatically log you in to the wrk container.

Now whenever you execute this command inside the wrk Docker container:

wrk -c3 -d1s -t2 -s /scripts/debug.lua http://app:3000 -- debug true

You will see what happens inside wrk because the debugging scipt has a lot of debugging messages (integrated simply with io.write).

Sample output is presented below:

...
------------------------------
Response 114 with status: 200 on thread 1
------------------------------
[response] Headers:
[response]  - Content-Length: 2
[response]  - ETag: W/"2-REvLOj/Pg4kpbElGfyfh1g"
[response]  - Connection: keep-alive
[response]  - Date: Tue, 23 Jun 2015 20:18:01 GMT
[response]  - Content-Type: text/html; charset=utf-8
[response]  - X-Debug: true
[response]  - X-Powered-By: Express
[response] Body:
ok

  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.42ms    4.23ms  36.86ms   93.15%
    Req/Sec   305.55    162.64   590.00     70.00%
  611 requests in 1.00s, 128.29KB read
Requests/sec:    608.23
Transfer/sec:    127.70KB
------------------------------
Requests
------------------------------
userdata{
  ["bytes"] = "49020",
  ["errors"] =   {
    ["write"] = "0",
    ["read"] = "0",
    ["status"] = "0",
    ["timeout"] = "0",
    ["connect"] = "0"
  },
  ["duration"] = "1009245",
  ["requests"] = "228"
}

Additionally the application container will log all request details. Simply open another terminal, enter the wrk-debugging-environment directory:

cd wrk-debugging-environment

Then execute this command:

docker logs -f --tail=0 $(docker-compose ps | grep '_application_1' | awk '{print $1}')

Whenever you execute the previous benchmak command (in the previous terminal) you should see output simmilar to this:

...

--- --- --- --- --- --- --- --- --- --- --- --- ---

[2015-06-23 20:28:30] Request 486

GET/1.1 / on :::3000

Headers:
 - host: app:3000

No cookies

Body:

Now just edit the environments/wrk/scripts/debug.lua file and see the output changing. Play around and have fun.

Summary

I hope that this article and the one on Digital Ocean gave you a better understanding about wrk and explained how to precisely shoot with heavy benchmarking bullets.

Resources