A quick guide to Redis Lua scripting

Redis is a popular in-memory grid used for interprocess communication and data storage. You might’ve heard that it lets you run Lua scripts, but you are still not sure why. If this sounds like you, read on.

Prerequisites

You should have Redis installed on your system to follow this guide. It might be helpful to check Redis commands reference while reading.

Why do I need Lua Scripts?

In short: performance gain. Most tasks you do in Redis involve many steps. Instead of doing these steps in the language of your application, you can do it inside Redis with Lua.

• This may result in better performance.
• Also, all steps within a script are executed in an atomic way. No other Redis command can run while a script is executing.

For example, I use Lua scripts to change JSON strings stored in Redis. I describe this in detail closer to the end of this article.

But I don’t know any Lua

Don’t worry, Lua is not very hard to understand. If you know any language of the C family, you should be okay with Lua. Also, I am providing working examples in this article.

Show me an example

Let’s start by running scripts via redis-cli. Start it with:

redis-cli

Now run the following command:

eval “redis.call(‘set’, KEYS[1], ARGV[1])” 1 key:name value

The EVAL command is what tells Redis to run the script which follows. The ”redis.call(‘set’, KEYS[1], ARGV[1])”string is our script which is functionally identical to the Redis’s set command. Three parameters follow the script text:

1. The number of provided keys
2. Key name
3. First argument

Script arguments fall into two groups: KEYS and ARGV.

We specify how many keys the script requires with the number immediately following it. In our example, it is 1. Immediately after this number, we need to provide these keys, one after another. They are accessible as KEYS table within the script. In our case, it contains a single value key:name at index 1.

Note, that Lua indexed tables start with index 1, not 0.

We can provide any number of arguments after the keys, which will be available in Lua as the ARGV table. In this example, we provide a single ARGV-argument: string value. As you already guessed, the above command sets the key key:name to value value.

It is considered a good practice to provide keys which the script uses as KEYS, and all other arguments as ARGV. So you shouldn’t specify KEYS as 0 and then provide all keys within the ARGV table.

Let’s now check if the script completed successfully. We are going to do this by running another script which gets the key from Redis:

eval “return redis.call(‘get’, KEYS[1])” 1 key:name

The output should be ”value”, which means that the previous script successfully set the key “key:name”.

Can you explain the script?

Our first script consists of a single statement: the redis.call function:

redis.call(‘set’, KEYS[1], ARGV[1])

Withredis.call you can execute any Redis command. The first argument is the name of this command followed by its parameters. In the case of the set command, these arguments are key and value. All Redis commands are supported. According to the documentation:

Redis uses the same Lua interpreter to run all the commands

Our second script does a little more than just running a single command — it also returns a value:

eval “return redis.call(‘get’, KEYS[1])” 1 key:name

Everything returned by the script is sent to the calling process. In our case, this process is redis-cli and you will see the result in your terminal window.

Something more complex?

I once used Lua scripts to return elements from a hash map in a particular order. The order itself was specified by hash keys stored in a sorted set.

Let’s first set up our data by running these commands in redis-cli:

hmset hkeys key:1 value:1 key:2 value:2 key:3 value:3 key:4 value:4 key:5 value:5 key:6 value:6
zadd order 1 key:3 2 key:1 3 key:2

These commands create a hash map at key hkeys and a sorted set at key order which contains selected keys from hkeys in a specific order.

You might want to check hmset and zadd commands reference for details.

Let’s run the following script:

eval “local order = redis.call(‘zrange’, KEYS[1], 0, -1); return redis.call(‘hmget’,KEYS[2],unpack(order));” 2 order hkeys

You should see the following output:

“value:3”
“value:1”
“value:2”

Which means that we got values of the keys we wanted and in the correct order.

Do I have to specify full script text to run it?

No! Redis allows you to preload a script into memory with the SCRIPT LOAD command:

script load “return redis.call(‘get’, KEYS[1])”

You should see an output like this:

“4e6d8fc8bb01276962cce5371fa795a7763657ae”

This is the unique hash of the script which you need to provide to the EVALSHA command to run the script:

evalsha 4e6d8fc8bb01276962cce5371fa795a7763657ae 1 key:name

Note: you should use actual SHA1 hash returned by the SCRIPT LOAD command, the hash above is only an example.

What did you mention about changing JSON?

Sometimes people store JSON objects in Redis. Whether it is a good idea or not is another story, but in practice, this happens a lot.

If you have to change a key in this JSON object, you need to get it from Redis, parse it, change the key, then serialize and set it back to Redis. There are a couple of problems with this approach:

1. Concurrency. Another process can change this JSON between our get and set operations. In this case, the change will be lost.
2. Performance. If you do these changes often enough and if the object is rather big, this might become the bottleneck of your app. You can win some performance by implementing this logic in Lua.

Let’s add a test JSON string to Redis under key obj:

set obj ‘{“a”:”foo”,”b”:”bar”}’

Now let’s run our script:

EVAL ‘local obj = redis.call(“get”,KEYS[1]); local obj2 = string.gsub(obj,”(“ .. ARGV[1] .. “\”:)([^,}]+)”, “%1” .. ARGV[2]); return redis.call(“set”,KEYS[1],obj2);’ 1 obj b bar2

Now we will have the following object under key obj:

{“a”:”foo”,”b”:”bar2"}

You can instead load this script with the SCRIPT LOAD command:

SCRIPT LOAD ‘local obj = redis.call(“get”,KEYS[1]); local obj2 = string.gsub(obj,”(“ .. ARGV[1] .. “\”:)([^,}]+)”, “%1” .. ARGV[2]); return redis.call(“set”,KEYS[1],obj2);’

and then run it like this:

EVALSHA <your_script_sha> 1 obj b bar2

Some notes:

• The .. is the string concatenation operator in Lua.
• We use a RegEx pattern to match key and replace its value. If you don’t understand this Regular Expression, you can check my recent guide.
• One difference of the Lua RegEx flavor from most other flavors is that we use % as both backreference mark and escape character for RegEx special symbols.
• We still escape ” with \ and not % because we escape Lua string delimiter, not RegEx special symbol.

Should I always use Lua scripts?

No. I recommend only using them when you can prove that it results in better performance. Always run benchmarks first.

If all you want is atomicity, then you should check Redis transactions instead.

Also, your script shouldn’t be too long. Remember that while a script is running, everything else is waiting for it to finish. If your script takes quite some time, it can cause bottlenecks instead of improving performance. The script stops after reaching a timeout (5 seconds by default).

Last Word

For more information on Lua check lua.org.

You can check my node.js library on GitHub for some examples of Lua scripts (see src/lua folder). You can also use this library in node.js to change JSON objects without writing any Lua scripts yourself.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Thank you for reading this article. Questions and comments are much appreciated. You are also welcome to follow me on Twitter.