Lua Script ​

Redis provides a rich command set, but users are still looking for ways to extend it with custom commands for specific domain issues. To accommodate this, Redis supports Lua scripting, allowing users to send Lua scripts to the server to execute custom actions and retrieve response data. The Redis server executes Lua scripts atomically in a single-threaded manner, ensuring that script execution is not interrupted by any other requests.

For example, in the section on distributed locks, we mentioned the pseudo-command del_if_equals, which atomically combines the matching and deletion of a key. Redis does not provide such functionality natively, but it can be achieved with a Lua script:

if redis.call("get", KEYS[1]) == ARGV[1] then
    return redis.call("del", KEYS[1])
else
    return 0
end

Executing the Script with EVAL ​

How do we execute the above script? Using the EVAL command:

127.0.0.1:6379> set foo bar
OK
127.0.0.1:6379> eval 'if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) else return 0 end' 1 foo bar
(integer) 1
127.0.0.1:6379> eval 'if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) else return 0 end' 1 foo bar
(integer) 0

The first parameter of the EVAL command is the script content string. In this example, we compressed the Lua script into a single line for convenience. Following that are the number of keys and each key string, along with a series of additional parameter strings. The number of additional parameters does not need to match the number of keys and can be completely absent.

EVAL SCRIPT KEY_NUM KEY1 KEY2 ... KEYN ARG1 ARG2 ....

In the above example, there is only one key, which is foo, and bar is the only additional parameter. In the Lua script, array indexing starts from 1, so KEYS[1] retrieves the first key, and ARGV[1] retrieves the first additional parameter. The redis.call function allows us to invoke native Redis commands; the code calls the get and del commands. The result returned will be sent back to the client.

SCRIPT LOAD and EVALSHA Commands ​

In the previous example, the script content was short. If the script is long and the client needs to execute it frequently, passing lengthy script content each time can waste network traffic. Therefore, Redis provides the SCRIPT LOAD and EVALSHA commands to solve this issue.

The SCRIPT LOAD command allows a client to send a Lua script to the server without executing it, returning a unique ID for that script. This ID uniquely identifies the Lua script cached on the server, generated by Redis using the SHA1 algorithm. With this unique ID, the client can repeatedly execute the script using the EVALSHA command.

We know Redis has an incrby command for incrementing integers but lacks a command for multiplication.

incrby key value  ==> $key = $key + value
mulby key value ==> $key = $key * value

Here’s how to use SCRIPT LOAD and EVALSHA to implement multiplication:

local curVal = redis.call("get", KEYS[1])
if curVal == false then
  curVal = 0
else
  curVal = tonumber(curVal)
end
curVal = curVal * tonumber(ARGV[1])
redis.call("set", KEYS[1], curVal)
return curVal

First, let’s condense the script into a single line, separating statements with semicolons:

local curVal = redis.call("get", KEYS[1]); if curVal == false then curVal = 0 else curVal = tonumber(curVal) end; curVal = curVal * tonumber(ARGV[1]); redis.call("set", KEYS[1], curVal); return curVal

Now load the script:

127.0.0.1:6379> script load 'local curVal = redis.call("get", KEYS[1]); if curVal == false then curVal = 0 else curVal = tonumber(curVal) end; curVal = curVal * tonumber(ARGV[1]); redis.call("set", KEYS[1], curVal); return curVal'
"be4f93d8a5379e5e5b768a74e77c8a4eb0434441"

The command line outputs a long string, which is the script's unique identifier. We can now execute the command using this ID:

127.0.0.1:6379> evalsha be4f93d8a5379e5e5b768a74e77c8a4eb0434441 1 notexistskey 5
(integer) 0
127.0.0.1:6379> evalsha be4f93d8a5379e5e5b768a74e77c8a4eb0434441 1 notexistskey 5
(integer) 0
127.0.0.1:6379> set foo 1
OK
127.0.0.1:6379> evalsha be4f93d8a5379e5e5b768a74e77c8a4eb0434441 1 foo 5
(integer) 5
127.0.0.1:6379> evalsha be4f93d8a5379e5e5b768a74e77c8a4eb0434441 1 foo 5
(integer) 25

Error Handling ​

The above script requires that the additional parameter passed must be an integer. What happens if we don’t pass an integer?

127.0.0.1:6379> evalsha be4f93d8a5379e5e5b768a74e77c8a4eb0434441 1 foo bar
(error) ERR Error running script (call to f_be4f93d8a5379e5e5b768a74e77c8a4eb0434441): @user_script:1: attempt to perform arithmetic on a nil value

The client outputs a generic error message from the server. Note that this is a dynamically thrown exception; Redis protects the main thread from crashing due to script errors, much like a try-catch statement surrounding the script. Errors encountered during Lua script execution are not reversible, just like Redis transactions. Therefore, caution is essential when writing Lua code to avoid conditions that could lead to incomplete script execution.

For those familiar with Lua, it’s worth noting that Lua does not natively provide try-catch statements. Instead, the alternative mechanism is the built-in pcall(f) function. pcall, which stands for protected call, runs function f in protected mode. If an error occurs, pcall returns false and the error message, while a regular call(f) will throw an exception on encountering an error. Redis execution of Lua scripts is wrapped in pcall calls.

Error Propagation ​

What happens if an error occurs in a redis.call function? Let’s look at another example:

127.0.0.1:6379> hset foo x 1 y 2
(integer) 2
127.0.0.1:6379> eval 'return redis.call("incr", "foo")' 0
(error) ERR Error running script (call to f_8727c9c34a61783916ca488b366c475cb3a446cc): @user_script:1: WRONGTYPE Operation against a key holding the wrong kind of value

The client again receives a generic error message instead of the specific WRONGTYPE error that the incr command would normally return. Internally, when redis.call encounters an error, it throws an exception, and the outer pcall catches this exception and returns a generic error to the client. If we replace the call with pcall, the result will differ, allowing specific error messages to be returned.

127.0.0.1:6379> eval 'return redis.pcall("incr", "foo")' 0
(error) WRONGTYPE Operation against a key holding the wrong kind of value

What to Do About Infinite Loops in Scripts ​

Redis executes commands in a single-threaded manner, meaning that if a Lua script contains an infinite loop, Redis could indeed hang. To address this, Redis provides the SCRIPT KILL command, which can dynamically terminate a Lua script that exceeds a specified timeout during execution. However, there’s an important caveat: SCRIPT KILL cannot be used if the currently executing script has modified Redis’s internal data state, as it would violate the atomicity of script execution.

For instance, if you run a script like this:

127.0.0.1:6379> eval 'while(true) do print("hello") end' 0

You will notice Redis becomes unresponsive, continually outputting "hello" in the logs. To terminate this, you’ll need to open a new redis-cli instance to execute:

127.0.0.1:6379> script kill
OK

Looking back at the earlier eval command, you’ll see:

127.0.0.1:6379> eval 'while(true) do print("hello") end' 0
(error) ERR Error running script (call to f_d395649372f578b1a0d3a1dc1b2389717cadf403): @user_script:1: Script killed by user with SCRIPT KILL...

Understanding the Mechanism of SCRIPT KILL ​

Now, let's delve into how SCRIPT KILL works. Lua’s scripting engine provides hooks that allow certain code to run during the execution of instructions. For example, Redis sets a hook to check the execution count every N instructions.

void evalGenericCommand(client *c, int evalsha) {
  ...
  lua_sethook(lua, luaMaskCountHook, LUA_MASKCOUNT, 100000);
  ...
}

In the hook function, Redis can process client requests when it detects that a Lua script might be running too long, with a default timeout of 5 seconds. This explains why you can still execute SCRIPT KILL while the script is seemingly hanging, as the engine periodically checks for client commands.

Thought Experiment ​

In the section on delayed queues, we used zrangebyscore and zdel commands to compete for tasks in a delayed queue. Using the return value of zdel, clients could determine who acquired the task. However, if a client misses out, it can be frustrating. Implementing this logic in Lua could atomically execute both commands, eliminating the issue of losing the task at the last moment.

Note: If you’re unfamiliar with Lua, it’s a simple language to learn, though not instantaneously. For further understanding, look for online tutorials specific to Lua.