Michal Charemza

In this post, I present an implementation of an asyncio read/write lock [also known as a shared/exclusive lock].

Since this post was written, the lower-level class, FifoLock, has been released separately.

What is a lock?

When you have concurrent tasks, there may be parts of the code that have to be protected from being run by multiple tasks concurrently. A typical example is non-atomic reading from a file, concurrent with non-atomic writing to that same file. If a reading task reads the file half-way through a writing task, the reader may get a corrupt file: seeing some of the file before the write, and some of the file after.

A way to prevent this is to use what's called a lock. Just before such code that needs to be protected, each task attempts to acquire the lock, and just after the code, it releases the lock so other task can acquire it. If a task cannot acquire the lock, it will wait until it can. [There are other possibilities if a task can't acquire the lock, but for simplicity, we just assume it waits.]

Exclusive lock

There are different sorts of lock, but a typical example is an exclusive lock. Only one task can acquire this lock, all others must wait. Using an exclusive lock is a way to prevent corruption when multiple tasks are reading from/writing to a file.

Python's standard library comes with an exclusive lock. This can be used, for example:

import asyncio
lock = asyncio.Lock()

# ...
async def locked():
    # ...
    async with lock:
       # only one task can be here at any given time

Note that a lock in this context isn't really analogous to a lock on a door. An exclusive lock is better compared to a talking stick or a talking spoon, in that only the person who has acquired the talking stick can talk, others must wait until they acquire it. For other sorts of locks, it's a bit more complex and the analogy can break down.

Read/write lock

Using an exclusive lock is often enough to prevent corruption, but it may block tasks that would otherwise be safe to proceed. For example, concurrent readers on a file is often safe, and only if there is a writer should other tasks be blocked. To allow this, we use a different sort of lock: a read/write lock.

A read/write lock is made of of two parts: a read lock and a write lock. The read lock can be acquired by multiple tasks concurrently, but only as long as no task holds the write lock; and a write lock can only be acquired if no tasks hold the read lock and no task holds the write lock.

This can be summarised in the following table, showing what parts of the lock can be held concurrently by two tasks, A and B.

Note that this table isn't quite enough to completely define how the lock behaves. When there are multiple tasks waiting to acquire the lock, some algorithm must be implemented to decide which tasks can be acquired. The implementation presented here allows tasks to proceed in a first-come-first-served order: a queue of tasks waiting to acquire the locks is maintained.

Note also that this type of lock isn't limited to reading or writing, hence this sort of lock is often called a shared/exclusive lock. However, it is so often used in the context of reading and writing, it is convenient and typical to call this a read/write lock [or similar].

Read/write lock: usage

The usage is very similar to the built-in exclusive lock, except that the lock object is callable with a required argument, an object representing which of the Read and Write lock modes is desired.

from asyncio_read_write_lock import FifoLock, Read, Write

lock = FifoLock()

# ...
async def read():
    # ...
    async with lock(Read):
       # multiple tasks can be here, but only if there are no writers

# ...
async def write():
    # ...
    async with lock(Write):
       # only one task can be here at any given time, and only if no readers

The lock class FifoLock is general, and can support more complex locks than read/write, although here we just use it as a read/write lock.

Read/write lock: comparison with aiorwlock.

There is already a read/write lock available for asyncio at aiorwlock. There are differences, which may be important for your use-case.

• This implementation has no support for Python versions less than 3.7, while aiorwlock appears to support earlier versions. This lack of support is a feature: it allows for simpler code.
• Python 3.7's contextlib.asynccontextmanager is used internally. It allows for very concise creation of an asyncio context manager, without sacrificing flexibility or ability to handle errors.
• This implementation has no support for acquiring a write lock already held by the current task: attempting to do so would result in a deadlock. My use cases do not require this, and my instinct is to try to write client code so that this is not required. This would make the code of the lock more complex, and I suspect in a lot of cases using this would make client code more difficult to reason about.
• It appears that the algorithm aiorwlock uses to acquire a lock is O(n) with respect to the number of tasks already holding a lock. This implementation is O(1) with respect to tasks holding a lock. Of course, this fact alone does not mean that this implementation is better or faster in practical terms.
• Locks are acquired on a first-come-first-served basis, compared to aiorwlock giving preference to waiting writers: giving preference to waiting writers can starve the readers if the write lock is continually requested. Which of these is more appropriate depends on your use case, but my instinct is that unless you are sure there will will never be enough writers to starve the readers for any meaningful length of time, it's better to avoid such a bias.

Read/write lock: implementation

The implementation is below.