Michal Charemza

Here's Queue, a small utility function to queue and run tasks with limited concurrency.

const Queue = (concurrency) => {
    var running = 0;
    const tasks = [];

    return async (task) => {
        tasks.push(task);
        if (running >= concurrency) return;

        ++running;
        while (tasks.length) {
            try {
                await tasks.shift()();
            } catch(err) {
                console.error(err);
            }
        }
        --running;
    }
}

You enqueue a task by passing it to the function returned by Queue.

// Create the queue with a concurrency limit
const concurrency = 4
const enqueue = Queue(concurrency);

// In this case, each task is an async upload function...
const upload = async (file) => {...};

// ... and each task uploads a file
const files = [...];
files.forEach((file) => {
    enqueue(async () => upload(file));
});

If there is available concurrency, the task will run [almost] immediately. Otherwise, it will wait until enough tasks ahead of it have completed for it to run within the concurrency limit.

Use case

A use for this is part of a file uploader that allows arbitrary numbers of uploads. Although browsers limit the number of connections, they also limit how many requests are queued: Chrome can throw net::ERR_INSUFFICIENT_RESOURCES if you attempt to queue too many. To avoid this, you can queue the uploads up front.

Features

• Can add tasks at any point after queue creation
• No recursion
• No polling
• Small API
• Small amount of code
• Leverages async/await syntax: no handling of promises or callbacks, either internally or in client code

That's it!