Fan-Out

The purpose of the Fan-Out design pattern is to divide the workload among multiple concurrent processors, or workers, efficiently. To grasp this concept better, let’s revisit the previous section but consider a specific problem: what if there’s a significant disparity in the amount of work at different stages in our pipeline?

For instance, fetching HTML content might take much longer than parsing it. In such cases, it makes sense to distribute the heavy lifting across multiple coroutines. In the previous example, each channel had only one coroutine reading from it. However, it’s possible for multiple coroutines to consume from a single channel, effectively sharing the workload.

To simplify the problem we’re about to discuss, let’s assume we have only one coroutine producing some results:

fun CoroutineScope.generateWork() = produce {
    for (i in 1..10_000) {
        send("page$i")
    }
    close()
}

And we’ll create a function that generates a new coroutine responsible for reading those results:

fun CoroutineScope.doWork(
    id: Int,
    channel: ReceiveChannel<String>
) = launch(Dispatchers.Default) {
    for (p in channel) {
        println("Worker $id processed $p")
    }
}

This function generates a coroutine that runs on the default dispatcher. Each coroutine listens to a channel and prints every message it receives to the console.

Now, let’s kick off our producer. Keep in mind that all the following code pieces should be wrapped in the runBlocking or a suspend main function, but for simplicity, we’ve omitted that part:

val workChannel = generateWork()

Next, we can create multiple workers that collaborate to distribute the work among themselves by reading from the same channel:

val workers = List(10) { id ->
    doWork(id, workChannel)
}

Now, let’s examine a portion of the program’s output:

... 
> Worker 4 processed page9994 
> Worker 8 processed page9993 
> Worker 3 processed page9992 
> Worker 6 processed page9987

Note that no two workers receive the same message, and the messages are not printed in the order they were sent.

Load balancing is a critical aspect of the Fan-Out design pattern. It ensures that the workload is evenly and efficiently distributed across the available resources, preventing situations where some workers are overloaded while others remain underutilized.

Kotlin channels inherently provide a level of fairness in load balancing. When multiple consumer coroutines are waiting to receive data from a channel, the channel distributes data fairly among them. Each consumer gets an opportunity to receive data in a round-robin fashion. This ensures that no single consumer is starved while others receive all the data.

Channels also offer mechanisms for backpressure handling. When data is produced faster than it can be consumed, channels can suspend producers until consumers are ready. This helps prevent overloading the system.

The Fan-Out design pattern enables efficient distribution of work across a number of coroutines, threads, and CPUs.

Next, we’ll take a look at associated design pattern that often complements Fan-Out.