🛒 Analogy
A supermarket opens extra checkout lanes when the queue grows (fan-out) — shoppers split across cashiers and get through faster. At the end of the day, every lane’s takings are merged into one ledger (fan-in). More lanes, more throughput; the receipts just aren’t in arrival order anymore.
The problem
One stage of your pipeline is the bottleneck — say a CPU-heavy transform. Running it single-file wastes the other cores. Fan-out runs several copies of that stage reading the same input; fan-in merges their outputs back into one channel so the rest of the pipeline is unchanged.
Structure
graph LR
IN["input chan"] --> W1["worker 1"]
IN --> W2["worker 2"]
IN --> W3["worker 3"]
W1 --> FI["fan-in<br/>merge"]
W2 --> FI
W3 --> FI
FI --> OUT["output chan"]
Idiomatic Go
Three workers read the same input channel (fan-out); fanIn merges their outputs with a WaitGroup that closes the merged channel once all are done. Edit and Run:
package main
import (
"fmt"
"sort"
"sync"
)
func gen(nums ...int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for _, n := range nums {
out <- n
}
}()
return out
}
// a stage that does some "expensive" work
func worker(in <-chan int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for n := range in {
out <- n * n
}
}()
return out
}
// fanIn merges several channels into one.
func fanIn(chans ...<-chan int) <-chan int {
out := make(chan int)
var wg sync.WaitGroup
wg.Add(len(chans))
for _, c := range chans {
go func(c <-chan int) {
defer wg.Done()
for v := range c {
out <- v
}
}(c)
}
go func() { wg.Wait(); close(out) }() // close once all sources drain
return out
}
func main() {
in := gen(1, 2, 3, 4, 5, 6, 7, 8)
// fan-out: three workers share one input channel
w1, w2, w3 := worker(in), worker(in), worker(in)
// fan-in: merge their results
var results []int
for v := range fanIn(w1, w2, w3) {
results = append(results, v)
}
sort.Ints(results) // order is not guaranteed with fan-out
fmt.Println(results)
}
🐹 Two things that make it correct
Many readers, one channel: several goroutines can receive from the same channel safely — Go hands each value to exactly one of them. Close-after-all: the merged channel must be closed only once every source goroutine has finished, which is exactly what the wg.Wait()-then-close goroutine guarantees. Closing too early would panic a still-sending worker.
Fan-out vs Worker Pool
They’re close cousins. Fan-out spins up a goroutine per “copy” of a stage; a Worker Pool keeps a fixed number of long-lived workers pulling from a queue. Use fan-out to parallelize a pipeline stage; use a pool when you need to bound how many run at once (see also Semaphore and errgroup.SetLimit).
Pitfalls
⚠️ Don't fan out unbounded over a huge input
Three workers is deliberate. Fanning out one-goroutine-per-item over a million inputs can exhaust memory or thrash the scheduler. When the input is large or the work hits a limited resource (DB, API), bound the parallelism with a worker pool or a semaphore.
When to use it — and when not
✅ Reach for it when
- A pipeline stage is slow or CPU-bound and can be parallelized across cores.
- Work items are independent and order doesn't matter (or can be restored).
- You want to scale throughput by adding more workers to one stage.
⛔ Think twice when
- Strict ordering is required and you can't re-sort afterward.
- The work is tiny — channel and goroutine overhead outweighs the gain.
- Work is I/O-bound with a resource cap — a bounded worker pool or semaphore fits better.
Related patterns
Process a stream of data through a series of stages connected by channels, where each stage is a goroutine.
CONCURRENCY Worker PoolBound concurrency by feeding jobs to a fixed number of long-lived worker goroutines.
CONCURRENCY errgroupRun a group of goroutines, wait for them all, capture the first error, and cancel the rest automatically.
CONCURRENCY SemaphoreLimit how many goroutines may run a section of code (or hold a resource) at the same time.
Check your understanding
Score: 0 / 31. What is 'fan-out'?
Multiple goroutines can safely receive from one channel; each grabs the next item, spreading the work — that's fan-out.
2. Why does fan-in need a sync.WaitGroup?
Each source feeds the merged channel in its own goroutine; a WaitGroup lets a closer goroutine wait for all of them before closing out, so the consumer's range ends cleanly.
3. What property do you lose when you fan a stage out?
Parallel workers finish at different times, so output order is nondeterministic. If you need order, attach an index and sort, or don't fan out.