How do you implement a semaphore in Go? Compare a buffered channel with golang.org/x/sync/semaphore.
Question 243MediumGo 1.22 to 1.25
Buffered channel as a counting semaphore, which is idiomatic and has no dependencies:
sem := make(chan struct{}, 10) // at most 10 in flight
var wg sync.WaitGroup
for _, job := range jobs {
sem <- struct{}{} // acquire (blocks at capacity)
wg.Go(func() {
defer func() { <-sem }() // release
process(job)
})
}
wg.Wait()
Acquiring before the go statement bounds the number of goroutines, not just the work, so you never have a million parked goroutines. The memory model's rule that the k-th receive happens-before the (k+C)-th send is what makes this correct.
Weighted semaphore, for when tasks have different costs or you need context-aware acquisition:
sem := semaphore.NewWeighted(512) // e.g. 512 MB budget
func run(ctx context.Context, j Job) error {
if err := sem.Acquire(ctx, j.MB); err != nil {
return err // ctx cancelled while waiting
}
defer sem.Release(j.MB)
return process(ctx, j)
}
semaphore.Weighted serves waiters in FIFO order, so a large request is not starved by small ones. It also has TryAcquire. Acquiring more than the total capacity blocks until the context is done.
A channel acquire can also be made cancellable with select { case sem <- struct{}{}: case <-ctx.Done(): }.
More on Concurrency Patterns & sync
- Q241Spot the goroutine leak.
- Q242How do you close a channel safely when there are multiple senders?
- Q244Explain errgroup: WithContext, SetLimit, TryGo. What are its semantics and gotchas?
- Q245You need to process 10 million records from a file with bounded memory and bounded parallelism. How do you design it?
- Q246How do you implement rate limiting in Go? Compare time.Ticker with golang.org/x/time/rate.
- Q247What problem does singleflight solve, and what are its gotchas?