Implement a sharded concurrent map (N shards with separate mutexes). When does it beat sync.Map and a single RWMutex?
Hash each key to one of N shards. Each shard has its own RWMutex and a plain map, so operations on different shards never contend. Go 1.24's maphash.Comparable hashes any comparable key with a random per-process seed, which also protects against HashDoS.
type shard[K comparable, V any] struct {
mu sync.RWMutex
m map[K]V
_ [32]byte // pad to a 64-byte cache line to avoid false sharing
}
type ShardedMap[K comparable, V any] struct {
shards []shard[K, V]
seed maphash.Seed
}
func NewShardedMap[K comparable, V any](n int) *ShardedMap[K, V] {
s := &ShardedMap[K, V]{shards: make([]shard[K, V], n), seed: maphash.MakeSeed()}
for i := range s.shards {
s.shards[i].m = make(map[K]V)
}
return s
}
func (s *ShardedMap[K, V]) shardFor(k K) *shard[K, V] {
return &s.shards[maphash.Comparable(s.seed, k)%uint64(len(s.shards))]
}
func (s *ShardedMap[K, V]) Get(k K) (V, bool) {
sh := s.shardFor(k)
sh.mu.RLock()
defer sh.mu.RUnlock()
v, ok := sh.m[k]
return v, ok
}
func (s *ShardedMap[K, V]) Set(k K, v V) {
sh := s.shardFor(k)
sh.mu.Lock()
sh.m[k] = v
sh.mu.Unlock()
}
// Update performs an atomic read-modify-write on one key.
func (s *ShardedMap[K, V]) Update(k K, fn func(old V, ok bool) V) {
sh := s.shardFor(k)
sh.mu.Lock()
defer sh.mu.Unlock()
old, ok := sh.m[k]
sh.m[k] = fn(old, ok)
}
When the sharded map wins: write-heavy or mixed workloads with many cores, the need for atomic compound operations like Update (counters, get-or-create with side effects), cheap Len, and typed values.
Single RWMutex: simplest and fine at low contention. Under heavy read load on many cores, the shared reader counter becomes a cache-line hotspot, so even an "RLock-only" workload stops scaling.
sync.Map: documented as optimized for two cases: keys written once and read many times (caches), and goroutines working on disjoint key sets. Since Go 1.24 it is backed by a concurrent hash-trie, which greatly improved mixed-write performance. Benchmark before you assume sharding wins. It still stores any, which costs boxing, and it has no atomic update beyond CompareAndSwap and LoadOrStore.
Gotchas: a Range across shards is not a consistent snapshot. Using a power-of-two shard count lets you replace % with a mask. Hot keys still serialize on a single shard.
More on Classic Concurrency Coding Problems
- Q526Count word frequencies in a large set of files concurrently and return the top K words. How do you merge per-worker maps efficiently?
- Q527Implement a debouncer and a throttler for a stream of events using time.Timer and channels.
- Q529Implement a readers-writer lock that prefers writers using only channels or sync.Mutex. Why is it hard to get right?
- Q530Implement a retry-with-timeout helper that runs a function in a goroutine and returns early when the per-attempt or overall deadline expires, without leaking goroutines.
- Q531Implement a heartbeat/watchdog: a worker must signal liveness every T, or a supervisor restarts it.