What is a cache stampede and how does singleflight solve it?

When a hot key expires, hundreds of concurrent requests all miss and hit the database simultaneously (thundering herd). golang.org/x/sync/singleflight deduplicates concurrent calls with the same key: the first caller executes the function, the rest block and receive the same result.

type UserService struct {
	cache *Cache[string, *User]
	group singleflight.Group
	db    *sql.DB
}

func (s *UserService) Get(ctx context.Context, id string) (*User, error) {
	if u, ok := s.cache.Get(id); ok {
		return u, nil
	}
	v, err, shared := s.group.Do(id, func() (any, error) {
		// Detach from a single caller's cancellation: others share this result.
		ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 3*time.Second)
		defer cancel()
		u, err := loadUser(ctx, s.db, id)
		if err == nil {
			s.cache.Set(id, u)
		}
		return u, err
	})
	_ = shared // true if the result was delivered to multiple callers
	if err != nil {
		return nil, err
	}
	return v.(*User), nil
}

Gotchas: errors are also shared (one failure fails all waiters); if the first caller's context is canceled, everyone fails — hence WithoutCancel; use DoChan to let each waiter respect its own deadline; and the returned value is shared, so treat it as immutable. Other mitigations: jittered TTLs, early probabilistic refresh, serving stale while revalidating.