Implement graceful shutdown for an HTTP server with background workers.
Question 248HardGo 1.22 to 1.25
func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
var workers sync.WaitGroup
jobs := make(chan Job, 100)
for range 4 {
workers.Go(func() {
for j := range jobs { // drains until jobs is closed
process(j)
}
})
}
// NOT the signal ctx: that would cancel in-flight requests the moment
// SIGTERM arrives, defeating the drain.
srv := &http.Server{
Addr: ":8080",
Handler: newRouter(jobs),
}
errCh := make(chan error, 1)
go func() { errCh <- srv.ListenAndServe() }()
select {
case err := <-errCh:
log.Fatalf("listen: %v", err)
case <-ctx.Done():
stop() // a second Ctrl-C now kills immediately
}
shutCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
if err := srv.Shutdown(shutCtx); err != nil { // stop accepting, drain in-flight
log.Printf("shutdown: %v", err)
}
close(jobs) // safe only if Shutdown returned nil (all handlers done)
workers.Wait() // finish queued work
log.Println("bye")
}
Ordering matters:
- Stop taking new work:
Shutdowncloses the listeners and waits for active requests. - Close the producer side (
jobs). - Drain the workers.
- Flush and close resources such as DB connections and tracers.
Gotchas:
- After
Shutdownis called,ListenAndServereturnshttp.ErrServerClosedimmediately. Shutdowndoes not wait for hijacked or WebSocket connections. UseRegisterOnShutdownfor those.- In Kubernetes, fail readiness first and sleep briefly so the endpoint is removed before you stop accepting.
- Always put a deadline on shutdown. If
Shutdowntimes out, some handlers may still be running, and one that enqueues onto the closedjobschannel panics. Either have handlers check a "shutting down" flag or context before sending, or exit without closingjobson timeout. - Do not set
BaseContextto the signal context. Request contexts would be cancelled on SIGTERM instead of being allowed to finish.
More on Concurrency Patterns & sync
- Q246How do you implement rate limiting in Go? Compare time.Ticker with golang.org/x/time/rate.
- Q247What problem does singleflight solve, and what are its gotchas?
- Q249Which newer context features matter for concurrent code: WithCancelCause, AfterFunc, WithoutCancel?
- Q250Is time.After in a select loop a leak? What changed in Go 1.23 timers?
- Q251What happens here, and when does the runtime NOT detect a deadlock?
- Q252Design a thread-safe loading cache. Why is naive double-checked locking with a plain flag wrong in Go?