How do you find a memory leak in production with pprof?
Question 363MediumGo 1.22 to 1.25
import _ "net/http/pprof" // registers /debug/pprof/* on DefaultServeMux
func main() {
go func() { log.Println(http.ListenAndServe("localhost:6060", nil)) }()
// ...
}
Steps:
- Take two heap snapshots a few minutes apart:
curl -o a.pb.gz localhost:6060/debug/pprof/heap, wait, then takeb.pb.gz. - Diff them:
go tool pprof -sample_index=inuse_space -base a.pb.gz b.pb.gz, then usetop,list Func, or the-http=:8080flame graph. - Also check
/debug/pprof/goroutine, because leaked goroutines hold heap memory.
The heap profile has four views:
inuse_space/inuse_objects: memory still live as of the last GC, shown where it was allocated. Use these to find leaks.alloc_space/alloc_objects: everything allocated since the program started. Use these to find allocation churn and GC pressure.
Gotchas: the profile shows where memory was allocated, not who is still holding it. You have to work out the retaining path yourself; the viewcore tool can help with core dumps. The profile is sampled, by default one sample per 512 KB allocated (runtime.MemProfileRate). It does not show memory allocated by C, goroutine stacks, or overhead from the runtime or fragmentation. Those are why RSS can exceed what the profile shows, so compare with runtime/metrics. Continuous profiling tools such as Pyroscope or Parca make diffs over time easy.
More on Memory, GC & Runtime Internals
- Q361How do goroutine leaks happen, and how do you detect them?
- Q362Is
time.Afterin a loop a memory leak? What changed in Go 1.23? - Q364How does GOMAXPROCS interact with containers, and how does it affect GC?
- Q365What are the runtime representations of Go's built-in types? What does this print on 64-bit?
- Q366What is bounds-check elimination and how do you help the compiler do it?
- Q367What is profile-guided optimisation (PGO) in Go and what does it actually change?