Walk through how you would investigate a service whose p99 latency doubled after a deploy.
Question 402HardGo 1.22 to 1.25
A structured answer matters more than any one tool:
- Correlate metrics: CPU, RSS, goroutine count, GC cycles/sec and GC CPU fraction (
runtime/metrics), request rate, downstream latency. Is it CPU-bound, GC-bound, lock-bound, or waiting on I/O? - Diff CPU profiles old vs new build under similar load:
go tool pprof -diff_base=old.pprof new.pprof. Look for new hot functions — often an accidentalregexp.MustCompilein a handler, JSON reflection, or logging at debug level. - Allocations:
-sample_index=alloc_spacediff. A new allocation in the hot path increases GC frequency, and GC assists add latency directly to request goroutines. - Contention: enable the mutex and block profiles — a new global lock or a shared
sync.Map/cache under write load shows up here, not in CPU. - Execution trace for a few seconds (or
trace.FlightRecordertriggered on a slow request): see scheduler latency, STW pauses, goroutines waiting on the network, poor parallelism. - Reproduce in a benchmark, fix, and prove it with
benchstatand a canary.
// Handy: tag profile samples by endpoint so pprof can filter them.
func withLabels(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
pprof.Do(r.Context(), pprof.Labels("route", r.Pattern), func(ctx context.Context) {
next.ServeHTTP(w, r.WithContext(ctx))
})
})
}
// (runtime/pprof) then: go tool pprof -tagfocus=route=/api/search cpu.pprof
Interviewer is looking for: measure before changing anything, knowing which profile answers which question, and that on-CPU profiles alone can't explain wait time. Also check the boring causes: GOMAXPROCS vs container CPU quota (throttling), changed timeouts/retries, or a dependency upgrade.
More on Performance, Profiling & Testing
- Q400How do you separate unit tests from slow integration tests, and what are golden files?
- Q401What are Example functions, and how are they verified?
- Q403How do you write a test that fails if a hot path starts allocating? Explain testing.AllocsPerRun and its caveats.
- Q404How do you benchmark concurrent code with b.RunParallel, and why does sync.RWMutex often fail to scale for read-heavy workloads?
- Q405Go 1.24 replaced the built-in map implementation with Swiss tables. What changed, and what does it mean for performance-sensitive code?
- Q406Why do pointers make the GC more expensive, and how can data layout, unique.Handle, and weak pointers reduce memory and GC cost?