Go

What does GOTRACEBACK control, and how do you get a core dump from a crashing Go program and analyze it?

Question 565HardGo 1.22 to 1.25

GOTRACEBACK controls how much stack trace the runtime prints on an unrecovered panic or a fatal error:

  • none: no goroutine traces.
  • single (the default): only the goroutine that failed.
  • all: every user goroutine.
  • system: all plus runtime frames and runtime goroutines.
  • crash: like system, then the process aborts with SIGABRT so the OS writes a core dump.
  • wer (Windows only): like crash, and also reports to Windows Error Reporting.

Programs can raise the level with debug.SetTraceback("all"), but cannot lower it below the environment setting.

ulimit -c unlimited                    # allow core files (check /proc/sys/kernel/core_pattern)
GOTRACEBACK=crash ./app                # a fatal panic now dumps core
dlv core ./app ./core.12345            # analyze it with the SAME binary
(dlv) goroutines
(dlv) goroutine 1 bt
(dlv) frame 3 locals

You can also snapshot a live process without killing it with gcore <pid> (gdb) and then open the result with dlv core.

Since Go 1.23, debug.SetCrashOutput(f, debug.CrashOptions{}) also copies fatal crash output to a file. A common pattern uses it to feed a supervisor/child process that reports crashes to your error tracker.

f, err := os.Create("/var/log/app/crash.log")
if err == nil {
	_ = debug.SetCrashOutput(f, debug.CrashOptions{})
}

Gotchas:

  • A core dump contains the whole heap, including secrets. Handle it as sensitive data.
  • Stripped binaries (-ldflags=-s) lose symbols. Keep an unstripped copy of every release for post-mortems.
  • In containers, core_pattern belongs to the host kernel.
  • Some failures cannot be recovered by recover and always crash: concurrent map writes, stack overflow, running out of memory, and deadlock ("all goroutines are asleep").

More on Observability, Debugging & Production Operations

All 14 Observability, Debugging & Production Operations questions