Code review: what is wrong with returning a small sub-slice of a large buffer, and what does s[low:high:max] fix?
func header(path string) ([]byte, error) {
data, err := os.ReadFile(path) // e.g. 500 MB
if err != nil {
return nil, err
}
return data[:64], nil // keeps all 500 MB reachable
}
A sub-slice shares the original backing array. As long as the 64-byte slice is alive, the garbage collector cannot free the other 500 MB. Keep many of these (a cache of headers, for example) and memory grows without bound. Fix it by copying: return bytes.Clone(data[:64]), nil. You can also use slices.Clone, or append([]byte(nil), data[:64]...).
A full slice expression s[low:high:max] sets the capacity to max-low. It does not fix retention, because it still shares the array. It fixes aliasing instead: the next append on the result must reallocate, so it cannot overwrite the parent's elements past high.
base := []int{1, 2, 3, 4, 5}
view := base[1:3:3] // [2 3], cap 2
view = append(view, 99) // reallocates
fmt.Println(base) // [1 2 3 4 5], index 3 was not overwritten
Without the third index, base would become [1 2 3 99 5]. Strings behave the same way: strings.Clone exists so that a small substring does not keep a huge string alive.
More on Tricky Output & Code-Review Puzzles
- Q483What does this append puzzle print?
- Q484What does the caller see after calling modify?
- Q486Array vs slice in range: what prints?
- Q487Why doesn't this loop update the users?
- Q488Why is err != nil true here even though the function returned a nil pointer?
- Q489Why doesn't this compile, and why does Go refuse it?