What is an "instantiation cycle"? Why does this recursive generic function fail to compile?

func Nest[T any](n int) {
    if n > 0 {
        Nest[[]T](n - 1)
    }
}

func main() { Nest[int](3) }

Result: compile error instantiation cycle: T instantiated as []T.

Go instantiates generic code at compile time. Nest[int] needs Nest[[]int], which needs Nest[[][]int], and so on without end. The compiler cannot know that n bounds the recursion at run time, so it rejects any instantiation graph in which a type parameter grows. The same rule applies to types: type Node[T any] struct{ next *Node[[]T] } is rejected.

Recursion with the same or a fixed type argument is fine:

type TreeNode[T any] struct {
    Val         T
    Left, Right *TreeNode[T]
}

func Depth[T any](n *TreeNode[T]) int {
    if n == nil {
        return 0
    }
    return 1 + max(Depth(n.Left), Depth(n.Right)) // T stays T
}

Languages that box everything (Java) or instantiate lazily at run time (C# with a JIT) accept polymorphic recursion like this. Go's stenciling model cannot. It is a good follow-up to the GC shape question.