slice pitfalls

• append

A slice is a reference to an underlying array. If two slices share the underlying array, changing one slice MAY affects another one.

func printSlice(s []int) {
	fmt.Printf("len=%d, cap=%d, s=%v\n", len(s), cap(s), s)
}

func main() {
	s1 := []int{1}
	s2 := s1
	printSlice(s1) // len=1, cap=1, s=[1]

	s1 = append(s1, 2)
	s2 = append(s2, 5) // <--- it will not change s1
	printSlice(s1)     // len=2, cap=2, s=[1 2]
	printSlice(s2)     // len=2, cap=2, s=[1 5]

	s1 = append(s1, 3, 4, 5, 6, 7)
	s2 = s1
	printSlice(s1) // len=7, cap=8, s=[1 2 3 4 5 6 7]
	printSlice(s2) // len=7, cap=8, s=[1 2 3 4 5 6 7]

	s1 = append(s1, 8)
	printSlice(s1)     // len=8, cap=8, s=[1 2 3 4 5 6 7 8]
	printSlice(s2)     // len=7, cap=8, s=[1 2 3 4 5 6 7]

	s2 = append(s2, 9) // <---- it will change s1
	printSlice(s1)     // len=8, cap=8, s=[1 2 3 4 5 6 7 9]
	printSlice(s2)     // len=8, cap=8, s=[1 2 3 4 5 6 7 9]

}

The build-in function append will append values to the slice and may realloc underlying array for that:

If the capacity of s is not large enough to fit the additional values, append allocates a new, sufficiently large underlying array that fits both the existing slice elements and the additional values. Otherwise, append re-uses the underlying array.

The KEY POINT is that, the new array may have some “extra” space for furture use. If the user only appends one value to the original slice, it will re-use the underlying array, i.e., use that “extra” space. And then, if the user only appends one value to the “copied” slice, it will also use that “extra” space, i.e., new value overwrites that space.

Another example is the Record struct in standard pkg slog. A Record holds information about a log event. Copies of a Record share state. As its doc said:

Do not modify a Record after handing out a copy to it. Call NewRecord to create a new Record. Use Record.Clone to create a copy with no shared state, and then the original record and the clone can both be modified without interfering with each other.

The following code shows that, the assignment operation makes a copy of original record and modifying one record will affect another.

func main() {

	/* The following code uses assignment to create a copy with
	 * shared state, and then the original record and the copy
	 * may interfer with each other.
	 */
	r1 := slog.NewRecord(time.Now(), slog.LevelDebug, "hello", 0)
	r1.Add("s1", 1, "s2", 2, "s3", 3, "s4", 4, "s5", 5)
	r1.Add("b1", 1)
	r1.Add("b2", 2, "b3", 3, "b4", 4, "b5", 5, "b6", 6, "b7", 7)
	r2 := r1

	s1 := []slog.Attr{}
	s2 := []slog.Attr{}
	gets1 := func(a slog.Attr) bool {
		s1 = append(s1, a)
		return true
	}
	gets2 := func(a slog.Attr) bool {
		s2 = append(s2, a)
		return true
	}
	r1.Attrs(gets1)
	r2.Attrs(gets2)

	fmt.Printf("r1 last attr: %v\n", s1[len(s1)-1]) // r1 last attr: b7=7
	fmt.Printf("r2 last attr: %v\n", s2[len(s2)-1]) // r2 last attr: b7=7

	r1.Add("b8", 8)
	s1 = []slog.Attr{}
	s2 = []slog.Attr{}
	r1.Attrs(gets1)
	r2.Attrs(gets2)
	fmt.Printf("r1 last attr: %v\n", s1[len(s1)-1]) // r1 last attr: b8=8
	fmt.Printf("r2 last attr: %v\n", s2[len(s2)-1]) // r2 last attr: b7=7

	r2.Add("b8", 99999) // <--- it will change r1
	s1 = []slog.Attr{}
	s2 = []slog.Attr{}
	r1.Attrs(gets1)
	r2.Attrs(gets2)
	fmt.Printf("r1 last attr: %v\n", s1[len(s1)-1]) // r1 last attr: b8=99999
	fmt.Printf("r2 last attr: %v\n", s2[len(s2)-1]) // r2 last attr: b8=99999

	/* The following code uses Record.Clone to create a copy with
	 * no shared state, and then the original record and the clone
	 * can both be modified without interfering with each other.
	 */
	r1 = slog.NewRecord(time.Now(), slog.LevelDebug, "hello", 0)
	r1.Add("s1", 1, "s2", 2, "s3", 3, "s4", 4, "s5", 5)
	r1.Add("b1", 1)
	r1.Add("b2", 2, "b3", 3, "b4", 4, "b5", 5, "b6", 6, "b7", 7)
	r2 = r1.Clone()

	r1.Add("b8", 8)
	r2.Add("b8", 99999) // <--- it will not change r1

	s1 = []slog.Attr{}
	s2 = []slog.Attr{}
	r1.Attrs(gets1)
	r2.Attrs(gets2)
	fmt.Printf("r1 last attr: %v\n", s1[len(s1)-1]) // r1 last attr: b8=8
	fmt.Printf("r2 last attr: %v\n", s2[len(s2)-1]) // r2 last attr: b8=99999
}

If we dig into the details of Record.Add() method, we will find that append is used for assembling the slice.

type Record struct {
	Time time.Time
	Message string
	Level Level
	PC uintptr

	// It holds the start of the list of Attrs.
	front [nAttrsInline]Attr

	// The number of Attrs in front.
	nFront int

	// The list of Attrs except for those in front.
	back []Attr
}

func (r *Record) Add(args ...any) {
	// ..
	// other codes

	if r.back == nil {
		r.back = make([]Attr, 0, countAttrs(args)+1)
	}
	r.back = append(r.back, a)
}

If we dig into the details of Record.Clone() method, will find that the slice will be “re-sliced” and remove extra space in the underlying array. When the user adds a new attribute, append will allocate a new array to store data. Thus original slice and the clone will have distinct underlying arrays. It is the mechnism of “cloning”.

func (r Record) Clone() Record {
	r.back = slices.Clip(r.back) // prevent append from mutating shared array
	return r
}

//in pkg slices
// Clip removes unused capacity from the slice, returning s[:len(s):len(s)].
func Clip[S ~[]E, E any](s S) S {
	return s[:len(s):len(s)]
}