Known length slice initialization speed – Golang Benchmark Wednesday

I stumbled over the hint, that it is better for performance if you initialize your slices with a dedicated length and capacity if you know it. Sounds as it would make sense, but I wouldn't be me if I just accept that without testing that hypothesis.

An example that I am using in real life is for creating a slice of ids for querying a database later on with that ids. Iterating over the original data structure (in my case a 'map[string]SimonsStruct{Id int, MORE FIELDS}') and copying the ids out.

Normally I used 'make([]int,0)' (len == 0 & cap == 0), so let's see if that would be faster with initializing the slice directly with it the right capacity and length.

Keep in mind the tests only work if you know the size of the target slice upfront. If not, sadly this Benchmark Tuesday will not help you.

Benchmark Code

Bad: Initialize slice empty even if you know the target size

const size int //See size values benchmarked later in table
func BenchmarkEmptyInit(b *testing.B) {
	for n := 0; n < b.N; n++ {
		data := make([]int,0)
		for k:=0;k<size;k++{
			data = append(data,k)
		}
	}
}

Best for big size: Initialize slice with known capacity and add data with append

const size int //See size values benchmarked later in table
func BenchmarkKnownAppend(b *testing.B) {
	for n := 0; n < b.N; n++ {
		data := make([]int,0,size)
		for k:=0;k<size;k++{
			data = append(data,k)
		}
	}
}

Best for small & medium size: Initialize slice with known capacity & length and add data with direct access

const size int //See size values benchmarked later in table
func BenchmarkKnownDirectAccess(b *testing.B) {
	for n := 0; n < b.N; n++ {
		data := make([]int,size,size)
		for k:=0;k<size;k++{
			data[k] = k
		}
	}
}

Results

The table shows the time it took for every example to init all its elements (only measured inside the benchmark loop) Have an eye on the unit! (ns/ms/s)

#Elements (size)	EmptyInit	KnownAppend	KnownDirectAccess
1	31.00 ns	1.52 ns	0.72 ns
100	852 ns	81.4 ns	59.1 ns
100 000	1.11 ms	0.22 ms	0.20 ms
1 000 000	10.76 ms	3.13 ms	3.14 ms
100 000 000	2.48 s	0.21 s	0.22 s
300 000 000	6.79 s	0.90 s	0.95 s

Interpretation

That initializing the slice with len & capacity 0 would be the worst was obvious, but I am still surprised that the append approach outperforms the direct access for bigger sizes.

But after tinkering about it total makes sense. The direct access approach needs to write every entry twice:

1) Initializing the whole array with its 'nil' value (in our case int with '0') 2) Writing the actual value into that slice

Step 1) is not needed with the append approach, as we just reserve a memory location but the previous values stay there until we write them in step 2. For bigger slices this setup overhead outweighs the performance benefit of direct access. This will be even more significant if the values in the slice are not only simple int but even bigger (e.g. struct with a lot of fields), as then the setup will have to initialize even more 'nil' values.

Conclusion

The hint I found only was right: If you know the size of your target slice always initialize it with that size as capacity. For medium & small size slices use the direct access approach. For very big slices use append.

Thanks for reading and see you next week!

You got any feedback? Would love to answer it on HackerNews

p.S. There is a RSS Feed