Apica’s observability stack is predominantly written in Go. As a team, we love the simplicity of the language and the tooling that comes with it. Go has a built-in garbage collector, making it easy to write programs without worrying about memory for the most part. However, as you would expect, nothing comes for free.
There are numerous scenarios where relying purely on language and runtime capabilities without considering what happens beneath the covers lands you in trouble. This article explores a model that can significantly improve overall performance and memory usage while reducing the amount of runtime the system spends on garbage collection. Go’s sync package provides an implementation of the Pool type. Here’s how Go describes the Pool type in their documentation:
A Pool is a set of temporary objects that may be individually saved and retrieved.
Any item stored in the Pool may be removed automatically at any time without notification. If the Pool holds the only reference when this happens, the item might be deallocated.
https://pkg.go.dev/sync
Let’s see how this works with a real-world example. We used sync.Pool for some of our frequently allocated objects. In cases where reuse of the objects was obvious (such as when log data comes in), we store data in an incoming object and persist it to the data store. This scenario is a great candidate for optimization as in steady-state, there should be a fixed number of required objects, which would be proportional to the ingest rate.
Let’s try out a Pool implementation. To visualize the pool’s effectiveness, we track the allocation using Prometheus counters. The Prometheus counter PoolStatsCountCollector tracks pool usage with the labels get, put, and new. “get” Tracks requests for the object, “put” tracks objects returned to the pool, and “new” tracks how many allocations happened when an object “get” did not find a pre-allocated object from the pool.
var freeLocalReceiverPartitionPGPool = sync.Pool{
New: func() interface{} {
client.PoolStatsCountCollector.WithLabelValues("LocalReceiverPartitionPG", "new").Inc()
return new(LocalReceiverPartitionPG)
},
}
func GetLocalReceiverPartitionPGFromPool() *LocalReceiverPartitionPG {
client.PoolStatsCountCollector.WithLabelValues("LocalReceiverPartitionPG", "get").Inc()
return freeLocalReceiverPartitionPGPool.Get().(*LocalReceiverPartitionPG)
}
func FreeLocalReceiverPartitionPG(lrpg *LocalReceiverPartitionPG) {
lrpg.Reset()
client.PoolStatsCountCollector.WithLabelValues("LocalReceiverPartitionPG", "put").Inc()
freeLocalReceiverPartitionPGPool.Put(lrpg)
}We can now plot this in Apica’s UI. Pool usage statistics are vital in monitoring various subsystems’ memory usage. A built-in Pool usage widget for the Prometheus Metric is available in all Apica deployments. The following image shows how Apica’s UI visualizes pool usage:
In the visualization above, we can see the number of times the application requested an object. In this scenario, the allocations are roughly 12% (45,429) of the actual object access (363,177), which amounts to an 87% reduction in heap allocations! The allocations exhibit a phenomenal improvement with reduced heap usage, improved latency, and drastically reduced garbage collection CPU cycles – all made possible by using sync.Pool.
Pool’s purpose is to cache allocated, unused items for later reuse, thereby relieving pressure on the garbage collector. Essentially, Pool makes building efficient and thread-safe free lists easy. However, it may not be suitable for all free lists.
