BUG/MEDIUM: chunk: make sure to flush the trash pool before resizing

Late in 3.1 we've added an integrity check to make sure we didn't keep
trash objects allocated before resizing the trash with commit 0bfd36e
("MINOR: chunk: add a BUG_ON upon the next init_trash_buffer()"), but
it turns out that the counter that is being checked includes the number
of objects left in local thread caches. As such it can trigger despite
no object being allocated. This precisely happens when setting
tune.memory.hot-size to a few megabytes because some temporarily used
trash objects will remain in cache.

In order to address this, let's first flush the pool before running
the check. That was previously done by pool_destroy() but the check
had to be inserted before it. So now we first flush the trash pool,
then verify it's no longer used, and finally we can destroy it.

This needs to be backported to 3.1. Thanks to Christian Ruppert for
reporting this bug.

src/chunk.c

@@ -89,6 +89,13 @@ static void free_trash_buffers_per_thread()
 /* Initialize the trash buffers. It returns 0 if an error occurred. */
 int init_trash_buffers(int first)
 {
+	/* first, make sure we don't keep any trash in object in pools nor cache */
+	if (pool_head_trash) {
+		if (!(pool_debugging & POOL_DBG_NO_CACHE))
+			pool_evict_from_local_cache(pool_head_trash, 1);
+		pool_flush(pool_head_trash);
+	}
+
 	BUG_ON(!first && pool_used(pool_head_trash) > 0); /* we tried to keep a trash buffer after reinit the pool */
 	pool_destroy(pool_head_trash);
 	pool_head_trash = create_pool("trash",