Skip to content

Commit

Permalink
Backport performance fix regarding sys_membarrier
Browse files Browse the repository at this point in the history
* Cherry-picked from scylladb/seastar@4fc35ff
* Following Linux change on membarriers (serialization):
* torvalds/linux@944d5fe
* Usage of Scylla of membarriers seems not suitable.
* A global lock to serialize userland the call has been put.
* Let's figure out if it fix the x2 CPU consumption
* ref: https://criteo.atlassian.net/wiki/spaces/STORAGE/pages/3263432335/2024-10-24+Kernel+6.6+high+CPU+with+Scylla+on+io-2021
  • Loading branch information
David Amsallem committed Oct 31, 2024
1 parent c38c5e2 commit 445a543
Showing 1 changed file with 25 additions and 0 deletions.
25 changes: 25 additions & 0 deletions src/core/systemwide_memory_barrier.cc
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#include <seastar/core/systemwide_memory_barrier.hh>
#include <seastar/core/cacheline.hh>
#include <seastar/util/log.hh>
#include <seastar/util/defer.hh>
#include <sys/mman.h>
#include <unistd.h>
#include <cassert>
Expand Down Expand Up @@ -102,7 +103,31 @@ systemwide_memory_barrier() {
assert(r2 == 0);
}

struct alignas(cache_line_size) aligned_flag {
std::atomic<bool> flag;
bool try_lock() noexcept {
return !flag.exchange(true, std::memory_order_relaxed);
}
void unlock() noexcept {
flag.store(false, std::memory_order_relaxed);
}
};
static aligned_flag membarrier_lock;

bool try_systemwide_memory_barrier() {
// In 944d5fe50f3f, Linux started serializing membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED)
// calls. This means that if all reactors enter sleep mode at the same time, they will sleep
// on a kernel mutex while doing so. While they wait on the mutex, they cannot be woken.
//
// To fix this, only we serialize membarrier calls ourselves, but instead of sleeping, we just
// return to the reactor poll loop. If an event is ready, we will wake up immediately.
if (!membarrier_lock.try_lock()) {
return false;
}
auto unlock = defer([] () noexcept {
membarrier_lock.unlock();
});

if (try_native_membarrier()) {
return true;
}
Expand Down

0 comments on commit 445a543

Please sign in to comment.