Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macos: basic support for darwin builds #20

Draft
wants to merge 6 commits into
base: mariadb-4.x
Choose a base branch
from
Draft

macos: basic support for darwin builds #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
15693ae
build(macos/other): librt is not needed for glibc >= 2.17
sitano Jul 31, 2024
aa6f9ab
macos: sym map is not supported by llvm clang
sitano Jul 31, 2024
b52c519
macos: tcp sock info support
sitano Jul 31, 2024
78dfe02
macos: apple is almost unix, support pthreads in gu
sitano Jul 31, 2024
5afb256
macos: darwin has time impl
sitano Jul 31, 2024
1abe2d1
macos: fix missing pthread mutex init after calloc
sitano Jul 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions cmake/check.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ if (GALERA_HAVE_SUBUNIT_LIB)
list(APPEND GALERA_UNIT_TEST_LIBS "${GALERA_HAVE_SUBUNIT_LIB}")
endif()

# needed on old systems with glibc < 2.17 for timer_create() in libcheck.
CHECK_LIBRARY_EXISTS(rt timer_create "" HAVE_TIMER_CREATE)
IF (HAVE_TIMER_CREATE)
LIST(APPEND GALERA_UNIT_TEST_LIBS rt)
ENDIF(HAVE_TIMER_CREATE)

list(APPEND GALERA_UNIT_TEST_LIBS m)
list(APPEND GALERA_UNIT_TEST_LIBS ${GALERA_SYSTEM_LIBS})

Expand Down
4 changes: 2 additions & 2 deletions cmake/os.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
#

find_library(PTHREAD_LIB pthread)
find_library(RT_LIB rt)
set(GALERA_SYSTEM_LIBS ${PTHREAD_LIB} ${RT_LIB})

set(GALERA_SYSTEM_LIBS ${PTHREAD_LIB})

if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
# Check if linkage with atomic library is needed for 8 byte atomics
Expand Down
2 changes: 2 additions & 0 deletions galera/src/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ target_compile_options(galera_smm
)

if (GALERA_VERSION_SCRIPT)
if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this strictly isn't a linux problem, but a gnu linker?

Know why https://github.com/freebsd/freebsd-ports/blob/main/databases/galera26/Makefile is unaffected?

https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER_LINKER_ID.html

IF(CMAKE_CXX_LINKER_ID STREQUAL "GNU") ....

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, will fix

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Know why https://github.com/freebsd/freebsd-ports/blob/main/databases/galera26/Makefile is unaffected?

I am not sure, but probably FreeBSD moved to LLVM linker in 12.0 (12.0 (December 11, 2018)) https://man.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=1&manpath=FreeBSD+12.0-RELEASE&arch=default&format=html since when they do fine and this patch codership@e907ec1 landed only in 2019. But I did not check to be sure.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MacOS ld does not support this flag

# Limit symbols visible from Galera DSO.
# Doing this allows to:
# - make the ABI more clean and concise
Expand All @@ -113,6 +114,7 @@ if (GALERA_VERSION_SCRIPT)
set(GALERA_LINK_OPTIONS
-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/galera-sym.map)
endif()
endif()

target_link_libraries(galera_smm galera ${GALERA_LINK_OPTIONS})

Expand Down
96 changes: 96 additions & 0 deletions galerautils/src/gu_asio.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,102 @@
#include <memory>
#include <string>

#ifdef __APPLE__

struct tcp_info {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if tcp_info is here https://developer.apple.com/documentation/kernel/tcp_info - what does it mean?

So this is a copied list from somewhere?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking of this one, I thought to just reimplement tcp_info from Linux here partially filling it from Apple https://developer.apple.com/documentation/kernel/tcp_connection_info. As it was only a draft I have left only a struct without any fields being filled up. It is copied from https://github.com/torvalds/linux/blob/master/include/uapi/linux/tcp.h. Do you think it makes sense? Or I could cut all unused fields and fill only them?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only usage in " gcomm::SocketStats gcomm::AsioTcpSocket::stats" includes a small subset namely the tcpi structure:

          ret.rtt            = tcpi.tcpi_rtt;
          ret.rttvar         = tcpi.tcpi_rttvar;
          ret.rto            = tcpi.tcpi_rto;
  #if defined(__linux__)
          ret.lost           = tcpi.tcpi_lost;
  #else   
          ret.lost           = 0;
  #endif /* __linux__ */
          ret.last_data_recv = tcpi.tcpi_last_data_recv;
          ret.cwnd           = tcpi.tcpi_snd_cwnd;

If you find a compatible structure you could alias it, but if there's a difference fields create a minimal structure of only what's needed an populate that. I haven't found the macos function that does this.

uint8_t tcpi_state;
uint8_t tcpi_ca_state;
uint8_t tcpi_retransmits;
uint8_t tcpi_probes;
uint8_t tcpi_backoff;
uint8_t tcpi_options;
uint8_t tcpi_snd_wscale : 4, tcpi_rcv_wscale : 4;
uint8_t tcpi_delivery_rate_app_limited:1, tcpi_fastopen_client_fail:2;

uint32_t tcpi_rto;
uint32_t tcpi_ato;
uint32_t tcpi_snd_mss;
uint32_t tcpi_rcv_mss;

uint32_t tcpi_unacked;
uint32_t tcpi_sacked;
uint32_t tcpi_lost;
uint32_t tcpi_retrans;
uint32_t tcpi_fackets;

/* Times. */
uint32_t tcpi_last_data_sent;
uint32_t tcpi_last_ack_sent; /* Not remembered, sorry. */
uint32_t tcpi_last_data_recv;
uint32_t tcpi_last_ack_recv;

/* Metrics. */
uint32_t tcpi_pmtu;
uint32_t tcpi_rcv_ssthresh;
uint32_t tcpi_rtt;
uint32_t tcpi_rttvar;
uint32_t tcpi_snd_ssthresh;
uint32_t tcpi_snd_cwnd;
uint32_t tcpi_advmss;
uint32_t tcpi_reordering;

uint32_t tcpi_rcv_rtt;
uint32_t tcpi_rcv_space;

uint32_t tcpi_total_retrans;

uint64_t tcpi_pacing_rate;
uint64_t tcpi_max_pacing_rate;
uint64_t tcpi_bytes_acked; /* RFC4898 tcpEStatsAppHCThruOctetsAcked */
uint64_t tcpi_bytes_received; /* RFC4898 tcpEStatsAppHCThruOctetsReceived */
uint32_t tcpi_segs_out; /* RFC4898 tcpEStatsPerfSegsOut */
uint32_t tcpi_segs_in; /* RFC4898 tcpEStatsPerfSegsIn */

uint32_t tcpi_notsent_bytes;
uint32_t tcpi_min_rtt;
uint32_t tcpi_data_segs_in; /* RFC4898 tcpEStatsDataSegsIn */
uint32_t tcpi_data_segs_out; /* RFC4898 tcpEStatsDataSegsOut */

uint64_t tcpi_delivery_rate;

uint64_t tcpi_busy_time; /* Time (usec) busy sending data */
uint64_t tcpi_rwnd_limited; /* Time (usec) limited by receive window */
uint64_t tcpi_sndbuf_limited; /* Time (usec) limited by send buffer */

uint32_t tcpi_delivered;
uint32_t tcpi_delivered_ce;

uint64_t tcpi_bytes_sent; /* RFC4898 tcpEStatsPerfHCDataOctetsOut */
uint64_t tcpi_bytes_retrans; /* RFC4898 tcpEStatsPerfOctetsRetrans */
uint32_t tcpi_dsack_dups; /* RFC4898 tcpEStatsStackDSACKDups */
uint32_t tcpi_reord_seen; /* reordering events seen */

uint32_t tcpi_rcv_ooopack; /* Out-of-order packets received */

uint32_t tcpi_snd_wnd; /* peer's advertised receive window after
* scaling (bytes)
*/
uint32_t tcpi_rcv_wnd; /* local advertised receive window after
* scaling (bytes)
*/

uint32_t tcpi_rehash; /* PLB or timeout triggered rehash attempts */

uint16_t tcpi_total_rto; /* Total number of RTO timeouts, including
* SYN/SYN-ACK and recurring timeouts.
*/
uint16_t tcpi_total_rto_recoveries; /* Total number of RTO
* recoveries, including any
* unfinished recovery.
*/
uint32_t tcpi_total_rto_time; /* Total time spent in RTO recoveries
* in milliseconds, including any
* unfinished recovery.
*/
};

#endif

namespace gu
{
// URI schemes for networking
Expand Down
36 changes: 35 additions & 1 deletion galerautils/src/gu_asio_socket_util.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,40 @@ static void bind(Socket& socket, const gu::AsioIpAddress& addr)
}
}

#ifdef __APPLE__

template <class Socket>
static struct tcp_info get_tcp_info(Socket& socket)
{
struct tcp_info tcpi;
memset(&tcpi, 0, sizeof(tcpi));
// struct tcp_connection_info
// https://developer.apple.com/documentation/kernel/tcp_connection_info
// http://git.haproxy.org/?p=haproxy-2.6.git;a=commitdiff_plain;h=7747d465d54a1e367e9bf9c07c263d7f1f7fd481;hp=5c83e3a1563cd7face299bf08037e51f976eb5e3
// TODO needed fields:
// ret.rtt = tcpi.tcpi_rtt;
// ret.rttvar = tcpi.tcpi_rttvar;
// ret.rto = tcpi.tcpi_rto;
#if defined(__linux__)
// ret.lost = tcpi.tcpi_lost;
#else
// ret.lost = 0;
#endif /* __linux__ */
// ret.last_data_recv = tcpi.tcpi_last_data_recv;
// ret.cwnd = tcpi.tcpi_snd_cwnd;
// gu::datetime::Date now(gu::datetime::Date::monotonic());
// Critical<AsioProtonet> crit(net_);
// ret.last_queued_since = (now - last_queued_tstamp_).get_nsecs();
// ret.last_delivered_since = (now - last_delivered_tstamp_).get_nsecs();
// ret.send_queue_length = send_q_.size();
// ret.send_queue_bytes = send_q_.queued_bytes();
// ret.send_queue_segments = send_q_.segments();

return tcpi;
}

#else

template <class Socket>
static struct tcp_info get_tcp_info(Socket& socket)
{
Expand All @@ -165,7 +199,7 @@ static struct tcp_info get_tcp_info(Socket& socket)
#endif /* __linux__ || __FreeBSD__ */
return tcpi;
}

#endif

static inline std::string
uri_string (const std::string& scheme, const std::string& addr,
Expand Down
8 changes: 4 additions & 4 deletions galerautils/src/gu_threads.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,11 +248,11 @@ int gu_barrier_init_SYS (gu_barrier_t_SYS *barrier,
errno = EINVAL;
return -1;
}
if(gu_mutex_init_SYS (&barrier->mutex, 0) < 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unclear why you swapped arguments here.

pthread_mutex_init seems to be the same on both, mutex then mutex attributed. If there as need to swap them make the gu_mutex_init_SYS macro have arguments that handle these portable.q

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was a patch 09848b6 by Jan from 2021-11-11 14:22:24 +0200 that swapped the order of the arguments to gu_mutex_init_SYS. so now its gu_mutex_init_SYS(const wsrep_mutex_key_t* key, gu_mutex_t_SYS *mutex) so that for keyless mutexes its using (NULL, &mutex).

the macros, if you mean gu_mutex_init is just a debug wrapper over the name, the only that I know: #define gu_mutex_init gu_mutex_init_SYS.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the patch replaced - #define gu_mutex_init_SYS pthread_mutex_init with its own impl that has different args (key, mutex)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#25

if(gu_mutex_init_SYS (NULL, &barrier->mutex) < 0)
{
return -1;
}
if(gu_cond_init_SYS (&barrier->cond, 0) < 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also consistent order in pthread_cond_init

https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/pthread.3.html

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see https://github.com/MariaDB/galera/pull/20/files#r1707091901 and https://github.com/MariaDB/galera/pull/20/files#r1707104348

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#25

if(gu_cond_init_SYS (NULL, &barrier->cond) < 0)
{
gu_mutex_destroy_SYS (&barrier->mutex);
return -1;
Expand All @@ -279,13 +279,13 @@ int gu_barrier_wait_SYS (gu_barrier_t_SYS *barrier)
barrier->count = 0;
gu_cond_broadcast_SYS (&barrier->cond);
gu_mutex_unlock_SYS (&barrier->mutex);
return GU_BARRIER_THREAD_SYS;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was introduced by dabe053 8 years ago, but I suppose it never compiled as at that moment there were no GU_BARRIER_THREAD_SYS.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is synonymous to PTHREAD_BARRIER_SERIAL_THREAD that is must return something non-usual on success (here its -1) for a thread synced with a barrier as per POSIX https://linux.die.net/man/3/pthread_barrier_wait

The constant PTHREAD_BARRIER_SERIAL_THREAD is defined in <pthread.h> and its value shall be distinct from any other value returned by pthread_barrier_wait().

Upon successful completion, the pthread_barrier_wait() function shall return PTHREAD_BARRIER_SERIAL_THREAD for a single (arbitrary) thread synchronized at the barrier and zero for each of the other threads.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(there is no such a constant in the code GU_BARRIER_THREAD_SYS)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#25

return GU_BARRIER_SERIAL_THREAD_SYS;
}
else
{
gu_cond_wait_SYS (&barrier->cond, &(barrier->mutex));
gu_mutex_unlock_SYS (&barrier->mutex);
return !GU_BARRIER_THREAD_SYS;
return !GU_BARRIER_SERIAL_THREAD_SYS;
}
}

Expand Down
4 changes: 0 additions & 4 deletions galerautils/src/gu_threads.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@

#include "gu_types.h" // bool

#if __unix__
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could do #if defined(__unix__) || defined(__APPLE__) as a minimal change without assuming too much about other cases.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I will fix that, it was a hack

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#25


#include <pthread.h>
#include <assert.h>

Expand Down Expand Up @@ -318,8 +316,6 @@ typedef pthread_barrier_t gu_barrier_t_SYS;

#endif /* native POSIX barriers */

#endif /* __unix__ */

/**
* Depending on compile-time flags application will either use
* normal or debug version of the API calls
Expand Down
Loading