macos: basic support for darwin builds #20

sitano · 2024-07-31T13:11:42Z

It's pretty simple to build Galera Cluster on Darwin (even arm64). Let it be here if anyone will ever need. I will try to rework it in the future into the mergeable form if there is a desire to adopt it.

State

At the moment, I have tested only Cluster Bootstrap, Node Join, and SST/IST streaming. Works on MBP M3.

grooverdan

Thanks for the draft. Most changes are non-invasive so I think they can be merged with a bit of clean-up.

The tcp_info is a bit messy. The usage inside gcomm::AsioTcpSocket::stats and its callers seems to be informational only. If too hard just exclude it for now,

If the tcp_connection_info rtt, rttval, rto map then just using those make sense and seems to be easy.

grooverdan · 2024-08-07T07:17:45Z

galera/src/CMakeLists.txt

@@ -104,6 +104,7 @@ target_compile_options(galera_smm
  )

 if (GALERA_VERSION_SCRIPT)
+if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")


So this strictly isn't a linux problem, but a gnu linker?

Know why https://github.com/freebsd/freebsd-ports/blob/main/databases/galera26/Makefile is unaffected?

https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER_LINKER_ID.html

IF(CMAKE_CXX_LINKER_ID STREQUAL "GNU") ....

agree, will fix

Know why https://github.com/freebsd/freebsd-ports/blob/main/databases/galera26/Makefile is unaffected?

I am not sure, but probably FreeBSD moved to LLVM linker in 12.0 (12.0 (December 11, 2018)) https://man.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=1&manpath=FreeBSD+12.0-RELEASE&arch=default&format=html since when they do fine and this patch codership@e907ec1 landed only in 2019. But I did not check to be sure.

MacOS ld does not support this flag

CMakeLists.txt

cmake/os.cmake

grooverdan · 2024-08-07T07:34:20Z

galerautils/src/gu_asio.hpp

@@ -27,6 +27,102 @@
 #include <memory>
 #include <string>

+#ifdef __APPLE__
+
+struct tcp_info {


if tcp_info is here https://developer.apple.com/documentation/kernel/tcp_info - what does it mean?

So this is a copied list from somewhere?

Speaking of this one, I thought to just reimplement tcp_info from Linux here partially filling it from Apple https://developer.apple.com/documentation/kernel/tcp_connection_info. As it was only a draft I have left only a struct without any fields being filled up. It is copied from https://github.com/torvalds/linux/blob/master/include/uapi/linux/tcp.h. Do you think it makes sense? Or I could cut all unused fields and fill only them?

The only usage in " gcomm::SocketStats gcomm::AsioTcpSocket::stats" includes a small subset namely the tcpi structure:

ret.rtt = tcpi.tcpi_rtt; ret.rttvar = tcpi.tcpi_rttvar; ret.rto = tcpi.tcpi_rto; #if defined(__linux__) ret.lost = tcpi.tcpi_lost; #else ret.lost = 0; #endif /* __linux__ */ ret.last_data_recv = tcpi.tcpi_last_data_recv; ret.cwnd = tcpi.tcpi_snd_cwnd;

If you find a compatible structure you could alias it, but if there's a difference fields create a minimal structure of only what's needed an populate that. I haven't found the macos function that does this.

grooverdan · 2024-08-07T07:40:44Z

galerautils/src/gu_threads.h

@@ -11,8 +11,6 @@

 #include "gu_types.h" // bool

-#if __unix__


could do #if defined(__unix__) || defined(__APPLE__) as a minimal change without assuming too much about other cases.

yes, I will fix that, it was a hack

grooverdan · 2024-08-07T07:45:09Z

galerautils/src/gu_threads.c

@@ -248,11 +248,11 @@ int gu_barrier_init_SYS (gu_barrier_t_SYS *barrier,
        errno = EINVAL;
        return -1;
    }
-    if(gu_mutex_init_SYS (&barrier->mutex, 0) < 0)


unclear why you swapped arguments here.

pthread_mutex_init seems to be the same on both, mutex then mutex attributed. If there as need to swap them make the gu_mutex_init_SYS macro have arguments that handle these portable.q

there was a patch 09848b6 by Jan from 2021-11-11 14:22:24 +0200 that swapped the order of the arguments to gu_mutex_init_SYS. so now its gu_mutex_init_SYS(const wsrep_mutex_key_t* key, gu_mutex_t_SYS *mutex) so that for keyless mutexes its using (NULL, &mutex).

the macros, if you mean gu_mutex_init is just a debug wrapper over the name, the only that I know: #define gu_mutex_init gu_mutex_init_SYS.

the patch replaced - #define gu_mutex_init_SYS pthread_mutex_init with its own impl that has different args (key, mutex)

grooverdan · 2024-08-07T07:46:15Z

galerautils/src/gu_threads.c

@@ -279,13 +279,13 @@ int gu_barrier_wait_SYS (gu_barrier_t_SYS *barrier)
        barrier->count = 0;
        gu_cond_broadcast_SYS (&barrier->cond);
        gu_mutex_unlock_SYS (&barrier->mutex);
-        return GU_BARRIER_THREAD_SYS;


why this change?

it was introduced by dabe053 8 years ago, but I suppose it never compiled as at that moment there were no GU_BARRIER_THREAD_SYS.

it is synonymous to PTHREAD_BARRIER_SERIAL_THREAD that is must return something non-usual on success (here its -1) for a thread synced with a barrier as per POSIX https://linux.die.net/man/3/pthread_barrier_wait

The constant PTHREAD_BARRIER_SERIAL_THREAD is defined in <pthread.h> and its value shall be distinct from any other value returned by pthread_barrier_wait().

Upon successful completion, the pthread_barrier_wait() function shall return PTHREAD_BARRIER_SERIAL_THREAD for a single (arbitrary) thread synchronized at the barrier and zero for each of the other threads.

(there is no such a constant in the code GU_BARRIER_THREAD_SYS)

grooverdan · 2024-08-07T07:47:26Z

galerautils/src/gu_threads.c

    {
        return -1;
    }
-    if(gu_cond_init_SYS (&barrier->cond, 0) < 0)


also consistent order in pthread_cond_init

https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/pthread.3.html

see https://github.com/MariaDB/galera/pull/20/files#r1707091901 and https://github.com/MariaDB/galera/pull/20/files#r1707104348

sitano · 2024-08-07T08:12:49Z

@grooverdan thank your feedback! I will improve everything to be mergeable as at the 1st stage I didn't know if the work worth an effort. Will address / respond to your comments soon.

sitano · 2024-08-07T15:27:14Z

@grooverdan wanted to ask if there is any CI pipeline that checks builds under different environments that I could use for dev of this PR? As I am pretty limited at the moment with my options to verify builds for other OSs. or maybe you could suggest what you are using for testing the code on your dev machines to ensure it works for different platforms that must be supported.

grooverdan · 2024-08-14T05:39:13Z

Sorry for the delay, what a week:

pull request 20:

https://buildbot.mariadb.org/#/grid?branch=refs%2Fpull%2F20%2Fmerge

Since glibc >=2.17 (2012-12-25), librt is a stub-only library. (https://abi-laboratory.pro/?view=changelog&l=glibc&v=2.17) * The `clock_*' suite of functions (declared in <time.h>) is now available directly in the main C library. Previously it was necessary to link with -lrt to use these functions. This change has the effect that a single-threaded program that uses a function such as `clock_gettime' (and is not linked with -lrt) will no longer implicitly load the pthreads library at runtime and so will not suffer the overheads associated with multi-thread support in other code such as the C++ runtime library. Current version on Arch Linux is 2.40. It is however is required for Debian 10, 11 and Ubuntu 20.04. Signed-off-by: Ivan Prisyazhnyy <[email protected]>

Signed-off-by: Ivan Prisyazhnyy <[email protected]>

calls constructor for a mutex in a struct value init-ed with gu_calloc. in path `gcs_core_create() -> gcs_group_init()`, the first one allocates `gcs_core_t* core` with gu_calloc() whereas `gcs_code_t` has `gcs_group_t group` with `gu::Mutex memb_mtx_`. After memory allocation gu::Mutex constructor was not called that lead to an error on Darwin in a call to pthread mutex lock. Signed-off-by: Ivan Prisyazhnyy <[email protected]>

sitano · 2024-09-17T07:33:13Z

any desire to move it forward?
I have a few PRs waiting for the review and approval after which I could add the rest if you are interested.

sitano · 2024-09-17T07:36:27Z

we have at the moment to review

macos: fix missing pthread mutex init after calloc #21 macos: fix missing pthread mutex init after calloc - this one should be closed as soon as the fix 5dc30e6 will make its way to the upstream
build(macos/other): librt is not needed #24 build(macos/other): librt is not needed
macos: fix gu_cond usage for darwin after 09848b6 #25 macos: fix gu_cond usage for darwin after 09848b6

This was referenced Jul 31, 2024

Build for MacOS codership/galera#660

Open

macos: fix missing pthread mutex init after calloc #21

Open

sitano force-pushed the ivan/macos branch from 629a03c to 28664e0 Compare July 31, 2024 13:55

sitano mentioned this pull request Jul 31, 2024

mariadb (sorry): fix missing pthread mutex init after calloc codership/galera#661

Closed

grooverdan reviewed Aug 7, 2024

View reviewed changes

sitano mentioned this pull request Aug 25, 2024

build(macos/other): librt is not needed #24

Open

sitano force-pushed the ivan/macos branch from 28664e0 to 23a0813 Compare August 27, 2024 08:39

sitano added 6 commits August 28, 2024 09:37

macos: sym map is not supported by llvm clang

aa6f9ab

Signed-off-by: Ivan Prisyazhnyy <[email protected]>

macos: tcp sock info support

b52c519

Signed-off-by: Ivan Prisyazhnyy <[email protected]>

macos: apple is almost unix, support pthreads in gu

78dfe02

Signed-off-by: Ivan Prisyazhnyy <[email protected]>

macos: darwin has time impl

5afb256

Signed-off-by: Ivan Prisyazhnyy <[email protected]>

sitano force-pushed the ivan/macos branch 2 times, most recently from b26bbc1 to 1abe2d1 Compare August 28, 2024 07:47

sitano mentioned this pull request Aug 28, 2024

macos: fix gu_cond usage for darwin after 09848b6 #25

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macos: basic support for darwin builds #20

macos: basic support for darwin builds #20

sitano commented Jul 31, 2024

grooverdan left a comment

grooverdan Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 7, 2024

grooverdan Aug 7, 2024

sitano Aug 7, 2024

grooverdan Aug 14, 2024

grooverdan Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 28, 2024

grooverdan Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 28, 2024

grooverdan Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 28, 2024

grooverdan Aug 7, 2024

sitano Aug 7, 2024

sitano Aug 28, 2024

sitano commented Aug 7, 2024

sitano commented Aug 7, 2024

grooverdan commented Aug 14, 2024

sitano commented Sep 17, 2024

sitano commented Sep 17, 2024

macos: basic support for darwin builds #20

Are you sure you want to change the base?

macos: basic support for darwin builds #20

Conversation

sitano commented Jul 31, 2024

State

grooverdan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sitano commented Aug 7, 2024

sitano commented Aug 7, 2024

grooverdan commented Aug 14, 2024

sitano commented Sep 17, 2024

sitano commented Sep 17, 2024