Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bitcoin-{node,gui} to release binaries for IPC #31802

Draft
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

Sjors
Copy link
Member

@Sjors Sjors commented Feb 5, 2025

Have depends make libmultiprocess by default, which has the effect of including it in the release binaries. Except for Windows and OpenBSD which are not supported yet, the latter due to a fairly trivial upstream issue.

The initial main use case is to enable experimental support for the Mining IPC interface. A working example of a Stratum v2 Template Provider client using this interface can be found here: Sjors#48.

Additionally the bitcoin-node and bitcoin-gui are added to Maintenance.cmake.

Builds on #30975 (which originally tried to do this) and #31741.

See #31756 for discussion of when this should happen. See also #31098 for any remaining issues specific to the Mining interface. Will keep this draft for now.

Guix hashes:

aarch64,x86_64
86dcd8576b824516564e16e8e39bac5c14d98cde7d0e01e82376cbd7bbb623ca  guix-build-4aaed63d4e6a/output/aarch64-linux-gnu/SHA256SUMS.part
c88cab96afdb03c22d0a4e285a19f2fd86d488d0682fc992c08143319154d10a  guix-build-4aaed63d4e6a/output/aarch64-linux-gnu/bitcoin-4aaed63d4e6a-aarch64-linux-gnu-debug.tar.gz
41114593b04b37903a35e175580afa709ab19cc71e493a8e646163549b60db29  guix-build-4aaed63d4e6a/output/aarch64-linux-gnu/bitcoin-4aaed63d4e6a-aarch64-linux-gnu.tar.gz
7d94ee2f975bffacfc55682e93fa625db75b9d5e3505d3bcee1ee5d48c2cb885  guix-build-4aaed63d4e6a/output/arm-linux-gnueabihf/SHA256SUMS.part
a0a6ad71827eee2bb483db0d9a6cbd9fc28be002ee5a0416bbcb37d40be6de92  guix-build-4aaed63d4e6a/output/arm-linux-gnueabihf/bitcoin-4aaed63d4e6a-arm-linux-gnueabihf-debug.tar.gz
f32b80f637d96a7208b87630469dd5c75a520fa680f1ebaa50be1c7c991c26f6  guix-build-4aaed63d4e6a/output/arm-linux-gnueabihf/bitcoin-4aaed63d4e6a-arm-linux-gnueabihf.tar.gz
135da5253af8c8b349b7136a146edd2a4dedd92991600f87dbd95e5d05666b7b  guix-build-4aaed63d4e6a/output/arm64-apple-darwin/SHA256SUMS.part
87c44c4e7f36cc60624a0668389f2ab1b3e234ba846b15a2c21283a873df3f5b  guix-build-4aaed63d4e6a/output/arm64-apple-darwin/bitcoin-4aaed63d4e6a-arm64-apple-darwin-unsigned.tar.gz
eefe25999f8c39fd7d80483f669f0f0133528121e39b0ce234412cf5bdaa7dd3  guix-build-4aaed63d4e6a/output/arm64-apple-darwin/bitcoin-4aaed63d4e6a-arm64-apple-darwin-unsigned.zip
44e0fe9969cd6708984540e37c1672969c3e3542d122a776e97fbf8b5417e9bb  guix-build-4aaed63d4e6a/output/arm64-apple-darwin/bitcoin-4aaed63d4e6a-arm64-apple-darwin.tar.gz
e204962bf72ddd720207bf18ee5837e445318409e97f6ee6cdbbb4adbf316427  guix-build-4aaed63d4e6a/output/dist-archive/bitcoin-4aaed63d4e6a.tar.gz
78f11b7807cbd8a39b269accf4a0ffa001586a0836b7b95ccb9f7f4451e92e7b  guix-build-4aaed63d4e6a/output/powerpc64-linux-gnu/SHA256SUMS.part
19d354032002ec61368c2eebf2e45153db4e8d72138284b5933251c73c31ce6b  guix-build-4aaed63d4e6a/output/powerpc64-linux-gnu/bitcoin-4aaed63d4e6a-powerpc64-linux-gnu-debug.tar.gz
c6f4e741f056ee3bfe180cfd27647124279b92715656e60398974b9197d496fc  guix-build-4aaed63d4e6a/output/powerpc64-linux-gnu/bitcoin-4aaed63d4e6a-powerpc64-linux-gnu.tar.gz
2e53b82e489e3e3696dc607139f4445cbafca3d990897372cf2998ab71b6fa7a  guix-build-4aaed63d4e6a/output/riscv64-linux-gnu/SHA256SUMS.part
560c9cea10923ce2bf8450cf5d930d180bb819cc6e532885ab15f1aa5a914b8e  guix-build-4aaed63d4e6a/output/riscv64-linux-gnu/bitcoin-4aaed63d4e6a-riscv64-linux-gnu-debug.tar.gz
b3609c9a8cfd649bc73631b9c54425f33cdb1dfaffc49a2774566d8b3ee2e6db  guix-build-4aaed63d4e6a/output/riscv64-linux-gnu/bitcoin-4aaed63d4e6a-riscv64-linux-gnu.tar.gz
3aa32a9e9b2348cb0f5ef89c504bfbced037e433564cbb2b88d5d80e63ccd765  guix-build-4aaed63d4e6a/output/x86_64-apple-darwin/SHA256SUMS.part
6cbafddc4c947c17a9d57e5449e9031531ee01c5946e060121f3a2c50f5524d4  guix-build-4aaed63d4e6a/output/x86_64-apple-darwin/bitcoin-4aaed63d4e6a-x86_64-apple-darwin-unsigned.tar.gz
b22b49a3231ad992d941085b03628e1f0deac2e6f5d48abe650a93a032872b50  guix-build-4aaed63d4e6a/output/x86_64-apple-darwin/bitcoin-4aaed63d4e6a-x86_64-apple-darwin-unsigned.zip
35ccde020762d6cfc212076d740812c422d5bccc7f5fd435a4b4e1796f517e93  guix-build-4aaed63d4e6a/output/x86_64-apple-darwin/bitcoin-4aaed63d4e6a-x86_64-apple-darwin.tar.gz
c4fbde9ca7a7c537c1c0b2f26658ab5f53c1cdf597721d5e570e2986b6fd7eff  guix-build-4aaed63d4e6a/output/x86_64-linux-gnu/SHA256SUMS.part
478cb959eb3064692d696db01df87804203f99e8abfd6a0ce343cfeaab9a9ccc  guix-build-4aaed63d4e6a/output/x86_64-linux-gnu/bitcoin-4aaed63d4e6a-x86_64-linux-gnu-debug.tar.gz
9dbd79434cd42c51607b2b2343122df52e6f6ec472905a765f54a94de07a19fd  guix-build-4aaed63d4e6a/output/x86_64-linux-gnu/bitcoin-4aaed63d4e6a-x86_64-linux-gnu.tar.gz
d6b7c9fa00ccc0482009f3d0ad32034a888f3e733ad38d6fb3788bb947ab2f4c  guix-build-4aaed63d4e6a/output/x86_64-w64-mingw32/SHA256SUMS.part
83fe61e4243dde62ecfc90e19640e811d67fc7c66d585467619857621013f06f  guix-build-4aaed63d4e6a/output/x86_64-w64-mingw32/bitcoin-4aaed63d4e6a-win64-debug.zip
55b5aa31920b98c27dba73795e06c752db778c2aefa5e2bcdcbd27cf092a5164  guix-build-4aaed63d4e6a/output/x86_64-w64-mingw32/bitcoin-4aaed63d4e6a-win64-setup-unsigned.exe
da78aeb881245df6cfc1786cc323c62648a945b0da9c9fe4e98fd1f0da9243e3  guix-build-4aaed63d4e6a/output/x86_64-w64-mingw32/bitcoin-4aaed63d4e6a-win64-unsigned.tar.gz
96007bf6025fe4d43f718c515133109889f9d4e538fa760ae0843f8eb06a4bdd  guix-build-4aaed63d4e6a/output/x86_64-w64-mingw32/bitcoin-4aaed63d4e6a-win64.zip

ryanofsky and others added 17 commits February 3, 2025 09:23
Bring in a few cmake changes to improve cmake support

chaincodelabs/libmultiprocess#140 build: don't clobber user/superproject c++ version
chaincodelabs/libmultiprocess#142 build: add option for external mpgen binary
chaincodelabs/libmultiprocess#143 cleanup: initialize vars in the EventLoop constructor in the correct order
chaincodelabs/libmultiprocess#146 cmake: Suppress compiler warnings from capnproto headers
chaincodelabs/libmultiprocess#147 cmake: EXTERNAL_MPGEN cleanups
chaincodelabs/libmultiprocess#148 util: fix -Wpessimizing-move warning
chaincodelabs/libmultiprocess#145 CTest: Module must be included at the top level
chaincodelabs/libmultiprocess#149 Avoid `-Wundef` compiler warnings
chaincodelabs/libmultiprocess#152 refactor: Fix compiler and clang-tidy warnings
git-subtree-dir: src/ipc/libmultiprocess
git-subtree-split: 9558ceb0d47ac1f62f88d29ec55f8f3099e32b20
Rename WITH_MULTIPROCESS to ENABLE_IPC, because ENABLE_IPC is a more accurate
name for the feature. It controls whether the src/ipc/ directory is built and
whether IPC features like -ipcbind, -ipcconnect, and -ipcfd are available. It
does NOT currently enable multiprocess features which are implemented in bitcoin#10102
building on top of the IPC features. It will also no longer (as of the next
commit), control whether a find_package is call is made so the "WITH_" prefix
is also inappropriate.

-BEGIN VERIFY SCRIPT-
git grep -l WITH_MULTIPROCESS | xargs sed -i s/WITH_MULTIPROCESS/ENABLE_IPC/g
-END VERIFY SCRIPT-
When ENABLE_IPC option is on, build with libmultiprocess subtree and
`add_subdirectory(src/ipc/libmultiprocess)` instead of external package
and `find_package(Libmultiprocess)` by default.

Behavior can be toggled with `WITH_LIBMULTIPROCESS` option. Using a subtree
should be more convenient for most bitcoin developers, but using an external
package is more convenient for developing in the libmultiprocess repository.

The `WITH_LIBMULTIPROCESS` option is also used to avoid needing to changing the
depends build here. But in later commits, the depends build is switched to use
the add_subdirectory build as well.

Co-authored-by: Cory Fields <[email protected]>
This change is technically not needed to add libmultiprocess as a subtree, but
it avoids a CI failure in followup PR bitcoin#30975 which enables multiprocess build
option in more CI jobs. In that PR, several jobs fail due to the mptest
executable not being built by default, as reported
bitcoin#30975 (comment)
This change is technically not needed to add libmultiprocess as a subtree, but
it avoids a CI failure in followup PR bitcoin#30975 which enables multiprocess build
option in more CI jobs. In that PR, the "macOS 14 native no depends job" fails
due to warnings in boost headers treated as errors, reported in
bitcoin#30975 (comment) and
chaincodelabs/libmultiprocess#138
This change is technically not needed to add libmultiprocess as a subtree, but
it avoids a CI failure in followup PR bitcoin#30975 which enables multiprocess build
option in more CI jobs. In that PR, clang-tidy job fails due to missing
generated example files as reported
bitcoin#30975 (comment)

Different fixes were suggested
bitcoin#30975 (comment) and
bitcoin#30975 (comment)

Co-authored-by: Hennadii Stepanov <[email protected]>
Without this change linter produces errors about:

- Use of std::filesystem the libmultiprocess example program.
- Use of locale-dependent functions in example program, in the build time code
  generator, and in the runtime library for debug logging.
- Include guards not beginning with BITCOIN_
Move parts of the int_get_build_id into a new int_get_build_properties
function. There is no change in behavior. This just organizes assignments
better so some build properties can be used to help compute build ids in the
next commit.
With newly introduced libmultiprocess subtree, there's no need for depends
system to download and track changes to the upstream repository.

Note that adding the libmultiprocess subtree does not allow dropping
libmultiprocess packages from the depends build, because libmultiprocess
includes a code generation tool called mpgen, and in cross-compiled builds,
bitcoin core's cmake build system doesn't have access to a native toolchain and
can't build mpgen itself, so the depends system (or the native environment if
not using depends) needs to supply it.
Except:
1. i686, DEBUG (changed to "no multiprocess")
2. Windows due to lack of support
Install capnp where needed.

The bitcoin-node binary is built on all platforms which have
multiprocess enabled, but for functional tests it's only used in
the macOS native (no depends) and CentOS native (depends) jobs.
Except for Windows and OpenBSD.
@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 5, 2025

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31802.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #31800 (depends: Avoid using the -ffile-prefix-map compiler option by hebasto)
  • #31793 (ci: Use clang-20 for sanitizer tasks by maflcko)
  • #31765 (cmake: Install man pages for configured targets only by hebasto)
  • #31375 (multiprocess: Add bitcoin wrapper executable by ryanofsky)
  • #31282 (refactor: Make node_id a const& in RemoveBlockRequest by maflcko)
  • #31161 (cmake: Set top-level target output locations by hebasto)
  • #30595 (kernel: Introduce initial C header API by TheCharlatan)
  • #28710 (Remove the legacy wallet and BDB dependency by achow101)
  • #25573 ([POC] guix: produce a fully -static-pie bitcoind by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants