Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] [python-package] build macOS x86_64 wheels on macOS 13 (Ventura) #6669

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
+24 −7
Open

[ci] [python-package] build macOS x86_64 wheels on macOS 13 (Ventura) #6669

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
18aeb61
[ci] [python-package] build macOS x86_64 wheels on macOS 13 (Ventura)
jameslamb Oct 9, 2024
a72bdb9
update Xcode version
jameslamb Oct 9, 2024
70fb7e7
Merge branch 'master' into macos-version
jameslamb Oct 16, 2024
c113f90
Merge branch 'master' into macos-version
jameslamb Nov 27, 2024
9a83af6
Update .ci/setup.sh
jameslamb Dec 11, 2024
43db19b
Merge branch 'master' into macos-version
jameslamb Dec 11, 2024
b14086b
Merge branch 'master' into macos-version
jameslamb Jan 20, 2025
5d4c27e
fix XCode version detection
jameslamb Jan 20, 2025
be588af
revert debugging change
jameslamb Jan 20, 2025
e52719c
remove dsymutil dirs
jameslamb Jan 20, 2025
6b5af3c
avoid producing dsymutil reproducer directories
jameslamb Jan 20, 2025
449dc09
restore all CI
jameslamb Jan 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions .ci/setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,18 @@ ARCH=$(uname -m)


if [[ $OS_NAME == "macos" ]]; then
# Check https://github.com/actions/runner-images/tree/main/images/macos for available
# versions of Xcode
macos_ver=$(sw_vers --productVersion)
if [[ "${macos_ver}" =~ 13. ]]; then
xcode_path="/Applications/Xcode_14.3.app/Contents/Developer"
else
xcode_path="/Applications/Xcode_15.0.app/Contents/Developer"
fi
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chose the oldest included versions of XCode from here:

And while doing this, proposing this more-direct approach for selecting the version... depending on the actual macOS version instead of things that indirectly select it from this project's CI configs, like $AZURE and $COMPILER.

sudo xcode-select -s "${xcode_path}" || exit 1
if [[ $COMPILER == "clang" ]]; then
brew install libomp
if [[ $AZURE == "true" ]]; then
sudo xcode-select -s /Applications/Xcode_13.1.0.app/Contents/Developer || exit 1
fi
else # gcc
# Check https://github.com/actions/runner-images/tree/main/images/macos for available
# versions of Xcode
sudo xcode-select -s /Applications/Xcode_14.3.1.app/Contents/Developer || exit 1
brew install 'gcc@12'
fi
if [[ $TASK == "mpi" ]]; then
Expand Down
14 changes: 14 additions & 0 deletions .github/workflows/r_package.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,20 @@ env:
CMAKE_BUILD_PARALLEL_LEVEL: 4
# on Debian-based images, avoid interactive prompts
DEBIAN_FRONTEND: noninteractive
# Fix issues like the following that can show up running 'R CMD check' on
# specific clang versions:
#
# * checking for detritus in the temp directory ... NOTE
# Found the following files/directories:
# ‘dsymutil-63923a’ ‘dsymutil-9aa721’ ‘dsymutil-b7e1bb’
#
# These are unlikely to show up in CRAN's checks. They come from
# 'dsymutil ---gen-reproducer' being run (not something LightGBM explicitly does).
#
# ref:
# - https://github.com/llvm/llvm-project/issues/61920
# - https://github.com/golang/go/issues/59026#issuecomment-1520487072
DSYMUTIL_REPRODUCER_PATH: /dev/null
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When dsymutil is run with e.g. --gen-reproducer or similar flags, it leaves behind these temporary directories with details for filing bug reports.

In some versions of clang, those are not cleaned up correctly (llvm/llvm-project#61920).

dsymutil is writing those files to TMPDIR, which is the "temp directory" referred to in this R CMD check NOTE.

Redirecting these files to anywhere else avoids the R CMD check NOTE. That can be done via environment variable DSYMUTIL_REPRODUCER_PATH. Here, I'm following the tip from golang/go#59026 (comment) and using /dev/null, so the files aren't even written to disk at all.

I think this is all fine for CI. If this kind of thing was a problem on CRAN, it'd affect every project on CRAN building with clang 14, as I LightGBM isn't doing anything custom with dsymutil.

# parallelize compilation (extra important for Linux, where CRAN doesn't supply pre-compiled binaries)
MAKEFLAGS: "-j4"
# hack to get around this:
Expand Down
2 changes: 1 addition & 1 deletion .vsts-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ jobs:
OS_NAME: 'macos'
PRODUCES_ARTIFACTS: 'true'
pool:
vmImage: 'macOS-12'
vmImage: 'macOS-13'
strategy:
matrix:
regular:
Expand Down
Loading