Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAOS-13386 cart: Execute CI tests with debug build of the UCX libraries. #12246

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from
Draft

DAOS-13386 cart: Execute CI tests with debug build of the UCX libraries. #12246

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
f230886
DAOS-13386 cart: upgrade to UCX 1.14.
jgmoore-or May 15, 2023
c33d023
DAOS-13386 cart: Upgrade to UCX 1.14
jgmoore-or May 15, 2023
b164196
DAOS-13386 cart: Upgrade to UCX 1.14.
jgmoore-or May 15, 2023
886a34f
DAOS-13386 cart: Upgrade to UCX 1.14
jgmoore-or May 19, 2023
5b2f1f8
Merge branch 'master' into jgm/daos-13386
jgmoore-or May 30, 2023
ba14a06
DAOS-13386 cart: Upgrade to UCX 1.14.
jgmoore-or May 31, 2023
5d48af3
DAOS-13386 cart: Test with UCX 1.14 debug build.
jgmoore-or May 31, 2023
18f2c7c
Merge branch 'master' into jgm/daos-13386-2
jgmoore-or Jun 2, 2023
18e26a7
DAOS-12286 cart: Execute CI tests with debug build of the UCX libraries.
jgmoore-or Jun 2, 2023
37fc1fd
DAOS-13386 cart: Run CI tests with UCX debug build.
jgmoore-or Jun 5, 2023
9a516b0
DAOS-13386 cart: Use UCX debug build.
jgmoore-or Jun 7, 2023
b6b14cb
DAOS-13886 cart: Test with UCX debug build.
jgmoore-or Jun 7, 2023
3a5be40
DAOS-13386 cart: Run with UCX debug build.
jgmoore-or Jun 7, 2023
1971169
DAOS-13386 cart: Run with debug build for UCX.
jgmoore-or Jun 7, 2023
f94bc0c
DAOS-13386 cart: Test with UCX debug build.
jgmoore-or Jun 13, 2023
c2d70ed
DAOS-13386 cart: Test with UCX debug build.
jgmoore-or Jun 13, 2023
7af02a9
DAOS-13386 cart: Run with UCX debug build.
jgmoore-or Jun 14, 2023
abb9b9f
Merge branch 'master' into jgm/daos-13386-2
jgmoore-or Jun 27, 2023
6749cd1
DAOS-13386 ucx: Run with UCX debug build.
jgmoore-or Jun 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions ci/provisioning/post_provision_config_common_functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,11 +273,6 @@ post_provision_config_nodes() {
return 1
fi

if lspci | grep "ConnectX-6" && ! grep MOFED_VERSION /etc/do-release; then
# Remove OPA and install MOFED
install_mofed
fi

if [ -n "$INST_REPOS" ]; then
local repo
for repo in $INST_REPOS; do
Expand Down Expand Up @@ -309,6 +304,11 @@ post_provision_config_nodes() {
fi
fi

if lspci | grep "ConnectX-6" && ! grep MOFED_VERSION /etc/do-release; then
# Remove OPA and install MOFED
install_mofed
fi

# shellcheck disable=SC2001
if ! rpm -q "$(echo "$INST_RPMS" |
sed -e 's/--exclude [^ ]*//' \
Expand Down
3 changes: 2 additions & 1 deletion ci/provisioning/post_provision_config_nodes_EL_8.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ install_mofed() {
gversion="${gversion%.*}"
fi

time dnf -y install ucx ucx-cma ucx-ib ucx-rdmacm
# Add a repo to install MOFED RPMS
repo_url=https://artifactory.dc.hpdd.intel.com/artifactory/mlnx_ofed/"$MLNX_VER_NUM-rhel$gversion"-x86_64/
dnf -y config-manager --add-repo="$repo_url"
Expand All @@ -64,7 +65,7 @@ install_mofed() {
rm -f RPM-GPG-KEY-Mellanox
dnf repolist || true

time dnf -y install mlnx-ofed-basic ucx-cma ucx-ib ucx-knem ucx-rdmacm ucx-xpmem
time dnf -y install mlnx-ofed-basic

# now, upgrade firmware
time dnf -y install mlnx-fw-updater
Expand Down
3 changes: 3 additions & 0 deletions src/cart/crt_init.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,9 @@ static int data_init(int server, crt_init_options_t *opt)
if (server)
setenv("UCX_IB_FORK_INIT", "n", 1);

setenv("D_LOG_STDERR_IN_LOG", "1", 1);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

setenv("UCX_SOCKADDR_TLS_PRIORITY", "rdmacm", 1);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Comment on lines +311 to +312
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
setenv("D_LOG_STDERR_IN_LOG", "1", 1);
setenv("UCX_SOCKADDR_TLS_PRIORITY", "rdmacm", 1);
setenv("D_LOG_STDERR_IN_LOG", "1", 1);
setenv("UCX_SOCKADDR_TLS_PRIORITY", "rdmacm", 1);

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(style) code indent should use tabs where possible

Comment on lines +311 to +312
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
setenv("D_LOG_STDERR_IN_LOG", "1", 1);
setenv("UCX_SOCKADDR_TLS_PRIORITY", "rdmacm", 1);
setenv("D_LOG_STDERR_IN_LOG", "1", 1);
setenv("UCX_SOCKADDR_TLS_PRIORITY", "rdmacm", 1);


/* This is a workaround for CART-871 if universe size is not set */
d_getenv_int("FI_UNIVERSE_SIZE", &fi_univ_size);
if (fi_univ_size == 0) {
Expand Down
2 changes: 1 addition & 1 deletion src/tests/ftest/launch.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@
[
("cxi", "ofi+cxi"),
("verbs", "ofi+verbs"),
("ucx", "ucx+dc_x"),
("ucx", "ucx+rc_x"),
("tcp", "ofi+tcp"),
("opx", "ofi+opx"),
]
Expand Down
1 change: 0 additions & 1 deletion src/tests/ftest/mdtest/small.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ timeout: 360
server_config:
name: daos_server
engines_per_host: 2
crt_timeout: 60
engines:
0:
pinned_numa_node: 0
Expand Down
2 changes: 1 addition & 1 deletion src/tests/ftest/util/network_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from exception_utils import CommandFailure
from general_utils import run_task, display_task, run_pcmd

SUPPORTED_PROVIDERS = ("ofi+sockets", "ofi+tcp;ofi_rxm", "ofi+verbs;ofi_rxm", "ucx+dc_x", "ofi+cxi")
SUPPORTED_PROVIDERS = ("ofi+sockets", "ofi+tcp;ofi_rxm", "ofi+verbs;ofi_rxm", "ucx+rc_x", "ofi+cxi")


class NetworkDevice():
Expand Down