clang-19 built kernel fails several stackinit_kunit tests w. "ASSERTION FAILED at lib/stackinit_kunit.c" vs. same kernel built with gcc-14 passes tests (v6.14-rc3, x86_32)

[ernsteiswuerfel]

Gave v6.14-rc3 a test ride on my Thinkpad T60 and noticed stackinit_kunit tests fail when I build the kernel with clang-19. Same kernel built with gcc-14 passes test.

This is also reproducible on qemu. I used qemu-system-i386 -smp 2 -m 2G -nographic -append "console=ttyS0 root=/dev/sda1" -kernel /boot/vmlinuz-6.14.0-rc3-P3 -hda Debian-VM_p3.img on qemu 9.1.2 on Gentoo Linux with a Debian 12.9 image.

KTAP version 1
1..1
    KTAP version 1
    # Subtest: stackinit
    # module: stackinit_kunit
    1..108
    ok 1 test_u8_zero
    ok 2 test_u16_zero
    ok 3 test_u32_zero
    ok 4 test_u64_zero
    ok 5 test_char_array_zero
    ok 6 test_small_hole_zero
    # test_big_hole_zero: ASSERTION FAILED at lib/stackinit_kunit.c:428
    Expected sum == 0, but
        sum == 124 (0x7c)

uninit bytes: 124
    not ok 7 test_big_hole_zero
    ok 8 test_trailing_hole_zero
    ok 9 test_packed_zero
    ok 10 test_small_hole_old_zero
    # test_big_hole_old_zero: ASSERTION FAILED at lib/stackinit_kunit.c:429
    Expected sum == 0, but
        sum == 124 (0x7c)

uninit bytes: 124
    not ok 11 test_big_hole_old_zero
    ok 12 test_trailing_hole_old_zero
    ok 13 test_packed_old_zero
    ok 14 test_same_sizes_zero
    # test_small_start_zero: ASSERTION FAILED at lib/stackinit_kunit.c:430
    Expected sum == 0, but
        sum == 31 (0x1f)

uninit bytes: 31
    not ok 15 test_small_start_zero
    # test_small_end_zero: ASSERTION FAILED at lib/stackinit_kunit.c:430
    Expected sum == 0, but
        sum == 2 (0x2)

uninit bytes: 2
    not ok 16 test_small_end_zero
    ok 17 test_same_sizes_old_zero
    # test_small_start_old_zero: ASSERTION FAILED at lib/stackinit_kunit.c:431
    Expected sum == 0, but
        sum == 31 (0x1f)

uninit bytes: 31
    not ok 18 test_small_start_old_zero
    # test_small_end_old_zero: ASSERTION FAILED at lib/stackinit_kunit.c:431
    Expected sum == 0, but
        sum == 2 (0x2)

uninit bytes: 2
    not ok 19 test_small_end_old_zero
    # test_small_hole_dynamic_partial: ASSERTION FAILED at lib/stackinit_kunit.c:434
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 20 test_small_hole_dynamic_partial
    ok 21 test_big_hole_dynamic_partial
    # test_trailing_hole_dynamic_partial: ASSERTION FAILED at lib/stackinit_kunit.c:434
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 22 test_trailing_hole_dynamic_partial
    ok 23 test_packed_dynamic_partial
    # test_small_hole_assigned_dynamic_partial: ASSERTION FAILED at lib/stackinit_kunit.c:437
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 24 test_small_hole_assigned_dynamic_partial
    ok 25 test_big_hole_assigned_dynamic_partial
    # test_trailing_hole_assigned_dynamic_partial: ASSERTION FAILED at lib/stackinit_kunit.c:437
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 26 test_trailing_hole_assigned_dynamic_partial
    ok 27 test_packed_assigned_dynamic_partial
    ok 28 test_same_sizes_dynamic_partial
    ok 29 test_small_start_dynamic_partial
    ok 30 test_small_end_dynamic_partial
    ok 31 test_same_sizes_assigned_dynamic_partial
    ok 32 test_small_start_assigned_dynamic_partial
    ok 33 test_small_end_assigned_dynamic_partial
    ok 34 test_small_hole_static_partial
    # test_big_hole_static_partial: ASSERTION FAILED at lib/stackinit_kunit.c:433
    Expected sum == 0, but
        sum == 124 (0x7c)

uninit bytes: 124
    not ok 35 test_big_hole_static_partial
    ok 36 test_trailing_hole_static_partial
    ok 37 test_packed_static_partial
    ok 38 test_small_hole_static_all
    # test_big_hole_static_all: ASSERTION FAILED at lib/stackinit_kunit.c:433
    Expected sum == 0, but
        sum == 124 (0x7c)

uninit bytes: 124
    not ok 39 test_big_hole_static_all
    ok 40 test_trailing_hole_static_all
    ok 41 test_packed_static_all
    # test_small_hole_dynamic_all: ASSERTION FAILED at lib/stackinit_kunit.c:434
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 42 test_small_hole_dynamic_all
    ok 43 test_big_hole_dynamic_all
    # test_trailing_hole_dynamic_all: ASSERTION FAILED at lib/stackinit_kunit.c:434
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 44 test_trailing_hole_dynamic_all
    ok 45 test_packed_dynamic_all
    # test_small_hole_runtime_partial: ASSERTION FAILED at lib/stackinit_kunit.c:435
    Expected sum == 0, but
        sum == 15 (0xf)

uninit bytes: 15
    not ok 46 test_small_hole_runtime_partial
    # test_big_hole_runtime_partial: ASSERTION FAILED at lib/stackinit_kunit.c:435
    Expected sum == 0, but
        sum == 127 (0x7f)

uninit bytes: 127
    not ok 47 test_big_hole_runtime_partial
    # test_trailing_hole_runtime_partial: ASSERTION FAILED at lib/stackinit_kunit.c:435
    Expected sum == 0, but
        sum == 12 (0xc)

uninit bytes: 12
    not ok 48 test_trailing_hole_runtime_partial
    # test_packed_runtime_partial: ASSERTION FAILED at lib/stackinit_kunit.c:435
    Expected sum == 0, but
        sum == 12 (0xc)

uninit bytes: 12
    not ok 49 test_packed_runtime_partial
    # test_small_hole_runtime_all: ASSERTION FAILED at lib/stackinit_kunit.c:435
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 50 test_small_hole_runtime_all
    # test_big_hole_runtime_all: ASSERTION FAILED at lib/stackinit_kunit.c:435
    Expected sum == 0, but
        sum == 124 (0x7c)

uninit bytes: 124
    not ok 51 test_big_hole_runtime_all
    # test_trailing_hole_runtime_all: ASSERTION FAILED at lib/stackinit_kunit.c:435
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 52 test_trailing_hole_runtime_all
    ok 53 test_packed_runtime_all
    # test_small_hole_assigned_static_partial: ASSERTION FAILED at lib/stackinit_kunit.c:436
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 54 test_small_hole_assigned_static_partial
    ok 55 test_big_hole_assigned_static_partial
    # test_trailing_hole_assigned_static_partial: ASSERTION FAILED at lib/stackinit_kunit.c:436
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 56 test_trailing_hole_assigned_static_partial
    ok 57 test_packed_assigned_static_partial
    # test_small_hole_assigned_static_all: ASSERTION FAILED at lib/stackinit_kunit.c:436
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 58 test_small_hole_assigned_static_all
    ok 59 test_big_hole_assigned_static_all
    # test_trailing_hole_assigned_static_all: ASSERTION FAILED at lib/stackinit_kunit.c:436
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 60 test_trailing_hole_assigned_static_all
    ok 61 test_packed_assigned_static_all
    # test_small_hole_assigned_dynamic_all: ASSERTION FAILED at lib/stackinit_kunit.c:437
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 62 test_small_hole_assigned_dynamic_all
    ok 63 test_big_hole_assigned_dynamic_all
    # test_trailing_hole_assigned_dynamic_all: ASSERTION FAILED at lib/stackinit_kunit.c:437
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 64 test_trailing_hole_assigned_dynamic_all
    ok 65 test_packed_assigned_dynamic_all
    ok 66 test_same_sizes_static_partial
    # test_small_start_static_partial: ASSERTION FAILED at lib/stackinit_kunit.c:439
    Expected sum == 0, but
        sum == 31 (0x1f)

uninit bytes: 31
    not ok 67 test_small_start_static_partial
    ok 68 test_small_end_static_partial
    ok 69 test_same_sizes_static_all
    # test_small_start_static_all: ASSERTION FAILED at lib/stackinit_kunit.c:439
    Expected sum == 0, but
        sum == 28 (0x1c)

uninit bytes: 28
    not ok 70 test_small_start_static_all
    # test_small_end_static_all: ASSERTION FAILED at lib/stackinit_kunit.c:439
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 71 test_small_end_static_all
    ok 72 test_same_sizes_dynamic_all
    ok 73 test_small_start_dynamic_all
    # test_small_end_dynamic_all: ASSERTION FAILED at lib/stackinit_kunit.c:440
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 74 test_small_end_dynamic_all
    ok 75 test_same_sizes_runtime_partial
    # test_small_start_runtime_partial: ASSERTION FAILED at lib/stackinit_kunit.c:441
    Expected sum == 0, but
        sum == 31 (0x1f)

uninit bytes: 31
    not ok 76 test_small_start_runtime_partial
    ok 77 test_small_end_runtime_partial
    ok 78 test_same_sizes_runtime_all
    # test_small_start_runtime_all: ASSERTION FAILED at lib/stackinit_kunit.c:441
    Expected sum == 0, but
        sum == 28 (0x1c)

uninit bytes: 28
    not ok 79 test_small_start_runtime_all
    ok 80 test_small_end_runtime_all
    ok 81 test_same_sizes_assigned_static_partial
    ok 82 test_small_start_assigned_static_partial
    ok 83 test_small_end_assigned_static_partial
    ok 84 test_same_sizes_assigned_static_all
    ok 85 test_small_start_assigned_static_all
    # test_small_end_assigned_static_all: ASSERTION FAILED at lib/stackinit_kunit.c:442
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 86 test_small_end_assigned_static_all
    ok 87 test_same_sizes_assigned_dynamic_all
    ok 88 test_small_start_assigned_dynamic_all
    # test_small_end_assigned_dynamic_all: ASSERTION FAILED at lib/stackinit_kunit.c:443
    Expected sum == 0, but
        sum == 3 (0x3)

uninit bytes: 3
    not ok 89 test_small_end_assigned_dynamic_all
    ok 90 test_small_hole_assigned_copy # SKIP XFAIL uninit bytes: 3
    ok 91 test_big_hole_assigned_copy # SKIP XFAIL uninit bytes: 124
    ok 92 test_trailing_hole_assigned_copy # SKIP XFAIL uninit bytes: 3
    ok 93 test_packed_assigned_copy
    ok 94 test_same_sizes_assigned_copy
    ok 95 test_small_start_assigned_copy # SKIP XFAIL uninit bytes: 28
    ok 96 test_small_end_assigned_copy
    # test_u8_none: ASSERTION FAILED at lib/stackinit_kunit.c:446
    Expected sum == 0, but
        sum == 1 (0x1)

uninit bytes: 1
    not ok 97 test_u8_none
    # test_u16_none: ASSERTION FAILED at lib/stackinit_kunit.c:446
    Expected sum == 0, but
        sum == 2 (0x2)

uninit bytes: 2
    not ok 98 test_u16_none
    # test_u32_none: ASSERTION FAILED at lib/stackinit_kunit.c:446
    Expected sum == 0, but
        sum == 4 (0x4)

uninit bytes: 4
    not ok 99 test_u32_none
    # test_u64_none: ASSERTION FAILED at lib/stackinit_kunit.c:446
    Expected sum == 0, but
        sum == 8 (0x8)

uninit bytes: 8
    not ok 100 test_u64_none
    # test_char_array_none: ASSERTION FAILED at lib/stackinit_kunit.c:446
    Expected sum == 0, but
        sum == 16 (0x10)

uninit bytes: 16
    not ok 101 test_char_array_none
    ok 102 test_switch_1_none # SKIP XFAIL uninit bytes: 80
    ok 103 test_switch_2_none # SKIP XFAIL uninit bytes: 80
    # test_small_hole_none: ASSERTION FAILED at lib/stackinit_kunit.c:447
    Expected sum == 0, but
        sum == 16 (0x10)

uninit bytes: 16
    not ok 104 test_small_hole_none
    # test_big_hole_none: ASSERTION FAILED at lib/stackinit_kunit.c:447
    Expected sum == 0, but
        sum == 128 (0x80)

uninit bytes: 128
    not ok 105 test_big_hole_none
    # test_trailing_hole_none: ASSERTION FAILED at lib/stackinit_kunit.c:447
    Expected sum == 0, but
        sum == 16 (0x10)

uninit bytes: 16
    not ok 106 test_trailing_hole_none
    # test_packed_none: ASSERTION FAILED at lib/stackinit_kunit.c:447
    Expected sum == 0, but
        sum == 16 (0x10)

uninit bytes: 16
    not ok 107 test_packed_none
    # test_user: ASSERTION FAILED at lib/stackinit_kunit.c:449
    Expected sum == 0, but
        sum == 16 (0x10)

uninit bytes: 16
    not ok 108 test_user
# stackinit: pass:57 fail:45 skip:6 total:108
# Totals: pass:57 fail:45 skip:6 total:108
not ok 1 stackinit

Full dmesg from my Thinkpad T60 attached, Kernel .config attached.

config_614-rc3_p3-gcc14.txt
config_614-rc3_p3-clang19.txt
dmesg_614-rc3_p3-clang19.log
dmesg_614-rc3_p3-gcc14.log

[nathanchance]

Are you able to test if this reproduces with clang-20? I am curious if these tests are exposing what Yabin fixed for #2033. If not, I can test tomorrow.

cc @kees since he wrote the tests

[nathanchance]

Also, does it happen if you turn off automatic pattern initialization?

[ernsteiswuerfel]

Did that. Interesting results. 🤓

clang 19.1.7 w. INIT_STACK_ALL_PATTERN=y         ...   45 test failures
clang 20.0.0-rc2 w. INIT_STACK_ALL_PATTERN=y     ...   19 test failures
clang 19 or 20 with INIT_STACK_ALL_ZERO=y only   ...   passes tests

dmesg_614-rc3_p3-clang20.log

[nathanchance]

This reproduces easily with

tools/testing/kunit/kunit.py run \
	--arch i386 \
	--kconfig_add CONFIG_INIT_STACK_ALL_PATTERN=y \
	--kconfig_add CONFIG_INIT_STACK_ALL_ZERO=n \
	--make_options LLVM=1 \
	stackinit.'*'

Have I mentioned how much I like the KUnit tooling? :)

Observation: 64-bit architectures appear to all pass but the two 32-bit architectures I can test with clang using KUnit do not.

	clang-19	clang-20
arm	Ran 108 tests: passed: 57, failed: 45, skipped: 6	Ran 108 tests: passed: 83, failed: 19, skipped: 6
arm64	Ran 108 tests: passed: 102, skipped: 6	Ran 108 tests: passed: 102, skipped: 6
i386	Ran 108 tests: passed: 57, failed: 45, skipped: 6	Ran 108 tests: passed: 83, failed: 19, skipped: 6
powerpc	Ran 108 tests: passed: 102, skipped: 6	Ran 108 tests: passed: 102, skipped: 6
riscv	Ran 108 tests: passed: 102, skipped: 6	Ran 108 tests: passed: 102, skipped: 6
s390	Ran 108 tests: passed: 102, skipped: 6	Ran 108 tests: passed: 102, skipped: 6
x86_64	Ran 108 tests: passed: 102, skipped: 6	Ran 108 tests: passed: 102, skipped: 6

Now to figure out if this is a test problem or compiler issue...

[kees]

My educated guess is ... I am^W^Wthe tests are to blame. :)

Specifically, pattern init hasn't gotten a lot of testing in the last couple years as everything moved to zero-init. I have considered actually removing the CONFIG option. But in the meantime, I will take a look at the tests. (Did they pass in Clang 18?)

[nathanchance]

Did they pass in Clang 18?

Does not seem like it:

LLVM 20: Ran 108 tests: passed: 83, failed: 19, skipped: 6
LLVM 19: Ran 108 tests: passed: 57, failed: 45, skipped: 6
LLVM 18: Ran 108 tests: passed: 57, failed: 45, skipped: 6
LLVM 17: Ran 108 tests: passed: 57, failed: 45, skipped: 6
LLVM 16: Ran 108 tests: passed: 57, failed: 45, skipped: 6
LLVM 15: Ran 108 tests: passed: 57, failed: 45, skipped: 6
LLVM 14: Ran 108 tests: passed: 57, failed: 45, skipped: 6
LLVM 13: Ran 108 tests: passed: 67, failed: 35, skipped: 6

[ernsteiswuerfel]

Specifically, pattern init hasn't gotten a lot of testing in the last couple years as everything moved to zero-init. I have considered actually removing the CONFIG option. But in the meantime, I will take a look at the tests. (Did they pass in Clang 18?)

At least INIT_STACK_ALL_PATTERN seems good enough to uncover some corner cases & bugs. 😉 Also the tests pass when GCC is used instead of clang.

If I would have to guess as a non-expert this hints to clang/llvm implementation problems of INIT_STACK_ALL_PATTERN, not to an issue with pattern init itself.