Steam notifications crash gamescope when the output size is set too large

[ThomasPrzybylinski]

Is there an existing issue for this?

• I have searched the existing issues

Are you using any gamescope patches or a forked version of gamescope?

• The issue occurs on upstream gamescope without any modifications

Current Behavior

Gamescope mostly works fine for me, but Steam notifications cause it to crash for me, both in Steam and during gameplay (e.g. achievements).

Fiddling around it seems like it has to do with setting an output size too large. My monitor and desktop resolution is 1440p, so that's what I normally set gamescope to, but it stops crashing if I set it smaller (largest I tried that worked was 1340p)

Steps To Reproduce

1. Launch steam gamescope -h 1440 -e -- steam (or the resolution of the desktop/monitor, I presume)
2. Download some game (the smaller the better)
3. Wait for it to finish
4. The notification sound will appear, but gamescope will immediately crash

Hardware information

- Distro: Ubuntu 24.10 (actually Kubuntu)
- CPU: AMD 9700X
- GPU: NVIDIA 4070 TI Super
- Driver Version: 560.35.03

Software information

- Desktop environment: 6.1.5
- Session type: wayland
- Gamescope version: gamescope version 3.16.1 (gcc 14.2.0) [Gamescope was compiled locally]
- Gamescope launch command(s): gamescope -h 1440 -e -- steam

Which gamescope backends have the issue you are reporting?

• Wayland (default for nested gamescope)
• DRM (default for embedded gamescope, i.e. gamescope-session)
• SDL
• OpenVR

Logging, screenshots, or anything else

Example run (this one might be running at a weird resolution like 1399)
Console output: gamescope.txt
Zip of core dump: core.zip (I don't think this had debugging symbols, so may be of limited use)

[matte-schwartz]

Since this involves Steam, it may be helpful to include Steam's logs folder as well. Grabbed this command from the steam-for-linux template: tar -zcvf ~/Desktop/steam-logs.tar.gz ~/.steam/steam/logs

[ThomasPrzybylinski]

steam-logs.tar.gz
It looks like the run that's referenced by the prior attachments occurred at 2025-01-02 20:49:37, but I was also doing a lot test runs around that time as well.

[Quince-Pie]

interestingly, notifications on big picture mode do not cause the crash on my machine but they do on normal mode.

[matte-schwartz]

@ThomasPrzybylinski So this is likely the point at which it crashes for you based off the timeline you provided:

[2025-01-02 20:50:28] SP Shared JS Context-'SharedJSCo': CreatingPopup name:notificationtoasts_1_desktop browser:65536 parent:0 pid:18399 type:4 flags:5534570: (-2147483648.00, -2147483648.00) 283.00x70.00: url:about:blank?createflags=5534474&browserType=4
[2025-01-02 20:50:28] Browser requested transparent background, but it is not supported
[2025-01-02 20:50:28] CBrowserComposerSystem::CreateOutputWindow: Creating browser window at: 805240832,805240832 size: 283x70
[2025-01-02 20:50:28] notificationtoasts_1_desktop: Created window: size: 283,70 pos: 805240832,805240832 mode: System window: 0x1e0009f
[2025-01-02 20:50:28] notificationtoasts_1_desktop: AfterCreated handle:1048590 type:4: (0, 0) 283x70
[2025-01-02 20:50:28] Browser - launching child process with: /proc/self/exe --type=utility --utility-sub-type=audio.mojom.AudioService --lang=en-US --service-sandbox-type=none --user-agent-product=Valve Steam Client --lang=en_US.UTF-8 --user-data-dir=/home/goldencoal/.steam/debian-installation/config/htmlcache --crashpad-handler-pid=18657 --buildid=1733265492 --steamid=0 --valve-initial-threadpool-size=4
[2025-01-02 20:50:28] notificationtoasts_1_desktop-'notificati': WasHidden 0: (0, 0) 283x70

I'm no Steam expert but I'm not really seeing anything in your logs within Steam to indicate it's a Steam crash.

If possible, it would be helpful to get a backtrace with symbols for gamescope, at least until I get home in a couple weeks and try to repro this. If your distro supports debuginfod, you should be able to use coredumpctl list, find the gamescope coredump, coredumpctl debug <coredump number>, and then choose yes for debuginfod symbols once gdb starts loading (if you don't have gdb you have to install it first). After that, once you're in gdb, you can run thread apply all bt and copy the output.

@Quince-Pie I'm assuming you're also on NVIDIA? Or what is your setup like?

[matte-schwartz]

hmmm, actually, looking at webhelper-linux.txt I see:

[2025-01-02 20:50:28] [0102/205028.651454:ERROR:atom_cache.cc(229)] Add _NET_WM_STATE_KEEP_ABOVE to kAtomsToCache
[2025-01-02 20:50:28] [0102/205028.793936:WARNING:crash_reporting.cc(271)] Failed to set crash key: UserID with value: 0
[2025-01-02 20:50:28] [0102/205028.793970:WARNING:crash_reporting.cc(271)] Failed to set crash key: BuildID with value: 1733265492
[2025-01-02 20:50:28] [0102/205028.793972:WARNING:crash_reporting.cc(271)] Failed to set crash key: SteamUniverse with value: Public
[2025-01-02 20:50:28] [0102/205028.793973:WARNING:crash_reporting.cc(271)] Failed to set crash key: Vendor with value: Valve
[2025-01-02 20:50:28] [0102/205028.793974:WARNING:crash_reporting.cc(271)] Failed to set crash key: Platform with value: Linux
[2025-01-02 20:50:28] [0102/205028.794327:INFO:crash_reporting.cc(238)] Crash reporting enabled for process: utility

this doesn't seem to be relevant to the crash either looking closer at your logs

[Quince-Pie]

@matte-schwartz

I am on AMD

- Distro: NixOS
- CPU: Intel i7-10700K 
- GPU: AMD Radeon RX 6800 XT
- Version: [gamescope] [Info]  console: gamescope version 3.15.13 (gcc 13.3.0)

gdb.txt
steam-logs.tar.gz

Update: debug build bt:
gdb.txt

[matte-schwartz]

thanks, I see the crash on my own ROG Ally X as well now with gamescope -h 1440 -- steam. I went as far back as Gamescope 3.14.29 and the crash was still happening there. Notably, when running gamescope with WAYLAND_DEBUG=1 gamescope -h 1440 -- steam, I get a suspicious looking error geometry right before the crash:

[1239232.639] {Display Queue} wl_display#1.delete_id(103)
[1239232.643] {Display Queue} wl_display#1.error(wp_viewport#35, 0, "invalid source geometry")
[1239237.627] {Default Queue}  -> wl_surface#26.attach(wl_buffer#42, 0, 0)
[1239237.647] {Default Queue}  -> wl_surface#26.damage_buffer(0, 0, 283, 70)
[1239237.653] {Default Queue}  -> wl_surface#26.frame(new id wl_callback#67)
[1239237.732] {Default Queue}  -> wl_surface#26.commit()
[1239237.780] {Default Queue}  -> wl_surface#50.attach(wl_buffer#57, 0, 0)
[1239237.787] {Default Queue}  -> wl_surface#50.frame(new id wl_callback#60)
[1239237.791] {Default Queue}  -> wl_surface#50.damage_buffer(0, 0, 1515, 900)
[1239237.794] {Default Queue}  -> wl_surface#50.commit()
[1239247.704] {Default Queue}  -> wl_buffer#29.destroy()
[1239247.786] {Default Queue}  -> zwp_linux_dmabuf_v1#5.create_params(new id zwp_linux_buffer_params_v1#38)
[1239247.795] {Default Queue}  -> zwp_linux_buffer_params_v1#38.add(fd 33, 0, 0, 1536, 33554432, 273070852)
[1239247.806] {Default Queue}  -> zwp_linux_buffer_params_v1#38.add(fd 34, 1, 196608, 512, 33554432, 273070852)
[1239247.809] {Default Queue}  -> zwp_linux_buffer_params_v1#38.create_immed(new id wl_buffer#61, 283, 70, 875713089, 0)
[1239247.812] {Default Queue}  -> zwp_linux_buffer_params_v1#38.destroy()
[1239248.215] {Default Queue}  -> zwp_linux_dmabuf_v1#5.create_params(new id zwp_linux_buffer_params_v1#58)
[1239248.224] {Default Queue}  -> zwp_linux_buffer_params_v1#58.add(fd 33, 0, 0, 1536, 33554432, 273070852)
[1239248.228] {Default Queue}  -> zwp_linux_buffer_params_v1#58.add(fd 34, 1, 196608, 512, 33554432, 273070852)
[1239248.231] {Default Queue}  -> zwp_linux_buffer_params_v1#58.create_immed(new id wl_buffer#62, 283, 70, 875713089, 0)
[1239248.235] {Default Queue}  -> zwp_linux_buffer_params_v1#58.destroy()
[1239248.239] {Default Queue}  -> wl_surface#26.attach(wl_buffer#62, 0, 0)
[1239248.242] {Default Queue}  -> wl_surface#26.damage_buffer(0, 0, 283, 70)
[1239248.244] {Default Queue}  -> wl_surface#26.commit()

@misyltoad any ideas on what could be going on here? I'm pretty sure you should be able to repro this on SteamOS Main w/ Plasma Wayland. I tried all our standard ConVar options (disabling explicit sync, no DMA-BUF modifiers, surface_update_force_only_current_surface) and none of them seem to make a difference. I also tried @sharkautarch's various branches for the hangs we've been working on without any success.

[sharkautarch]

Thread 1 (Thread 0x7f503adf26c0 (LWP 144916)):
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007f503cfb2b03 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2  0x00007f503cf60576 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007f503cf48935 in __GI_abort () at abort.c:79
#4  0x0000000000421e08 in gamescope::CWaylandInputThread::ThreadFunc (this=0xaaa2d10) at ../src/Backends/WaylandBackend.cpp:2374
#5  0x0000000000421f39 in operator() (__closure=<optimized out>) at ../src/Backends/WaylandBackend.cpp:2293
#6  std::__invoke_impl<void, gamescope::CWaylandInputThread::CWaylandInputThread()::<lambda()> > (__f=...) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/invoke.h:61

Mmm looks like there’s some issue that’s making the Wayland input thread call abort()

[sharkautarch]

I vaguely remember someone w/ a custom Wayland compositor complaining about gamescope crashing when gamescope was given a {0,0} geometry or something
Not sure if this is the same issue
Now, I’ll admit it seems weird that a {0,0} geometry would be causing the input thread to crash
But, I think the reason the Wayland input thread would be crashing from it, is if gamescope mis-handles a {0,0} geometry in a way that causes a Wayland protocol error, and then that error causes the Wayland queue or display used by the Wayland input thread to be invalid.
If the Wayland queue or display becomes invalid, then the wayland input thread would have to abort

[sharkautarch]

AHHH yes so the message
[1239232.643] {Display Queue} wl_display#1.error(wp_viewport#35, 0, "invalid source geometry")
makes sense now
wayland protocol spec for wp_viewport::set_source says:

If all of x, y, width and height are -1.0, the source rectangle is unset instead. Any other set of values where width or height are zero or negative, or x or y are negative, raise the bad_value protocol error.

Note the 0 in wl_display#1.error(wp_viewport#35, 0, "invalid source geometry")
the documentation for wp_viewport::error says for error value 0:

bad_value 0 negative or zero values in width or height

[sharkautarch]

@ThomasPrzybylinski @matte-schwartz @Quince-Pie
I just pushed out a branch with a fix for the protocol error:
https://github.com/sharkautarch/gamescope/tree/steam_notification_crash_fix_attempt

Let me know if that fixes all of the steam notification issues, and if so, I will make a PR to fix this issue

[ThomasPrzybylinski]

@sharkautarch
That seemed to have fixed it for me with the caveat that I changed:
assert( oState.pBuffer ); at WaylandBackend.cpp line 1011 to assert( sanitizedPlane.pBuffer ); (which seems to match the rest of the changes you made) in order to fix an error I got when compiling the branch.

Thanks!

[sharkautarch]

Good to hear that the branch fixes the crash
I've just updated it now to fix that compiler error

I heard from matt that he also says it fixes the crash, but apparently the steam notifications are still invisible, so I would like to also try to address that as well, if I can, before making a PR with the fix

[sharkautarch]

I made another branch in an attempt to fix the remaining issue w/ the steam notifications window being invisible:
https://github.com/sharkautarch/gamescope/tree/steam_notification_crash_fix_attempt_v2
Let me know if that branch fixes both the issue w/ crashing and the issue w/ the steam notification window being invisible

[ThomasPrzybylinski]

The _v2 is crashing for me again, unfortunately.

Running with WAYLAND_DEBUG=1, I assume this error is the reason:
[1348927.736] {Display Queue} wl_display#1.error(wp_viewport#35, 0, "invalid destination size")
Full log: log.txt

Just for the record, I was able to see the notifications show up in the previous branch.

[sharkautarch]

The _v2 is crashing for me again, unfortunately.

Running with WAYLAND_DEBUG=1, I assume this error is the reason:
[1348927.736] {Display Queue} wl_display#1.error(wp_viewport#35, 0, "invalid destination size")
Full log: log.txt

Hmmm weird, because it shouldn't be going against the spec, but maybe the spec is actually wrong?
I just force pushed to the v2 branch, to adjust the behavior of v2 to use the behavior of v1 for dst

Let me know if v2 now works the same as v1 for you

Just for the record, I was able to see the notifications show up in the previous branch.

Hmm interesting, could perhaps be due to matt using steam with a different set of params than you or something

[matte-schwartz]

I'll doublecheck in a bit just to be safe, but all I ran was gamescope -h 1440 -- steam while testing the original branch yesterday on Arch.

[sharkautarch]

@matte-schwartz
Ok
But yeah if you could give the v1 branch another try
And then give the v2 branch a try
And then let me know what the status is for both

[ThomasPrzybylinski]

Let me know if v2 now works the same as v1 for you

Yes. Looks like v2 works correctly for me now.

[sharkautarch]

@ThomasPrzybylinski
Great!
Hopefully steam notifications will show up for matt on the v2 branch
I'll go ahead and make a PR from v2 right now, since even if v2 doesn't fix matt's issues with the steam notification window not being visible, there's nothing I can do about that to my knowledge, and at this point the priority will be to get the fix into upstream gamescope.

[matte-schwartz]

for me, notifications started working once I also specified a width rather than only using gamescope -h 1440 -e -- steam. So yes, V2 seems fine here with gamescope -h 1440 -w 2560 -e -- steam

[sharkautarch]

Misyl, the official valve-contracted gamescope dev (note that she works on other valve projects too), told me that she’s planning on reviewing my pr that fixes this issue today or tomorrow (actually not sure which because of timezone-related ambiguity).
Hopefully, as long as there are no unexpected delays, the fix will make its way into upstream soon, but please be patient, misyl can often have a lot on their plate, especially since, iirc, she’s actually the only official gamescope dev.
As for when you can expect the fix to make its way into a steamdeck update and/or distro packages, well uhh I’m not sure exactly.
I’m just a contributor to gamescope, so don’t quote me on this, but I imagine the fix might at least be cherry-picked into the next steamdeck gamescope branch, since it seems like each steamdeck gamescope branch will have some newer fixes/tweaks cherry-picked into it (ex: https://github.com/ValveSoftware/gamescope/commits/jupiter-3.6/)
As for distro packages, you’ll have to hope they’ll either update to the next gamescope release soon, or that they’ll manually patch in the fix…