Questions about SDL2 and graphics backends

[MichaIng]

Hi, I reviewed our (DietPi) Amiberry builds, found the current information here that DispmanX on RPi does not provide performance benefits anymore and the new experimental RPi 4 OpenGLES target. For updating our builds I have some questions, also some more fundamental ones which had im mind since a longer time.

Some questions may be noobish since I never coded GUI/3D software or worked with graphics APIs, so bear with me. I hope it is still okay to ask them here since it is in the context of Amiberry.

1. Does DispmanX still have a notable performance benefit on RPi Buster images with Linux 5.10 and older libraspberrypi0 (compared to Bullseye) (AFAIK) but with latest SDL 2.26.5?
2. How does Amiberry actually use DispmanX vs SDL2? I remember I tested once to compile SDL2 without any graphics backend and Amiberry with DispmanX target, but that did not work. So I guess DispmanX is used additionally for certain types of 2D overlay drawings which is what DMX does an is/was faster at?
3. With the experimental OpenGLES target I guess it is the same: Some drawings are done via OpenGLES directly and others still via SDL2 and whichever graphics backend it was compiled with?
4. Also in SDL2, aside of KMS/DRM, there is either OpenGL or OpenGLES requires, right? I'm not sure anymore but I think I tested with only KMS/DRM once and that did not work, which somehow makes sense at least for 3D applications.
5. Since RPi supports both, GL and GLES, does one have benefits (for SDL2>Amiberry)? Is there any point in enabling both backends (currently I use GLES only on ARM targets, GL on x86).

[midwan]

The alternative to DispmanX going forward, will probably KMSDRM (running from the console). This works well, but has one significant drawback: it cannot use multi-threading for drawing the buffer and then swapping it out (which we can do with Dispmanx but also under X11 and OpenGLES). If you enable it, you will only get a black screen showing.

So, considering the above:

1. Yes, if you compare it to KMSDRM (at least for now), due to multi-threaded drawing. If SDL2 fixes that back-end in a way where multi-threading works, then this situation will change. There's no other benefit to Dispmanx currently.
2. SDL2 is used always for everything else (sound, input handling, events, etc). Dispmanx is used for graphics initialization and drawing. The surfaces used for drawing the pixels are SDL2 surfaces, which are then copied to the framebuffer (or a Texture if using pure SDL2).
3. SDL2 will use whatever back-end it finds available automatically, depending on how it was compiled of course (e.g. KMSDRM used to be disabled by default, but you could compile it with an option to enable it). If we want to use OpenGLES directly, we would also need to make use of the relevant calls to create a GL context and update it when necessary. I do have an experimental version of Amiberry that uses OpenGL(ES) when enabled, and performance-wise it's just as fast as the X11 version (it also supports multi-threading), however the GUI toolkit has some limitations when using it - the TTF fonts cannot be rendered on it, so I had a temporary work-around by using bitmap fonts for that version.
4. Amiberry does not use/need any 3D stuff, so 2D is good enough. We can use KMSDRM, Wayland, X11 and even OpenGL(ES) if available. But for targetting OpenGL(ES) directly, see answer above.
5. The RPI has OpenGLES, see answer above also.

In general, SDL2 should use whatever back-end it finds available automatically for you. You just ask for a Window/Screen and it will try to provide it, falling back to software rendering as the last available option if no hardware-accelerated one is found.
We can use OpenGLES, there is a relevant option in the Makefile, but then the GUI cannot use TTF fonts (I have some bitmap fonts included in the data folder for this reason) and it looks kind of ugly. Performance-wise however, it's as fast as the X11 back-end with multi-threading.

You can enable multi-threading in the amiberry.conf file, the option is named use_sdl2_render_thread: https://github.com/BlitterStudio/amiberry/wiki/Amiberry.conf-options

[MichaIng]

Many thanks for the info 👍.

1. Does this mean that even on Bullseye, the info from Wiki is not entirely true?
   • https://github.com/BlitterStudio/amiberry/wiki/Available-Platforms

   On the latest versions of the KMS Driver, and latest versions of the SDL2 library, there is no more performance benefit in using the Dispmanx version.

2. The surfaces used for drawing the pixels are SDL2 surfaces, which are then copied to the framebuffer (or a Texture if using pure SDL2).

   • I guess this this the reason why there is still a graphics backend required for SDL2 when using an Amiberry DispmanX target.
3. I checked a bit further and this thread sounds like SDL2 indeed uses/can use the KMS/DRM and GLES2 backends both for different tasks: https://discourse.libsdl.org/t/kms-drm-context-for-the-gl-gles-renderer/21813

   SDL2 on the Pi uses dispmanx for EGL window (in this case, KMS/DRM/GBM would be used as the windowing system to provice a context for the GLES tenderer) and then uses GLES2 to render 2D bitmaps to screen.

   • So KMS/DRM for the window, and GLES for painting it (roughly)? 🙈
4. As far as I understood, at least the RPi 4 GPU indeed supports OpenGL 2.1 (non-ES). But there does not seem to be much reason to use it if the software supports GLES as the hardware seems to be tailored for GLES. Hard to find solid information about this as often "OpenGL" term is used when it is actually talked about GLES.

Another question is whether Vulkan would be another beneficial SDL2 graphics backend on RPi 4, if SDL2 uses multiple backends dynamically for different tasks?