Added the ability for `runcommand` to load the SDL2 gamecontroller mappings created from the EmulationStation mappings and expose them as a SDL2 hint. Since the user may already have a mechanism to set the hint (from `.bashrc`/`.profile` or perhaps as part of the on-start scripts), don't overwrite the SDL2 hint if exists.
NB: since 2.0.22 the file containing the mappings can be set through the `SDL_HINT_GAMECONTROLLERCONFIG_FILE`, but to accomodate older SDL2 version, we're using `SDL_HINT_GAMECONTROLLERCONFIG` as a hint, which contains the actual mapping strings separated by newlines.
There are 2 (uinput) related changes here:
* The SDL2/Uinput based Joy2Key is too slow for Pi1/0 devices, so allow the user to choose the older version. The older version is simpler since it doesn't need or load SDL2/Uinput, thus faster to load on those systems. By default, on ARMv6 devices the old version will be configured.
* The Uinput based event generation of keyboard events is not able to trigger the `runcommand` menu when launching images splash is done via `feh`, since the image display will get always have focus and thus the keybord events will not reach the `runcommand` terminal. So, in order for the `runcommand` menu to work after the splash image, make sure we stop `feh` before launch, just like `fbi` is stopped on non-desktop systems.
Added the SDL12-Compat backend, using a shim library which implements the `sdl1.2` API over SDL2.
The new `sdl12-compat` backend should be used for SDL1 applications when:
* the platform doesn't have `dispmanx` anymore (e.g. `rpi` platforms with RaspiOS Bullseye and later)
* `sdl2` has a working accelerated video & rendering drivers
* not running under `x11` (i.e. KMS)
Note: for SDL1+Videocore specific applications, which use directly the dispmanx/legacy GL stack (e.g. `pcsx-rearmed`, `advmame-1.4`, etc.), the backend will not help.
Replaced the `mesa-drm` scriptmodule and the `modetest` utility with a new program, based on the [kmsxx](https://github.com/tomba/kmsxx) project.
1. Added a new scriptmodule to install the new modesetting utilities:
* kmsblank - blank screen(s)
* kmstest - set modes and planes and show test pattern on crtcs/planes,
and test page flips
* kmsprint(-rp) - print information about DRM objects (connectors, encoders, video modes, crtcs)
* fbtest - show info about the console framebuffer
It's based on https://github.com/tomba/kmsxx, forked to add a custom `kmsprint-rp` utility which we can use in `runcommand` instead of `modetest`.
Advantages over `modetest`:
- it's based on a simpler standalone project and not part of Mesa, less code.
- it's has a modularized C++ API for working with Cards/Encoders/Connectors/CRTCs/etc, modeled after the DRM API entities
- works with any DRM/KMS card and doesn't have any card specific code like `modetest`
- it's faster than modetest (both when KMS is enabled or disabled)
- can print the analog video modes when the 'composite' output is enabled for RPI. I think `modetest` can be modified to support it, but with `kmsxx` is working without any special handling.
Note that a Debian package of the `kmsxx` upstream project is packaged in Rasberry PI OS (as `kms++-utils`), but it's not part of Debian. Packaging it as part of RetroPie helps with:
- integration with `runcommand`, since we can run our own custom query utility
- resolution switching on other KMS platforms
2. Changes to `runcommand`:
- replaced `modetest` with `kmsprint-rp`. The format for the modeline printing has changed a bit, to make it easier to parse and integrate the result in `runcommand`. In addition to the rounded/integral refresh rate, there's a new column to show a decimal refresh rate; the _[p|n][h|v]sync_ flags have been reformatted as _[h|v]sync[-|+]_.
- simplified the modeline parsing for KMS, taking advantage of the new format
- refresh rate (from KMS modeline) is now be a decimal number (.2f), supported by RetroArch
Check if '/dev/dri/card0' exists before running `$KMSTOOL` (now 'modetest') to check for kernel modesettings support.
This avoids 'modetest' being executed and removes almost 2 sec from the `runcommand` runtime on a Pi3.
Add the `xset` commands to disable the display powersave and screen blanking, since gamepad activity is ignored by Xorg and the display will go to sleep if no keyboard/mouse is active.
I removed the script launchers ccbee7c05a, but the `xset` commands haven't been added `XINIT-WM` launchers.
Extend the XINIT: launch prefix with two new ones in order to start a window manager before an x11 application is started.
XINIT-WM: will start a window manager without the cursor
XINIT-WMC: will start a window manager with the cursor
This is similar to how the 'x11' backend starts an application, but without additional configuration set in 'backends.cfg'.
`bash` 5.2 introduced a new option (`patsub_replacement`) which affects how `%ROM%` is replaced in the command line:
From https://tiswww.case.edu/php/chet/bash/NEWS:
> x. New shell option: patsub_replacement. When enabled, a `&' in the replacement
> string of the pattern substitution expansion is replaced by the portion of
> the string that matched the pattern. Backslash will escape the `&' and
> insert a literal `&'.
This breaks the substitution of `%ROM%` when the replacement string (ROM name) contains `&`.
1. with `bash` version 5.1
$ COMMAND="run %ROM%" && COMMAND=${COMMAND//%ROM%/a & b.rom} && echo $COMMAND
run a & b.rom
2. with `bash` version 5.2
$ COMMAND="run %ROM%" && COMMAND=${COMMAND//%ROM%/a & b.rom} && echo $COMMAND
run a %ROM% b.rom
3. with `bash` version 5.2, option disabled
$ shopt -u patsub_replacement
$ COMMAND="run %ROM%" && COMMAND=${COMMAND//%ROM%/a & b.rom} && echo $COMMAND
run a & b.rom
Disable this option in `runcommand` for now so users won't be affected by it when using Ubuntu 22.10+/Debian 12 (bookworm)
Added 2 new parameters for the runtime-generated `retroarch.cfg`:
* `libretro_log_level` will add more debug messages from the libretro core, in addition to the frontend debugging messages.
Cand be useful to diagnose core issues, since `--verbose` activates just the frontend logging.
* `libretro_directory` is set to the core file folder (i.e. `/opt/retropie/libretrocores/lr-<core>`).
It is used by during netplay, as a client, when RetroArch attempts to re-load the core used during netplay.
Basically, when connecting to a remote netplay host, RetroArch will try to locate a `core.info` file and then try to re-load the core file. If any of them is missing, the connection fails with "Couldn't find a suitable core or content file".
The netplay issue was reported and diagnosed in #3161. Rather than symlink each installed libretro core to `$HOME/.config/retroarch/cores` (the default value for `libretro_directory`), we'll just set it at runtime with the above parameter. We still need to add the `.info` files for the installed cores, but that's a separate patch.
Move much of the helpers.sh start/stop logic and default parameters to a joy2key wrapper script.
Switch runcommand.sh to use new wrapper script
Add tab button to "y" key for use in edit dialogs
Remove runcommand $md_inst on update. If old joy2key is present in runcommand install, trigger joy2key module install.
Remove system.sh python3-sdl2 dependency check - moved to joy2key_depends
Switch to letters for items rather than numerical values to avoid gaps and to make it easier to add items to the
menu without having to renumber items etc, which has happened previously (hence most menus excluding
generated lists don't use number ids in RetroPie).
Remove the "Remove emulator/video mode choice" items and replace with an option in the sub-menu for clearing / removing.
Previously when choosing a default emulator for a rom, (or a default video mode for emulator, emulator + rom etc),
an additional menu item was added below to remove the setting. Although this enabled quick access to remove options
from the main menu, it wasn't that intuitive and doesn't match the way menus work elsewhere in RetroPie.
Now when choosing a default emulator / video mode, there is an option to clear it at the top of the list.
The framebuffer menu now uses the resolution as a key rather than a numbered list (this also fixes the default-item
not working). The video and framebuffer dialogs were merged into one function with some additional descriptions
and menu title information also.
While making these changes a couple of bugs were fixed with default-item functionality - Framebuffer resolution
default-item support was broken as mentioned, but also the main menu also lacked this, so the position wouldn't be
remebered after entering a video menu selection etc.
Added 2 fixes:
* reject malformed hat values for an input in the joystick auto-config file(s).
If it's not one of `h<X>up`, `h<X>right`, `h<X>down`, `h<X>left`, reject the input value.
* fix `SDL_GetError` not being properly shown because of the wrong f-string.
`python3.8(+)` throws a warning for using 'is/is not' with with certain literal types [1].
Simplifying the debug message that shows the `python3-sdl2` wrapper version gets rid of this warning, the end result is the same.
[1] https://docs.python.org/3.8/whatsnew/3.8.html#changes-in-python-behavior
Added a new `joy2key` implementation, using PySDL2 for joystick event handling.
PySDL2 is a python module that wraps SDL2 (and other SDL libraries) using the built-in `ctypes` module.
Pros:
* event handling is simplified a bit, using SDL's event loop.
* (subjective) the code is a bit more structured and easy to follow.
* joystick handling is rewritten based on EmulationStation code, movement is smoother and scrolling is improved.
* support for input repeat to improve scrolling, just keep the input pressed and scrolling continues
Cons:
* module is a bit larger (296 LOC vs 224 in current joy2key.py)
* needs PySDL2 (which might not be packaged, see the notes below) and SDL2
* arguably, device config to keyboard event mapping is more complex (abstracted as InputDev)
Notes:
* availability of PySDL2 as a system package is good, but it's not standard in Debian 10 (stable) at the moment.
The module is present though (as a backport) in Ubuntu 18.04 (and later) and Raspberry Pi OS (Buster).
When it will be universally available, we should probably revisit the code that checks for the version and make it a hard requirement in `get_retropie_depends`, similar to `python3-pyudev`.
* added PgUp/PgDown to the default parameters for `joy2key`, they are mapped in a similar fashion to EmulationStation to the shoulder buttons. This doesn't apply to Runcommand's invocation, since it uses a different set of parameters.
* added a configuration option in `runcommand` for selecting the joy2key version used. Default is the SDL version (when PySDL2 is installed), with the ability to fallback to the previous version (udev based).
/dev/shm/runcommand.info is created early on before the runcomand menu, mode switching and variable
replacements have taken place. This means it can contain incorrect information due to a configuration change,
but will also not include some variable replacements like %XRES%, %YRES% etc.
Saving again fixes this. The original earlier save is still done, in case any runcommand-onstart.sh scripts
utilise it.
This replaces the dispmanx module functionality and reworks the way backend configuration is done,
including adding support for choosing to use X11 for modules using sdl1 or sdl2. A new supplementary
module "backends" handles the configuration (replacing the dispmanx module).
On upgrade /opt/retropie/configs/all/dispmanx.cfg is renamed to /opt/retropie/configs/all/backends.cfg
Existing settings in this file will still work, but will be updated when using the backends configuration.
Previously for some modules that use sdl1, a flag "dispmanx" was set to specify that the code worked
with the sdl1 dispmanx driver on videocore/fkms on the RPI.
This has been replaced with the flags "sdl1" "sdl2" and "sdl1-videocore"
The backend module will use the flags to determine which driver backends are available for current system.
For modules such as advmame which use sdl1 on videocore and sdl2 on other systems, the sdl1-videocore flag
can be used which will only apply to the rpi1-3 using the legacy drivers.
The setDispmanx call to default a module to the dispmanx driver has been replaced with setBackend,
but the setDispmanx function is left for 3rd party module compatibility.
Calls to setDispmanx should be replaced with a call such as:
setBackend "$md_id" "dispmanx"
But support for dispmanx should be checked in the module - eg
hasPlatform "dispmanx" && setBackend "$md_id" "dispmanx"
Module hooks for configure_dispmanx_on / conigure_dispmanx_off are no longer used (only used by fuse).
Instead a function _backend_set_ID is called, with the backend to use, and a force parameter.
By default no existing backend configuration is changed when calling setBackend, unless an additional
parameter of 1 is added. This is so a module won't overwrite a user's existing configuration.
If a module is set to launch under dispmanx, runcommand will handle it as before, but will also
now handle launching sdl1 and sdl2 modules under X. It will also run the matchbox-window-manager
which is required by some software.
If switching a module to launch under X it will check for the correct dependencies. If not installed
the user will be asked to confirm installation.
Added sdl1 patch to allow overriding of graphics_get_display_size as it returns 0 on fkms for width and height,
which skips the scaling code and ability to adjust aspect ratio.
New ENV vars are SDL_DISPMANX_WIDTH and SDL_DISPMANX_HEIGHT which we now set manually in runcommand on fkms
when a dispmanx backend is used with sdl1.
This should resolve aspect ratio issues rpi4 users were having with some sdl1 emulators that worked with the dispmanx
backend, but always stretched to fill the entire screen.
This also enables the use of SDL_DISPMANX_RATIO to tweak the ratio when scaling - from 0.2 to 6 - currently
unused in any RetroPie modules by default, but used to be used for tweaking the vice scaling, to more accurately
produce a correct aspect ratio for the C64. But can be used by users manually via emulators.cfg if required.
bumped sdl1 version to force an update - including for Stretch, although this is primarily a buster/rpi4 issue
Before setting the SDL2 env variables that configure the modesetting for DRM/KMS, validate the CrtcId/ModeId tuple against the current list reported by `modetest`. Changes in the kernel DRM driver could make the existing modesetting configuration (stored in `videomodes.cfg`) invalid because of different CrtcId and/or ModeId in new versions of the driver.
When current mode is custom set and DRM doesn't find it via EDID detection, `runcommand` fails to detect the current video mode.
This might happen when:
* EDID is incorrect and user sets a video mode with `hdmi_group/hdmi_mode` or via the kernel parameters.
* new video mode added via `hdmi_cvt`/`hdmi_timings`
Forum posts:
* https://retropie.org.uk/forum/topic/25526/
* https://retropie.org.uk/forum/topic/25834/
* Ubuntu 20.04 will no longer include python-pyudev
* Use stderr for /dev/tty opening error msg
* Use python3 in runcommand module for compilation
* Open with 'append' which was added in f614804f fails with python3 on rpi4 console. Use lower level os.open and O_WRONLY to avoid possible buffer issues. Could not reproduce the ghost inputs the append was added for
* if no TTY var is set (as with the ES launch script), try and get it - otherwise xinit will try and start on a VT we don't have access to
* use a single TTY var name in emulationstation launch script
Certain KMS systems may enumerate multiple CRTCs, some of which have
empty/invalid modes defined. If the first grepped entry happens to be such a
mode, modesetting will break. Fix by using a more accurate grep expression
to find an active CRTC with a valid mode that is marked with the "preferred"
flag.
Issue can be reproduced on Pi 3B with the vc4-kms-v3d overlay.
* replace manual invocations of "xinit" with XINIT: command prefix
* build a custom xinitrc that handles modesetting (via xrandr, setting
a mode matching the resolution and refresh instead of the mode id)
and launch of content.
* if we are already in an Xorg context, the xinitrc will be bypassed,
allowing content prefixed with XINIT: to work in all contexts without
any problem (and still supporting modesetting).
