Avoid executing ELF files directly

[fornwall]

This is change to work around the Android 10 R^W violation (execution of binary files are not allowed).

See the README for more information.

Helping with testing

Help to test this would be great! For context and more information, see Problem 1 and Solution 1 in the updated README.

NOTE: May totally break your system and need a reinstallation (or require a manual repair in a failsafe session - see below).

Steps (on aarch64 - let me know if you want to test on some other arch):

• Be sure you have updated packages:
  • pkg up
• Take a backup (if this is not a installation that you can reinstall termux quickly):
  • cp $PREFIX/lib/libtermux-exec.so $PREFIX/lib/libtermux-exec.so.bak
• Download and install a test build of termux-exec containing this code:
  • curl -o ~/te-997.deb https://fornwall.me/te-997.deb && apt install --reinstall ~/te-997.deb

After this, new terminal sessions will be using the updated termux-exec version. A way to check this is by running ls -l /proc/self/exe - it should show a symlink pointing to a path ending in bin/linker64, since the linker is what is being executed.

Try things out (run scripts, compile programs, etc) and report if something does not work as a comment in this issue!

This should only affect behaviour on Android 10+ devices, but if you have an old Android version you are welcome to test that as well - that nothing has changed there.

Helping with testing: Debugging tools

• export TERMUX_EXEC_DEBUG=1 to get verbose logging to stderr
  • This log output may interfere with programs
• export TERMUX_EXEC_OPTOUT=1 to opt out from termux-exec - it will not do any modification to execve calls

Helping with testing: Known issues

• Statically linked executables such as zig does not work (linker errors with has unexpected e_type: 2)

Helping with testing: To restore

If things are not completely broken, apt install --reinstall termux-exec=1:1.0 will install the stable version of termux-exec.

If the installation is completely broken, so you can't even start a shell:

• Start a failsafe session by dragging out the drawer from the left, long press on the NEW SESSION button, and click FAILSAFE in the dialog that shows up.
• cd $PREFIX/lib && cp libtermux-exec.so.bak libtermux-exec.so
• You can now start a normal shell session again
  • Restore the termux-exec package properly with: apt install --reinstall termux-exec=1:1.0

Updates

• 2023-10-17 (file te-997.deb, apt show termux-exec should show version 9:9.7):
  • Various fixes to edge cases and logging
  • Introduce TERMUX_SELF_EXE, the absolute path to the executed file, which can be used as a /proc/self/exec replacement.
• 2023-10-05
  • Various fixes to edge cases and logging

[agnostic-apollo]

Very interesting solution. But yeah, it likely won't be acceptable by play store policies, but could be tried. Although not sure if android could add further selinux restrictions to prevent this.

Btw are you sure in testing, you uninstalled all termux apps, rebooted, and reinstalled, since otherwise untrusted_app_27 selinux process would still be assigned to the app, you can confirm with Termux in-app settings -> About, just making sure.

The executable being /system/bin/linker64 issues may be significant though.

[agnostic-apollo]

Do not use hardcoded com.termux values here and elsewhere, it affects termux forks. To get prefix, first check TERMUX__PREFIX env variable, then PREFIX, otherwise default to package build time path @TERMUX_BASE_DIR@. In future, multiple rootfs support may also be added and $PREFIX has been deprecated in favour of TERMUX__PREFIX (note double dash as scope), as it conflicts with other packages like npm, etc. Haven't pushed change to termux-app yet.

[fornwall]

Thanks - I'll update!

[fornwall]

Changed to look at TERMUX__PREFIX now, with a TERMUX_BASE_DIR value given at compilation time as fallback.

I think PREFIX is too dangerous to use, as you say it can totally break when reused for other purposes.

[agnostic-apollo]

Thanks. But do not use literal com.termux value. The whole path @TERMUX_BASE_DIR@ should be replaced during build time in the source by value set in properties.sh. Possibly tests should use this too.

https://github.com/termux/termux-packages/blob/26b05538619ffd93cfd52f252f34d1c025d51be7/scripts/properties.sh#L34

For inspiration, check termux-tools design by grimler.

https://github.com/termux/termux-tools/blob/master/configure.ac

https://github.com/termux/termux-tools/blob/master/Makefile.am

Also do not limit prefix length to 48, package names can be much longer. For example /data/user/10/net.dinglisch.android.taskerm/files will fail with length 49. Even /data/data paths can be longer. Package name alone on playstore can be of length 50. Probably just check if it starts with / since technically packages can be compiled for any prefix, even /data/local/tmp for adb or /mnt paths for external sd.

[agnostic-apollo]

Yeah, good idea not to use $PREFIX, will have to push TERMUX__PREFIX export to termux-app as well.

[fornwall]

But do not use literal com.termux value

With the current version (I made changes and squashed commits): We start by using the TERMUX__PREFIX env variable now. If that does not exist, we fall back to TERMUX_BASE_DIR.

TERMUX_BASE_DIR is given at compilation time, as in make TERMUX_BASE_DIR=/a/path/to/package/prefix (the
It's not really hardcoded any longer, but supplied at compile time (the existing package already supplies that at build time: https://github.com/termux/termux-packages/blob/master/packages/termux-exec/build.sh#L10).

Remaining usage of com.termux are ok I think:

• In Makefile, as a default value if make is executed without any argument
• In termux-exec.c, but only inside #ifdef UNIT_TEST, so only used for testing. We also test with other prefixes as well in the tests.
• In test-program.c, which is just a utility test program that can be run manually, not part of the package

[fornwall]

Thanks for your thorough review and feedback @agnostic-apollo !

[agnostic-apollo]

Will check code in detail later, outside, but if app is installed on adoptable external sd, then its app data parent directory prefix will be /mnt/expand/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/user/0, which alone has length 55, so 100 is not going to be enough, especially if multiple sub dirs under package name.

You are very welcome. :)

[fornwall]

so 100 is not going to be enough

Thanks again! So what about 200, that should be enough for everyone perhaps :)? That allows a path such as /mnt/expand/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/user/0/yyyyyyyyyy-yyyyyyyyyy/yyyyyyyyyy-yyyyyyyyyyyyyyyyyyyy/yyyyyyyyyyyyyyyyyyyy/yyyyyyyyyyyyyyyyyyyy-yyyyyy yyyy/xxxxx-xxxxx/yyyyy-yyyyy/zzzzzz-zzzzz/.

The reason I don't want an arbitrary big length is that it would be good to have a reasonable tight limit - this code is going to be injected in a lot of places, so not taking up more stack or heap space than necessary would be great, to decrease risks of interfering with the surrounding problem.

[agnostic-apollo]

lolz technically the only valid length is PATH_MAX(4096 bytes), but that would be for the full path, not just the prefix. That is also what libraries including mine use as the standard for passing to sys calls. You could reduce it if necessary, but couple of hundred bytes won't matter, but at least log a warning if value is ignored.

[sylirre]

But yeah, it likely won't be acceptable by play store policies, but could be tried.

As Play Store accepts apps that run binaries by proot and have target SDK 29+, I doubt that would be an issue. Maybe they consider the whole purpose of app (e.g. terminal emulators run binaries and need exec but games do not), who knows...

[agnostic-apollo]

I think it may be more likely that they don't know proot is being used, otherwise if someone took interest, proot and this clearly does violate the policy.

An app distributed via Google Play may not modify, replace, or update itself using any method other than Google Play's update mechanism. Likewise, an app may not download executable code (such as dex, JAR, .so files) from a source other than Google Play. This restriction does not apply to code that runs in a virtual machine or an interpreter where either provides indirect access to Android APIs (such as JavaScript in a webview or browser).
Apps or third-party code, like SDKs, with interpreted languages (JavaScript, Python, Lua, etc.) loaded at run time (for example, not packaged with the app) must not allow potential violations of Google Play policies.

https://support.google.com/googleplay/android-developer/answer/9888379?hl=en#zippy=%2Cexamples-of-common-violations

[fornwall]

Btw are you sure in testing, you uninstalled all termux apps, rebooted, and reinstalled, since otherwise untrusted_app_27 selinux process would still be assigned to the app, you can confirm with Termux in-app settings -> About, just making sure.

The executable being /system/bin/linker64 issues may be significant though.

Thanks, I noticed the behaviour you described as well, was really confusing first! But now I can confirm that I am indeed testing with effective targetSdk of 34! I'll share a branch of termux-app for testing shortly.

[fornwall]

I think we should focus on the technical issue here, and leave Google Play policy out of the discussion, since I think that's a separate one. This change is hopefully good on its own, for bumping targetSdk also for distribution outside of Google Play.

[agnostic-apollo]

Thanks, I noticed the behaviour you described as well, was really confusing first! But now I can confirm that I am indeed testing with effective targetSdk of 34! I'll share a branch of termux-app for testing shortly.

Welcome. Android also sometimes assigns latest target sdk SE_INFO, even if using older target sdk, and requires a reboot to fix. Have added checks for that too locally. I am far ahead of termux-app master locally, most of the non-terminal stuff has been rewritten/refactored, will be pushing in coming weeks when I manage to complete the changes.

leave Google Play policy out of the discussion

Fair enough. targetSdkVersion is far important, I also don't care too much about playstore either.

[twaik]

That solution may be problematic. Linker on old Android version does not allow to do this. But I am not sure when it starts.

[chenxiaolong]

That solution may be problematic. Linker on old Android version does not allow to do this. But I am not sure when it starts.

Looks like it was introduced in https://android.googlesource.com/platform/bionic/+/8f639a40966c630c64166d2657da3ee641303194, which was first included in Android 8.

[Grimler91]

That solution may be problematic. Linker on old Android version does not allow to do this. But I am not sure when it starts.

Looks like it was introduced in https://android.googlesource.com/platform/bionic/+/8f639a40966c630c64166d2657da3ee641303194, which was first included in Android 8.

Only supporting android >= 8 (or even a version or two higher) for an app re-introduced to google play seems reasonable to me

[sylirre]

Android 10+ solution is not needed to be used on older versions. It can be enabled on-demand, possibly with letting user to opt-out from this solution. Don't forget that:

• Not everyone uses Android 10 or higher there. Some people actually use Termux to make obsolete devices useful, e.g. to run home server.
• Some people have SELinux in permissive mode, whether intentionally or because custom ROM doesn't support enforcing. In this case exec restriction doesn't work.

Though I found one problem with this solution: users will no longer be able to run static binaries. When I attempt to run static binary, I get error unexpected e_type: 2. Can't remember any Termux package using static binaries, but sideloaded third-party prebuilt software definitely will no longer run without proot.

[agnostic-apollo]

I have pushed the new changes to https://github.com/termux/termux-exec/tree/2.0.0-prerelease

@fornwall you can push them yourself to this pull branch by the running following commands from this repo. If we merge my changes from this pull, then it will require changing commit hash in a commit message and docs and will need to merged locally with fast forward.

git checkout master
git reset --hard f3ae554e
git remote add upstream [email protected]:termux/termux-exec.git
git fetch upstream 2.0.0-prerelease
git checkout -b 2.0.0-prerelease upstream/2.0.0-prerelease
git checkout master
git merge --ff-only 2.0.0-prerelease
git push --force origin master

New Changes Since Last Push

• For system_linker_exec, executable or interpreter paths will now be checked under TERMUX_APP__DATA_DIR or TERMUX_APP__LEGACY_DATA_DIR instead of TERMUX__ROOTFS as files could be executed from TERMUX__APPS_DIR and TERMUX__CACHE_DIR, which are not under the Termux rootfs. Additionally, Termux rootfs may not exist under app data directory at all and could be under another directory under Android rootfs /, like if compiling packages for shell user for the com.android.shell package with the Termux rootfs under /data/local/tmp instead of /data/data/com.android.shell (and using force mode) or compiling packages for /system directory.
• The $TERMUX__PREFIX (and $TERMUX_APP__DATA_DIR) environment variables are now lazy read only if they are actually needed instead of being always read.
• Read ANDROID__BUILD_VERSION_SDK environment variable exported by Termux app to get Android build version sdk, and if its not set, then from the android_get_device_api_level() call provided by <android/api-level.h>, which gets it from the system properties, which should be slower (untested).
• Use find to build or move all matched files instead of specifying them individually.
• Create or move all build files under the build/ directory. It's better to build a directory structure for the files to be added to prefix under the build/usr directory than have a mix of using files under both src/ and build/ during installation. This way src/ directory also doesn't get modified and will not contain the files with replaced constants that were created from *.in files. This also makes it easier to clean and build the deb since all built files exist under the same directory, which termux-create-package can also use with source_recurse.
• The recursive replacement of termux constants is done with find -exec sed with TERMUX_CONSTANTS_SED_ARGS passed to it. Using foreach with the call function for replace-termux-constants does not work for some reason if multiple files exists, because somehow function definition itself like @sed gets passed to sed command as argument. So replace-termux-constants is only called for individual files. The sed replacements surround arguments with double quotes instead of single quotes, so it could potentially break on more cases during shell expansion, although regex variables weren't being escaped before either.
• The termux-exec-package.json will now have correct version and be consistent with build.sh, and also include tests files, which wasn't being done before.
• The termux-exec-package.json previously had hardcoded aarch64 as architecture, now we find and replace it for the compiler based on which predefined architecture macro is defined.
• There was a typo in default TERMUX_API_APP_ scope used for version name environment variable that was preventing termux-api tests to run when termux-exec was built with make manually. @Grimler91 reported this earlier.
• Use TERMUX_EXEC_PKG__VERSION instead of TERMUX_EXEC__VERSION.
• Added termux-core, tudo and sudo tests.
• Added tests for Termux app data and prefix directory, and checking if path is under Termux app data directory.
• Changed termux-exec-tests scope from TERMUX_EXEC_TESTS__ to TERMUX_EXEC__TESTS__.
• Using new design for reading TERMUX*__ scoped variables in termux-exec-tests. Previously, the way build time placeholders @TERMUX_VAR@ were being used inside shell code everywhere would break shell syntax checkers and highlighters and made code harder to read. Now the required variables from the scope set in TERMUX_ENV__S_* (like FOO_ instead of default TERMUX_) are read into the TERMUX*__ scoped variables with the same variable sub name at the start of the script, and then the TERMUX*__ scoped variables are used afterwards where ever they need to be referenced, like FOO__PREFIX may be copied to TERMUX__PREFIX. The termux-scoped-env-variable from the termux-core package does not need to be used to get the variable values and names as termux-exec is built with our build infrastructure in which @TERMUX_ENV__S_*@ can be replaced during build time. The TERMUX_API_APP__APP_VERSION_NAME is read from the termux-apps-info.env file using termux-apps-app-version-name.bash, the file isn't created yet by Termux app.

 

Check termux-scoped-env-variable, termux-apps-info-env-variable and termux-apps-info-app-version-name utils docs added in termux-core package for more info on new scoped design. I need to implement the new scoped environment in Termux app, haven't done that yet, things had to be finalized first, I seem satisfied now.

 

• The build docs have been updated to use the new local source building design added in bump|addpkg(main/{termux-core,termux-exec,tudo,sudo}) termux-packages#18872. Check Build Local Package and Package Build Local Source URLs docs for more info on how to building packages from local sources.

 

• The Termux Execution Environment docs, updated Termux Filesystem Layout docs and other building packages docs were uploaded a few backs as well.

 

Run tests with TERMUX_EXEC__LOG_LEVEL=1 TERMUX_API_APP__APP_VERSION_NAME=0.50.1 TERMUX_ROOTFS__PACKAGE_MANAGER=apt "${TERMUX__PREFIX:-$PREFIX}/libexec/installed-tests/termux-exec/termux-exec-tests" -vv all.

Install the termux-exec, and optionally the termux-core, tudo and sudo packages from termux/termux-packages#18872 action run or build them locally, build.sh files have comments for local file:// builds.

Exporting TERMUX_API_APP__APP_VERSION_NAME and TERMUX_ROOTFS__PACKAGE_MANAGER variables is necessary for running tests on older Termux app versions without scoped environment variable changes for package manager and Termux:API app tests to run.

To only run termux-exec package specific tests and not other Termux packages/commands, pass --only-termux-exec-tests.
To skip termux-core package tests, pass --skip-termux-core-tests, as under normal circumstances with Termux app in foreground, tests take ~5min to run depending on device as there are 8708 test variants that are run. Older devices will take longer.
If running sudo package has been installed and its tests are run, then sometimes stdout is returned in an unsynchronized manner which fails the validation, rerun the tests and they should pass.

 

Respective pulls/releases for termux-exec, termux-core, tudo and sudo packages are at following links. tudo and sudo will be provided by Termxu repositories now and sudo will replace tsu. All these packages and termux-exec support Android 5/6 as well now.

• termux-packages

  • bump|addpkg(main/{termux-core,termux-exec,tudo,sudo}) termux-packages#18872
  • bump|addpkg(main/{termux-core,termux-exec,tudo,sudo}) for android-5 termux-packages#22629

• termux-exec

  • https://github.com/termux/termux-exec/blob/2.0.0-prerelease/site/pages/en/projects/releases/2/v2.0.0.md

• termux-core

  • Release: 0.1.0 prerelease termux-core-package#1
  • https://github.com/termux/termux-core-package/blob/0.1.0-prerelease/site/pages/en/projects/releases/0/v0.1.0.md

• tudo

  • Release: 1.0.0 prerelease agnostic-apollo/tudo#3
  • https://github.com/agnostic-apollo/tudo/blob/1.0.0-prerelease/site/pages/en/projects/releases/1/v1.0.0.md

• sudo

  • Release: 1.0.0 prerelease agnostic-apollo/sudo#7
  • https://github.com/agnostic-apollo/sudo/blob/1.0.0-prerelease/site/pages/en/projects/releases/1/v1.0.0.md

 

The primary issue currently remaining is that Pixel 7 Pro on Android 14 has some unknown performance problem. It takes ~26min to run the same termux-core 8708 tests that run in 3-5mins on AVD or older devices. The issue is not termux-exec as with all the changes added in this pull, there has been no performance loss as per tests compared to 1.0. Additionally, I tried the tests with 1.0 as well with targetSdkVersion=28 and it took the same time. Also tried with selinux disabled.

Although, interestingly, tests take longer on each Android version increment tested.

Not sure what the issue is. termux-exec, system_linker_exec, targetSdkVersion and selinux don't seem to be it. External storage is never used so that can't affect time with all its emulation. It would be good if others test it on devices with newer/latest Android versions, maybe its just performance regression that has already been fixed, my Pixel 7 Pro has security patch from Jan 2024, I could update to new patches or Android 15, but maybe would be good to find the cause before updating.

A few tests were added after some of the below tests were already run, so there can be few second variance in test time, although that already happens on reruns.

Android 14 aarch64 (Pixel 7 Pro):
termux-exec 1.0 (direct, targetSdkVersion=28): 24min, 58s
termux-exec 2.0.0 (direct, targetSdkVersion=28): 26min, 49s
termux-exec 2.0.0 (system_linker_exec, targetSdkVersion=35): 27min, 23s
termux-exec 2.0.0 (direct, selinux_disabled, targetSdkVersion=35): 27min, 23s

Android 15 x86_64 (AVD):
termux-exec 1.0 (direct, targetSdkVersion=28): 4min, 7s
termux-exec 2.0.0 (direct, targetSdkVersion=28): 3min, 58s
termux-exec 2.0.0 (direct, root, targetSdkVersion=28): 4min, 14s
termux-exec 2.0.0 (system_linker_exec, targetSdkVersion=28): 3min, 44s
termux-exec 2.0.0 (system_linker_exec, targetSdkVersion=35): 3min, 44s //fixme

Android 11 aarch64 (Poco F2 Pro):
termux-exec 1.0 (direct, targetSdkVersion=28): 5min, 24s
termux-exec 2.0.0 (direct, targetSdkVersion=28): 5min, 34s
termux-exec 2.0.0 (system_linker_exec, targetSdkVersion=28): 5min, 37s

Android 11 x86 (AVD):
termux-exec 2.0.0 (direct, targetSdkVersion=28): 3min, 5s

Android 7 aarch64 (LG G5):
termux-exec 2.0.0 (direct, targetSdkVersion=28): 14min, 17s

Android 7 arm (AVD, emulated on x86_64 host):
termux-exec 2.0.0 (direct, targetSdkVersion=28): 46min, 29s

Android 6 x86_64 (AVD):
termux-exec 2.0.0 (direct, targetSdkVersion=28): 1min, 50s

Android 5 x86_64 (AVD):
termux-exec 2.0.0 (direct, targetSdkVersion=28): 1min, 28s

[agnostic-apollo]

The termux-core, termux-exec, tudo and sudo packages can now be tested from the https://github.com/termux/termux-packages/actions/runs/12453170903?pr=18872 run of the termux/termux-packages#18872 pull for Android >= 7.

[fornwall]

@agnostic-apollo I've invited you to get write permissions to the https://github.com/termux-play-store/termux-exec repository, so you should be able to push to the master branch (from which this PR is created from) - does it seem to work?

[agnostic-apollo]

Looks like that worked, thanks a lot. That should make things easier.