Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement VFS functions to load shared libraries from within LUAZIP archives #562

Merged
merged 7 commits into from
Jan 17, 2025

Conversation

rdw-software
Copy link
Member

@rdw-software rdw-software commented Sep 4, 2024

Things left to do:

  • Test on Windows (Should use .dll suffix instead of .so?)
  • Test on macOS (Use of .so should be fine, but what about .dylib?)
  • One of the "unit" tests is actually way too involved; can speed up? If not, move to integration test suite
  • Tests should ensure any temporary files are always cleaned (compare directory tree before/after each call)
  • Cleanup and possibly renames/refactoring - maybe later? Have to use the feature some more to get a feel for it
  • I don't like the ergonomics of having to manually vfs.decode before vfs.dlopen can be used (separate issue)
  • Update the documentation (pending PR, to be merged after the next tagged release is available)

Resolves #488.

@rdw-software
Copy link
Member Author

rdw-software commented Oct 20, 2024

Needs a design review. Not too happy with the API as well; holding off on completing this work. Maybe postpone to next release?

Update: This seems OK, pending a future redesign of the VFS caching that can help eliminate the first argument.

@rdw-software rdw-software force-pushed the 488-vfs-dlopen-dlname branch 5 times, most recently from ca20ea8 to 5521113 Compare January 17, 2025 14:14
@rdw-software rdw-software force-pushed the 488-vfs-dlopen-dlname branch from 5521113 to dbb7970 Compare January 17, 2025 14:21
@rdw-software rdw-software marked this pull request as ready for review January 17, 2025 14:22
@rdw-software rdw-software force-pushed the 488-vfs-dlopen-dlname branch 5 times, most recently from c14036d to 48f8496 Compare January 17, 2025 17:11
rdw-software and others added 7 commits January 17, 2025 18:27
When using ffi.load this is seemingly handled by the underlying platform, but when attempting to load libraries from the VFS that isn't going to work.

There's no way around resolving names manually, since the files have to be extracted to a temporary directory and then loaded from disk (using the fully-qualified file name). Providing a separate function for the lookup  makes it easier to test and also allows users to check that resolution works as expected.
zlib is unlikely to ever be removed and quick to build, so there's no drawbacks to generating a shared library version as well.

Although this variant won't be used by the runtime itself, it does come in handy for testing vfs.dlopen in a real-world scenario.
Relying on a shared library that's created by a build script removes some complexity from the test suite, at the cost of depending on this specific library always being built.

While certainly not ideal, in the case of zlib it's probably safe to assume that this will always be the case.
This is just the first step towards a more flexible VFS interface. No tests because I don't want to lock in on the approach just yet.
There's a number of questionable things here that should likely be improved upon. Some may require redesigning the vfs and CLI library interfaces, so I'll save it for later.
Doesn't cover the main functionality; which is just too involved for a mere unit test.
This makes it possible to chain the function with ffi.load, so that the same code will work both during local development and when bundled as a self-contained LUAZIP application.
@rdw-software rdw-software force-pushed the 488-vfs-dlopen-dlname branch from 48f8496 to c584d3a Compare January 17, 2025 17:27
@rdw-software rdw-software merged commit ead7b8d into main Jan 17, 2025
11 checks passed
@rdw-software rdw-software deleted the 488-vfs-dlopen-dlname branch January 17, 2025 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable loading shared objects from the LUAZIP virtual file system (falling back to ffi.load for regular apps)
1 participant