-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deduplicate the C definitions for static FFI exports to make the bindings easier to maintain #506
Conversation
It would be much easier to maintain the various bindings if the definition weren't duplicated, as keeping them in sync can be error-prone and they're overly verbose in at least some cases. As a first step in that direction, introduce a distinction between the stored export table and the C definitions (that are currently still stored in both the C++ and Lua sources, hence the placeholder).
Unfortunately the types used by LuaJIT's FFI won't always be quite identical as the C++ layer can use the full range of types, whereas LuaJIT often has to make do with opaque pointers for library data structures or callback types. In order to deduplicate the shared type definitions anyway, they'll have to be stored separately (and without any of the clutter that LuaJIT might be unable to process). This way, the compiler can check everything before LuaJIT even sees them. And most importantly they don't have to be kept in sync manually, which is tedious and error-prone. Also, library types still need to be copy/pasted - which isn't ideal...
If everything went smoothly, this should be exactly the same as before - except now there's a single source of truth for the typedefs, and the compiler gets to check them first. Phew!
The benefit here isn't as great since the compiler can't check the type definitions that are used by LuaJIT only. Still, moving them to separate headers reduces the boilerplate and file size in the Lua sources and enables syntax highlighting in the IDE. It also makes the dump output more useful for those libraries since it's not includingall the cdefs, but not sure how useful that really is.
The exports tables are static, but when appending the aliases, then indeed the pointer could become invalid - however unlikely that may be here.
4ec8e37
to
f04bae6
Compare
Can't wait to replace this with std::embed in the year 2030 or so.
b515855
to
e7bb888
Compare
Symbol names have different conventions on macOS (Mach-O).
Not quite sure why ld can't find the symbol, but it's actually not needed since the BLOB is null-terminated and will be copied into the Lua environment anyway.
Clearly, I misunderstood how the macOS format works (and testing via GitHub actions isn't practical). One underscore is enough, hence the linker errors that should now be resolved.
This reverts commit a46304f.
40cb1f5
to
a46304f
Compare
Just a quick note for future me as to why embedding the cdefs turned out to be so overly complicated:
In short, all solutions are terrible and I'm not sure which one is the least of a headache. Give me |
I really am not a fan of this given the increase in complexity. The drawbacks simply don't seem worth it (without
Since one goal is to make sure the FFI and C++ definitions are always in sync, maybe there's a simpler way to get that benefit:
This would still necessitate code duplication, but if the test suite catches any discrepancy then it might not be a huge problem. |
Closing for now due to the above concerns. It's too large a PR to merge as-is anyway; will split off some parts when applicable. |
Some of the definitions unfortunately can't be checked by the compiler. But overall, reduces the amount of boilerplate code:
bindings
module now contains both the exports and the cdefs for each embedded libraryThey're embedded as strings (via non-portable inline assembly) and so passed to LuaJIT, which is somewhat hackyMore cleanup will be needed to remove the
bindings
member of each library, but this is far too large already.