Add PyHeap_GetType() to limited C API to get the value of ht_type from heap types using PyObject pointer.

[AraHaan]

Feature or enhancement

Proposal:

Currently on the Limited C API one can use PyType_FromSpec() using a spec for a type to obtain a special heap type object. However, in order to then call PyModule_AddObject to bind the resulting type object created from this to the module one must first access the PyHeapTypeObject structure which is sadly not defined at all, nor any of it's members when using the limited C API.

The only workaround for this would be a new PyHeap_GetType() to obtain the value (PyTypeObject *) from the input PyHeapTypeObject as a PyObject * without first casting the result from PyType_FromSpec to PyHeapTypeObject. The result is then better usage in C extensions wanting to use the limited API but cannot due to this limitation.

Alternatives as well:

• A copy of PyType_FromSpec for the limited API that takes in the module object (PyObject *) and calls the PyModule_AddObject function for the user so they never need to worry about heap type object stuffs.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

I seen discussions about allowing PyHeapTypeObject in C extensions on GitHub in the past, but no idea how far the progress is currently on that. I think having PyHeap_GetType or whatever name of a function it could be and/or a function that internally calls PyType_FromSpec AND PyModule_AddObject for C extension developers exposed to the C API could work as well.

See also: #105970 (there could be more)

[godlygeek]

in order to then call PyModule_AddObject to bind the resulting type object created from this to the module one must first access the PyHeapTypeObject structure

There's no need for that. You can use the return value from PyType_FromSpec directly as the value argument for PyModule_AddObject. The return value from PyType_FromSpec is the newly created class that you want to add to the module's namespace.

[AraHaan]

in order to then call PyModule_AddObject to bind the resulting type object created from this to the module one must first access the PyHeapTypeObject structure

There's no need for that. You can use the return value from PyType_FromSpec directly as the value argument for PyModule_AddObject. The return value from PyType_FromSpec is the newly created class that you want to add to the module's namespace.

When building and testing my updated code, I normally use PyVectorcall_Function to then test if the callable supports vectorcall and then use PyVectorcall_Call if it does, otherwise I fallback to using PyObject_Call both for calling normal functions, and for constructing instances of said heap types.

What I discovered is that somehow my code caused the interpreter to crash within the inline function for PyVectorcall_Function that checks if the vectorcall flag is provided on the type.

[picnixz]

What I discovered is that somehow my code caused the interpreter to crash within the inline function for PyVectorcall_Function that checks if the vectorcall flag is provided on the type.

Can you show us your code?

[AraHaan]

I can share it privately if possible.

[picnixz]

You can invite me to a private repo where you have a reproducer. Otherwise, can you give us a reproducer which is like your private code but without the private parts (namely just show us how you declared your types and registered them). I don't see a connection with vector call functions. Are you sure the crash doesn't happen due to other reasons (like a dereferenced pointer being NULL)?

I don't think we need to expose such API and I would still think we need a more specific use case and example of usage (and/or how a simple cuatom type implementing vectorcalls would crash).

[picnixz]

cc @ZeroIntensity @encukou @vstinner

[erlend-aasland]

in order to then call PyModule_AddObject to bind the resulting type object created from this to the module one must first access the PyHeapTypeObject structure

There's no need for that. You can use the return value from PyType_FromSpec directly as the value argument for PyModule_AddObject. The return value from PyType_FromSpec is the newly created class that you want to add to the module's namespace.

+1. Closing as not-a-bug. If you need help with the C APIs, please see the docs or create a topic on Discourse.