How to run ExecuTorch on Linux with aarch64-oe-linux-gcc11.2?

[suhyun01150]

Hi, I am new to ExecuTorch and currently trying to build and run it on a Linux-based Qualcomm board (QCS/QCM8550). The board's specifications are:

OS: Linux
Compiler: aarch64-oe-linux-gcc11.2
SOC Model: 66
Hexagon Arch: V73
I noticed that most guides are focused on Android environments. Could you please provide any hints or suggestions for building and running ExecuTorch on Linux with this setup?
Any help or resources would be greatly appreciated!
Thank you in advance!

[kimishpatel]

Can you add more details? For example, are you trying to export a model and run via ExecuTorch on CPU backend on your board? Or is it for qnn/hexagon backend? Is the issue with follow initial setup for ExecuTorch follow some tutorial (link the tutorial page)

[suhyun01150]

I am trying to run the model using the qnn backend. I referred to the instructions in this link: https://github.com/pytorch/executorch/blob/main/examples/models/llama/README.md. I am very new to this area, and while I know how to run models using CUDA, I would like to learn how to execute models on Ubuntu Python environments. Any guidance or additional steps would be greatly appreciated!

[kimishpatel]

Much of the executorch stack really is composed of 1) export model 2) run model. 2nd step is largely in cpp while 1st in python. There is no python/eager equivalent of CUDA for ET, either qnn or other backend. Is that what your question is?

If by python environment you mean just run using python APIs than 2 can be run via python API but that is by making python API is backed by cpp runner

[suhyun01150]

Hello! Thank you so much for your response! I looked into the Python API documentation and found this: https://github.com/pytorch/executorch/blob/main/docs/source/runtime-python-api-reference.rst. Is this all the available documentation? Could you provide more guidance or examples on how to execute a .pte file using the Python APIs with the qnn backend?

Once again, thank you so much for your help and guidance!

[SS-JIA]

Hi @suhyun01150, thanks for your interest in ExecuTorch! I looked through the docs and indeed most of our tutorials involve executing a .pte file using a C++ API. However, we have Python tests than export, load and run the model so loading + executing the model via Python is an option as well.

Please check out the QNN delegate test script for how this is done. The function that performs the end to end test can be found here.