Ship a prebuilt C++ SDK in the ExecuTorch Linux wheel - #21477
Draft
shoumikhin wants to merge 2 commits into
Draft
Conversation
Contributor
Author
|
Stack from ghstack (oldest at bottom): |
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21477
Note: Links to docs will display an error until the docs builds have been completed. ❌ 44 New Failures, 265 Pending, 1 Unrelated Failure, 5 Unclassified FailuresAs of commit e50d9ab with merge base 0b13b6a ( NEW FAILURES - The following jobs have failed:
UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
shoumikhin
added a commit
that referenced
this pull request
Jul 29, 2026
## Why this is needed Today, using the ExecuTorch runtime from a C++ program means building ExecuTorch from source: clone the repo, sync submodules, and run a CMake build before you can compile and link your own runner. The pip wheel only ships the Python runtime module (`_portable_lib`) plus a small set of headers meant for authoring custom operators. There is no way to just `pip install executorch` and link a standalone C++ application against the runtime. This is friction for anyone whose deployment path is C++ (the common case for on-device inference) and who already has the wheel installed for export. The runtime is compiled during the wheel build and then discarded. This change ships the runtime as a linkable shared library, its public headers, and a CMake package config inside the Linux wheel, so a C++ program can link the ExecuTorch runtime with no source checkout and no separate build. ## What is inside Added to the Linux wheel (nothing removed; other platforms unchanged): - `executorch/lib/libexecutorch.so` (SONAME-versioned, with the standard `libexecutorch.so -> .so.1 -> .so.<version>` chain): the consolidated shared runtime. It bundles the runtime core plus the common runtime extensions (module, tensor, data_loader, flat_tensor, named_data_map). - `executorch/include/executorch/extension/...`: the public headers for the Module, Tensor, DataLoader, FlatTensor (.ptd reader), NamedDataMap, and header-only MallocMemoryAllocator APIs. Runtime/Program/backend headers were already shipped and are reused. - `executorch/share/cmake/executorch-config.cmake`: a CMake package config that exposes an `executorch::runtime` imported target (plus convenience aliases `executorch::core`, `executorch::extension_*`). - `executorch/utils/cmake_prefix_path`: a small helper (mirrors `torch.utils.cmake_prefix_path`) so CMake can find the config in one line. ## Why a shared library (not static archives) The runtime is shipped shared on purpose. ExecuTorch keeps a single process-global backend/kernel registry. Shipping the runtime as one shared `libexecutorch.so` lets a separately distributed backend or delegate shared library register into that one registry: the backend `.so` is built without its own copy of the runtime (its `register_backend` reference is undefined and resolves against `libexecutorch.so` at load), and its static-init registration runs when the `.so` is loaded (via whole-archive for a C++ app, or an explicit import/dlopen for Python, which is how ExecuTorch already ships the QNN backend today). Static archives would give each consumer its own private registry, which cannot support loading multiple independently distributed backends into one runtime. The set is intentionally libtorch-free and excludes the general CPU operator/kernel libraries, because a delegate supplies its own compute. It is also Linux only: the `.so` naming and SONAME symlink chain are Unix specific, so the Windows and macOS wheels are byte-identical to before. ## How to use it ```bash pip install executorch ``` ```cmake find_package(executorch CONFIG REQUIRED) add_executable(my_runner main.cpp) target_link_libraries(my_runner PRIVATE executorch::runtime) ``` ```bash cmake -S . -B build \ -DCMAKE_PREFIX_PATH="$(python -c 'import executorch.utils as u; print(u.cmake_prefix_path)')" cmake --build build ``` Existing consumers that use `find_package(executorch)` to link the Python `_portable_lib` for custom-op extensions keep working unchanged. The new C++ SDK availability is reported separately via `EXECUTORCH_SDK_FOUND`, so the legacy `EXECUTORCH_FOUND` / `EXECUTORCH_LIBRARIES` contract is preserved. ## Test plan Verified on Linux x86_64: - Built the wheel with `python setup.py bdist_wheel` and confirmed it contains `libexecutorch.so` with its SONAME symlink chain, the CMake config, the `utils` helper, and the new extension headers, and that headers with no shipped implementation are excluded. - Installed the wheel into a clean virtual environment and built a small standalone C++ runner against it with `find_package(executorch)` and `executorch.utils.cmake_prefix_path`. The runner compiled, linked, ran, and initialized the runtime. - Built a separate "coreless" backend shared library (no bundled runtime, `register_backend` left undefined) and confirmed that loading it against the installed `libexecutorch.so` registers the backend into the runtime's registry (`get_backend_class` goes from not-found to found). This is the mechanism that lets independently distributed backends coalesce into one runtime. - Confirmed the runner and the runtime link no libtorch/libc10 (via `ldd`). - Confirmed the Windows and macOS wheel code paths add nothing new, so those wheels are unaffected. - Lint and format pass (flake8, ufmt, cmake-format). Known limitation: the wheel-build step stores the SONAME symlinks as plain file copies rather than symlinks. Linking and loading still work because the SONAME target is present as a real file; a follow-up can preserve them as true symlinks to save space. ## CI Adds a PR job (test-cpp-sdk-wheel-linux in pull.yml, calling .ci/scripts/test_cpp_sdk_wheel.sh) that builds the wheel, installs it into a clean venv, and uses find_package(executorch) to build a C++ consumer linking executorch::runtime, then loads a coreless backend shared library and asserts it registers into libexecutorch.so. This gives the feature direct coverage: before, no PR job built the full wheel or linked the C++ SDK. ## Fix: scope EXECUTORCH_BUILD_SHARED to wheel builds only EXECUTORCH_BUILD_SHARED is now enabled only for bdist_wheel, not for editable/develop/install builds. It forces global PIC and changes link behavior, so leaking it into every build perturbed unrelated CI (unittest, unittest-editable). The executorch_shared build target and the libexecutorch.so packaging are gated on the same flag so non-wheel builds are byte-identical to before this change. ghstack-source-id: 9289ca6 ghstack-comment-id: 5123363253 Pull-Request: #21477
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this is needed
Today, using the ExecuTorch runtime from a C++ program means building
ExecuTorch from source: clone the repo, sync submodules, and run a CMake
build before you can compile and link your own runner. The pip wheel only
ships the Python runtime module (
_portable_lib) plus a small set ofheaders meant for authoring custom operators. There is no way to just
pip install executorchand link a standalone C++ application against theruntime.
This is friction for anyone whose deployment path is C++ (the common case
for on-device inference) and who already has the wheel installed for
export. The runtime is compiled during the wheel build and then discarded.
This change ships the runtime as a linkable shared library, its public
headers, and a CMake package config inside the Linux wheel, so a C++
program can link the ExecuTorch runtime with no source checkout and no
separate build.
What is inside
Added to the Linux wheel (nothing removed; other platforms unchanged):
executorch/lib/libexecutorch.so(SONAME-versioned, with the standardlibexecutorch.so -> .so.1 -> .so.<version>chain): the consolidatedshared runtime. It bundles the runtime core plus the common runtime
extensions (module, tensor, data_loader, flat_tensor, named_data_map).
executorch/include/executorch/extension/...: the public headers for theModule, Tensor, DataLoader, FlatTensor (.ptd reader), NamedDataMap, and
header-only MallocMemoryAllocator APIs. Runtime/Program/backend headers
were already shipped and are reused.
executorch/share/cmake/executorch-config.cmake: a CMake package configthat exposes an
executorch::runtimeimported target (plus conveniencealiases
executorch::core,executorch::extension_*).executorch/utils/cmake_prefix_path: a small helper (mirrorstorch.utils.cmake_prefix_path) so CMake can find the config in one line.Why a shared library (not static archives)
The runtime is shipped shared on purpose. ExecuTorch keeps a single
process-global backend/kernel registry. Shipping the runtime as one shared
libexecutorch.solets a separately distributed backend or delegate sharedlibrary register into that one registry: the backend
.sois built withoutits own copy of the runtime (its
register_backendreference is undefinedand resolves against
libexecutorch.soat load), and its static-initregistration runs when the
.sois loaded (via whole-archive for a C++ app,or an explicit import/dlopen for Python, which is how ExecuTorch already
ships the QNN backend today). Static archives would give each consumer its
own private registry, which cannot support loading multiple independently
distributed backends into one runtime.
The set is intentionally libtorch-free and excludes the general CPU
operator/kernel libraries, because a delegate supplies its own compute. It
is also Linux only: the
.sonaming and SONAME symlink chain are Unixspecific, so the Windows and macOS wheels are byte-identical to before.
How to use it
Existing consumers that use
find_package(executorch)to link the Python_portable_libfor custom-op extensions keep working unchanged. The newC++ SDK availability is reported separately via
EXECUTORCH_SDK_FOUND, sothe legacy
EXECUTORCH_FOUND/EXECUTORCH_LIBRARIEScontract ispreserved.
Test plan
Verified on Linux x86_64:
python setup.py bdist_wheeland confirmed itcontains
libexecutorch.sowith its SONAME symlink chain, the CMakeconfig, the
utilshelper, and the new extension headers, and thatheaders with no shipped implementation are excluded.
standalone C++ runner against it with
find_package(executorch)andexecutorch.utils.cmake_prefix_path. The runner compiled, linked, ran,and initialized the runtime.
register_backendleft undefined) and confirmed that loading it againstthe installed
libexecutorch.soregisters the backend into the runtime'sregistry (
get_backend_classgoes from not-found to found). This is themechanism that lets independently distributed backends coalesce into one
runtime.
ldd).wheels are unaffected.
Known limitation: the wheel-build step stores the SONAME symlinks as plain
file copies rather than symlinks. Linking and loading still work because the
SONAME target is present as a real file; a follow-up can preserve them as
true symlinks to save space.
CI
Adds a PR job (test-cpp-sdk-wheel-linux in pull.yml, calling
.ci/scripts/test_cpp_sdk_wheel.sh) that builds the wheel, installs it into a
clean venv, and uses find_package(executorch) to build a C++ consumer linking
executorch::runtime, then loads a coreless backend shared library and asserts
it registers into libexecutorch.so. This gives the feature direct coverage:
before, no PR job built the full wheel or linked the C++ SDK.