Skip to content

Ship a prebuilt C++ SDK in the ExecuTorch Linux wheel - #21477

Draft
shoumikhin wants to merge 2 commits into
mainfrom
gh/shoumikhin/76/head
Draft

Ship a prebuilt C++ SDK in the ExecuTorch Linux wheel#21477
shoumikhin wants to merge 2 commits into
mainfrom
gh/shoumikhin/76/head

Conversation

@shoumikhin

Copy link
Copy Markdown
Contributor

Why this is needed

Today, using the ExecuTorch runtime from a C++ program means building
ExecuTorch from source: clone the repo, sync submodules, and run a CMake
build before you can compile and link your own runner. The pip wheel only
ships the Python runtime module (_portable_lib) plus a small set of
headers meant for authoring custom operators. There is no way to just
pip install executorch and link a standalone C++ application against the
runtime.

This is friction for anyone whose deployment path is C++ (the common case
for on-device inference) and who already has the wheel installed for
export. The runtime is compiled during the wheel build and then discarded.

This change ships the runtime as a linkable shared library, its public
headers, and a CMake package config inside the Linux wheel, so a C++
program can link the ExecuTorch runtime with no source checkout and no
separate build.

What is inside

Added to the Linux wheel (nothing removed; other platforms unchanged):

  • executorch/lib/libexecutorch.so (SONAME-versioned, with the standard
    libexecutorch.so -> .so.1 -> .so.<version> chain): the consolidated
    shared runtime. It bundles the runtime core plus the common runtime
    extensions (module, tensor, data_loader, flat_tensor, named_data_map).
  • executorch/include/executorch/extension/...: the public headers for the
    Module, Tensor, DataLoader, FlatTensor (.ptd reader), NamedDataMap, and
    header-only MallocMemoryAllocator APIs. Runtime/Program/backend headers
    were already shipped and are reused.
  • executorch/share/cmake/executorch-config.cmake: a CMake package config
    that exposes an executorch::runtime imported target (plus convenience
    aliases executorch::core, executorch::extension_*).
  • executorch/utils/cmake_prefix_path: a small helper (mirrors
    torch.utils.cmake_prefix_path) so CMake can find the config in one line.

Why a shared library (not static archives)

The runtime is shipped shared on purpose. ExecuTorch keeps a single
process-global backend/kernel registry. Shipping the runtime as one shared
libexecutorch.so lets a separately distributed backend or delegate shared
library register into that one registry: the backend .so is built without
its own copy of the runtime (its register_backend reference is undefined
and resolves against libexecutorch.so at load), and its static-init
registration runs when the .so is loaded (via whole-archive for a C++ app,
or an explicit import/dlopen for Python, which is how ExecuTorch already
ships the QNN backend today). Static archives would give each consumer its
own private registry, which cannot support loading multiple independently
distributed backends into one runtime.

The set is intentionally libtorch-free and excludes the general CPU
operator/kernel libraries, because a delegate supplies its own compute. It
is also Linux only: the .so naming and SONAME symlink chain are Unix
specific, so the Windows and macOS wheels are byte-identical to before.

How to use it

pip install executorch
find_package(executorch CONFIG REQUIRED)

add_executable(my_runner main.cpp)
target_link_libraries(my_runner PRIVATE executorch::runtime)
cmake -S . -B build \
  -DCMAKE_PREFIX_PATH="$(python -c 'import executorch.utils as u; print(u.cmake_prefix_path)')"
cmake --build build

Existing consumers that use find_package(executorch) to link the Python
_portable_lib for custom-op extensions keep working unchanged. The new
C++ SDK availability is reported separately via EXECUTORCH_SDK_FOUND, so
the legacy EXECUTORCH_FOUND / EXECUTORCH_LIBRARIES contract is
preserved.

Test plan

Verified on Linux x86_64:

  • Built the wheel with python setup.py bdist_wheel and confirmed it
    contains libexecutorch.so with its SONAME symlink chain, the CMake
    config, the utils helper, and the new extension headers, and that
    headers with no shipped implementation are excluded.
  • Installed the wheel into a clean virtual environment and built a small
    standalone C++ runner against it with find_package(executorch) and
    executorch.utils.cmake_prefix_path. The runner compiled, linked, ran,
    and initialized the runtime.
  • Built a separate "coreless" backend shared library (no bundled runtime,
    register_backend left undefined) and confirmed that loading it against
    the installed libexecutorch.so registers the backend into the runtime's
    registry (get_backend_class goes from not-found to found). This is the
    mechanism that lets independently distributed backends coalesce into one
    runtime.
  • Confirmed the runner and the runtime link no libtorch/libc10 (via ldd).
  • Confirmed the Windows and macOS wheel code paths add nothing new, so those
    wheels are unaffected.
  • Lint and format pass (flake8, ufmt, cmake-format).

Known limitation: the wheel-build step stores the SONAME symlinks as plain
file copies rather than symlinks. Linking and loading still work because the
SONAME target is present as a real file; a follow-up can preserve them as
true symlinks to save space.

CI

Adds a PR job (test-cpp-sdk-wheel-linux in pull.yml, calling
.ci/scripts/test_cpp_sdk_wheel.sh) that builds the wheel, installs it into a
clean venv, and uses find_package(executorch) to build a C++ consumer linking
executorch::runtime, then loads a coreless backend shared library and asserts
it registers into libexecutorch.so. This gives the feature direct coverage:
before, no PR job built the full wheel or linked the C++ SDK.

[ghstack-poisoned]
@shoumikhin

shoumikhin commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Stack from ghstack (oldest at bottom):

@pytorch-bot

pytorch-bot Bot commented Jul 29, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21477

Note: Links to docs will display an error until the docs builds have been completed.

❌ 44 New Failures, 265 Pending, 1 Unrelated Failure, 5 Unclassified Failures

As of commit e50d9ab with merge base 0b13b6a (image):

NEW FAILURES - The following jobs have failed:

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 29, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@shoumikhin shoumikhin added ciflow/periodic ciflow/trunk ciflow/binaries ciflow/binaries/all Release PRs with this label will build wheels for all python versions ciflow/nightly labels Jul 29, 2026
[ghstack-poisoned]
shoumikhin added a commit that referenced this pull request Jul 29, 2026
## Why this is needed

Today, using the ExecuTorch runtime from a C++ program means building
ExecuTorch from source: clone the repo, sync submodules, and run a CMake
build before you can compile and link your own runner. The pip wheel only
ships the Python runtime module (`_portable_lib`) plus a small set of
headers meant for authoring custom operators. There is no way to just
`pip install executorch` and link a standalone C++ application against the
runtime.

This is friction for anyone whose deployment path is C++ (the common case
for on-device inference) and who already has the wheel installed for
export. The runtime is compiled during the wheel build and then discarded.

This change ships the runtime as a linkable shared library, its public
headers, and a CMake package config inside the Linux wheel, so a C++
program can link the ExecuTorch runtime with no source checkout and no
separate build.

## What is inside

Added to the Linux wheel (nothing removed; other platforms unchanged):

- `executorch/lib/libexecutorch.so` (SONAME-versioned, with the standard
  `libexecutorch.so -> .so.1 -> .so.<version>` chain): the consolidated
  shared runtime. It bundles the runtime core plus the common runtime
  extensions (module, tensor, data_loader, flat_tensor, named_data_map).
- `executorch/include/executorch/extension/...`: the public headers for the
  Module, Tensor, DataLoader, FlatTensor (.ptd reader), NamedDataMap, and
  header-only MallocMemoryAllocator APIs. Runtime/Program/backend headers
  were already shipped and are reused.
- `executorch/share/cmake/executorch-config.cmake`: a CMake package config
  that exposes an `executorch::runtime` imported target (plus convenience
  aliases `executorch::core`, `executorch::extension_*`).
- `executorch/utils/cmake_prefix_path`: a small helper (mirrors
  `torch.utils.cmake_prefix_path`) so CMake can find the config in one line.

## Why a shared library (not static archives)

The runtime is shipped shared on purpose. ExecuTorch keeps a single
process-global backend/kernel registry. Shipping the runtime as one shared
`libexecutorch.so` lets a separately distributed backend or delegate shared
library register into that one registry: the backend `.so` is built without
its own copy of the runtime (its `register_backend` reference is undefined
and resolves against `libexecutorch.so` at load), and its static-init
registration runs when the `.so` is loaded (via whole-archive for a C++ app,
or an explicit import/dlopen for Python, which is how ExecuTorch already
ships the QNN backend today). Static archives would give each consumer its
own private registry, which cannot support loading multiple independently
distributed backends into one runtime.

The set is intentionally libtorch-free and excludes the general CPU
operator/kernel libraries, because a delegate supplies its own compute. It
is also Linux only: the `.so` naming and SONAME symlink chain are Unix
specific, so the Windows and macOS wheels are byte-identical to before.

## How to use it

```bash
pip install executorch
```

```cmake
find_package(executorch CONFIG REQUIRED)

add_executable(my_runner main.cpp)
target_link_libraries(my_runner PRIVATE executorch::runtime)
```

```bash
cmake -S . -B build \
  -DCMAKE_PREFIX_PATH="$(python -c 'import executorch.utils as u; print(u.cmake_prefix_path)')"
cmake --build build
```

Existing consumers that use `find_package(executorch)` to link the Python
`_portable_lib` for custom-op extensions keep working unchanged. The new
C++ SDK availability is reported separately via `EXECUTORCH_SDK_FOUND`, so
the legacy `EXECUTORCH_FOUND` / `EXECUTORCH_LIBRARIES` contract is
preserved.

## Test plan

Verified on Linux x86_64:

- Built the wheel with `python setup.py bdist_wheel` and confirmed it
  contains `libexecutorch.so` with its SONAME symlink chain, the CMake
  config, the `utils` helper, and the new extension headers, and that
  headers with no shipped implementation are excluded.
- Installed the wheel into a clean virtual environment and built a small
  standalone C++ runner against it with `find_package(executorch)` and
  `executorch.utils.cmake_prefix_path`. The runner compiled, linked, ran,
  and initialized the runtime.
- Built a separate "coreless" backend shared library (no bundled runtime,
  `register_backend` left undefined) and confirmed that loading it against
  the installed `libexecutorch.so` registers the backend into the runtime's
  registry (`get_backend_class` goes from not-found to found). This is the
  mechanism that lets independently distributed backends coalesce into one
  runtime.
- Confirmed the runner and the runtime link no libtorch/libc10 (via `ldd`).
- Confirmed the Windows and macOS wheel code paths add nothing new, so those
  wheels are unaffected.
- Lint and format pass (flake8, ufmt, cmake-format).

Known limitation: the wheel-build step stores the SONAME symlinks as plain
file copies rather than symlinks. Linking and loading still work because the
SONAME target is present as a real file; a follow-up can preserve them as
true symlinks to save space.

## CI

Adds a PR job (test-cpp-sdk-wheel-linux in pull.yml, calling
.ci/scripts/test_cpp_sdk_wheel.sh) that builds the wheel, installs it into a
clean venv, and uses find_package(executorch) to build a C++ consumer linking
executorch::runtime, then loads a coreless backend shared library and asserts
it registers into libexecutorch.so. This gives the feature direct coverage:
before, no PR job built the full wheel or linked the C++ SDK.

## Fix: scope EXECUTORCH_BUILD_SHARED to wheel builds only

EXECUTORCH_BUILD_SHARED is now enabled only for bdist_wheel, not for
editable/develop/install builds. It forces global PIC and changes link
behavior, so leaking it into every build perturbed unrelated CI
(unittest, unittest-editable). The executorch_shared build target and the
libexecutorch.so packaging are gated on the same flag so non-wheel builds
are byte-identical to before this change.

ghstack-source-id: 9289ca6
ghstack-comment-id: 5123363253
Pull-Request: #21477
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/binaries/all Release PRs with this label will build wheels for all python versions ciflow/binaries ciflow/nightly ciflow/periodic ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant