fix(manifests): reject non-string metadata instead of crashing on it - #3943
Merged
mnriem merged 2 commits intoAug 3, 2026
Merged
Conversation
`ExtensionManifest` and `PresetManifest` checked only key PRESENCE for `id`/`name`/`version`/`description`, then fed the values straight to `re.match()` and `packaging.Version()`. Both raise a bare `TypeError` on a non-string, which is neither `ValidationError` nor `PresetValidationError`, so it escaped every caller that already handles a malformed manifest. YAML makes this an easy authoring slip rather than a contrived one: an unquoted `version: 1.0` parses as a float and `id: 2` as an int. The user-visible symptom is the one the in-tree comment above the section guards was written to prevent (github#3898 for presets, and its extension twin): `list_installed()` degrades a bad manifest to "⚠️ Corrupted extension" but catches only the domain error, so a single bad manifest made `specify extension list` / `specify preset list` exit 1 with a raw traceback and *no output at all* — hiding every healthy extension/preset too, not just the broken one. Also unguarded on the same path: - extension `provides.commands[].name` → `TypeError` from the command-name pattern match. The sibling `file` field was already safe, since `relative_extension_path_violation()` rejects a non-string. - preset `provides.templates[].name`/`.file` → `TypeError` from `re.match` and `os.path.normpath` respectively. The third manifest twin, `IntegrationDescriptor`, is already hardened: it type-checks the same four fields and catches `TypeError` alongside `InvalidVersion`. This brings the other two in line with it. Tests: 68 added across both suites, covering each field against float, int, None, list, dict, and bool, plus an end-to-end guard per manifest type asserting a healthy entry still lists while the bad one degrades. All 68 fail with the source change reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Hardens extension and preset manifest validation against non-string YAML values.
Changes:
- Adds string-type validation for manifest metadata and entries.
- Adds 68 regression cases, including end-to-end listing behavior.
- One preset template-type validation gap remains.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/extensions/__init__.py |
Validates extension metadata and command names. |
src/specify_cli/presets/__init__.py |
Validates preset metadata and template fields. |
tests/test_extensions.py |
Adds extension validation regressions. |
tests/test_presets.py |
Adds preset validation regressions. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 1
- Review effort level: Balanced
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Review details
Suppressed comments (1)
tests/test_presets.py:253
- The validator now explicitly hardens all three required template fields, but this matrix omits
type. In particular, list/dicttypevalues previously leaked a rawTypeErrorfrom set membership, so that newly fixed path has no regression coverage. Includetypein this parameterization.
@pytest.mark.parametrize("field", ["name", "file"])
- Files reviewed: 4/4 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Collaborator
|
Thank you! |
mnriem
pushed a commit
that referenced
this pull request
Aug 4, 2026
`requires.speckit_version` was presence-checked but never type-checked in both the extension and preset manifest validators, so an unquoted YAML `speckit_version: 1.0` (a float) passed validation and reached `SpecifierSet(required)` in `check_compatibility()`. That call is guarded by `except InvalidSpecifier` alone, which a non-string escapes two different ways: - a float/int/bool/None raises `TypeError: 'float' object is not iterable` from the `SpecifierSet` constructor; - a list or dict is an *iterable*, so `SpecifierSet` accepts it and the failure surfaces much later as `AttributeError: 'str' object has no attribute 'filter'` from inside `.contains()`. Neither is a `CompatibilityError`/`PresetCompatibilityError`, so both bypass the CLI's "Compatibility Error" handler in `_commands.py` and exit 1 with a raw traceback that names no field, leaving the author with no hint which manifest key is wrong. Type-check the field in both validators, requiring a non-empty string, and additionally guard `check_compatibility()` in both managers since each is public and reachable with a hand-built or mutated manifest. This mirrors the sibling `IntegrationDescriptor`, which already requires a non-empty string for the same key, and completes the type-checking pass started in #3943 for the neighbouring `extension`/`preset` fields. Adds 33 regression tests across both modules covering every escape path; 26 of them fail without this change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Assisted-by: Claude Code (model: Claude Opus 5, supervised)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ExtensionManifest._validate()andPresetManifest._validate()check only key presence forid/name/version/description, then feed the values straight tore.match()andpackaging.Version(). Both raise a bareTypeErroron a non-string — andTypeErroris neitherValidationErrornorPresetValidationError, so it escapes every caller that already handles a malformed manifest.YAML makes this an easy authoring slip rather than a contrived one:
Symptom
This is exactly the failure mode the in-tree comment above the section guards was written to prevent (the preset side landed as #3898).
list_installed()degrades a bad manifest to⚠️ Corrupted extension, but itsexceptcatches only the domain error:So a single bad manifest makes
specify extension listexit 1 with a rawTypeErrortraceback and no output at all — hiding every healthy extension too, not just the broken one. Same forspecify preset list.Before this change:
After:
Also unguarded on the same path
provides.commands[].name→TypeErrorfrom the command-name pattern match. The siblingfilefield was already safe, sincerelative_extension_path_violation()rejects a non-string — line 404 even special-cases one for its error label, so non-string values were known to arrive here.provides.templates[].name/.file→TypeErrorfromre.matchandos.path.normpathrespectively.Why this shape of fix
The third manifest twin,
IntegrationDescriptor(integrations/catalog.py), is already hardened — it type-checks the same four fields and catchesTypeErroralongsideInvalidVersion. This PR brings the other two in line with that existing reference implementation, so the three manifest validators stay consistent.Testing
68 tests added across
tests/test_extensions.pyandtests/test_presets.py:float,int,None,list,dict,boolcommands[].nameand presettemplates[].name/.file, same matrixAll 68 fail with the source change reverted and pass with it. Full
tests/test_extensions.py+tests/test_presets.pyrun shows no new failures: the 7 failures / 66 errors present are identical before and after (pre-existing Windows symlink-elevation and pytest tmpdirWinError 5env issues, unrelated to this change).Assisted-by: Claude Opus 5 (1M context)