Skip to content

fix(manifests): reject non-string metadata instead of crashing on it - #3943

Merged
mnriem merged 2 commits into
github:mainfrom
Noor-ul-ain001:fix/manifest-nonstring-metadata-crash
Aug 3, 2026
Merged

fix(manifests): reject non-string metadata instead of crashing on it#3943
mnriem merged 2 commits into
github:mainfrom
Noor-ul-ain001:fix/manifest-nonstring-metadata-crash

Conversation

@Noor-ul-ain001

Copy link
Copy Markdown
Contributor

Problem

ExtensionManifest._validate() and PresetManifest._validate() check only key presence for id/name/version/description, then feed the values straight to re.match() and packaging.Version(). Both raise a bare TypeError on a non-string — and TypeError is neither ValidationError nor PresetValidationError, so it escapes every caller that already handles a malformed manifest.

YAML makes this an easy authoring slip rather than a contrived one:

extension:
  id: myext
  version: 1.0      # unquoted -> parses as a float, not a string

Symptom

This is exactly the failure mode the in-tree comment above the section guards was written to prevent (the preset side landed as #3898). list_installed() degrades a bad manifest to ⚠️ Corrupted extension, but its except catches only the domain error:

except ValidationError:
    # Corrupted extension

So a single bad manifest makes specify extension list exit 1 with a raw TypeError traceback and no output at all — hiding every healthy extension too, not just the broken one. Same for specify preset list.

Before this change:

$ specify extension list
TypeError: 'float' object is not iterable      # exit 1, nothing listed

After:

$ specify extension list

Installed Extensions:

    good-ext (v1.0.0)
     ...
  ✗ bad-ext (v1.0)
     ⚠️ Corrupted extension

Also unguarded on the same path

  • extension provides.commands[].nameTypeError from the command-name pattern match. The sibling file field was already safe, since relative_extension_path_violation() rejects a non-string — line 404 even special-cases one for its error label, so non-string values were known to arrive here.
  • preset provides.templates[].name / .fileTypeError from re.match and os.path.normpath respectively.

Why this shape of fix

The third manifest twin, IntegrationDescriptor (integrations/catalog.py), is already hardened — it type-checks the same four fields and catches TypeError alongside InvalidVersion. This PR brings the other two in line with that existing reference implementation, so the three manifest validators stay consistent.

Testing

68 tests added across tests/test_extensions.py and tests/test_presets.py:

  • each of the four metadata fields × float, int, None, list, dict, bool
  • extension commands[].name and preset templates[].name/.file, same matrix
  • one end-to-end guard per manifest type asserting a healthy entry still lists normally while only the bad one degrades to "Corrupted"

All 68 fail with the source change reverted and pass with it. Full tests/test_extensions.py + tests/test_presets.py run shows no new failures: the 7 failures / 66 errors present are identical before and after (pre-existing Windows symlink-elevation and pytest tmpdir WinError 5 env issues, unrelated to this change).

Assisted-by: Claude Opus 5 (1M context)

`ExtensionManifest` and `PresetManifest` checked only key PRESENCE for
`id`/`name`/`version`/`description`, then fed the values straight to
`re.match()` and `packaging.Version()`. Both raise a bare `TypeError` on a
non-string, which is neither `ValidationError` nor `PresetValidationError`,
so it escaped every caller that already handles a malformed manifest.

YAML makes this an easy authoring slip rather than a contrived one: an
unquoted `version: 1.0` parses as a float and `id: 2` as an int.

The user-visible symptom is the one the in-tree comment above the section
guards was written to prevent (github#3898 for presets, and its extension twin):
`list_installed()` degrades a bad manifest to "⚠️ Corrupted extension" but
catches only the domain error, so a single bad manifest made
`specify extension list` / `specify preset list` exit 1 with a raw
traceback and *no output at all* — hiding every healthy extension/preset
too, not just the broken one.

Also unguarded on the same path:
- extension `provides.commands[].name` → `TypeError` from the command-name
  pattern match. The sibling `file` field was already safe, since
  `relative_extension_path_violation()` rejects a non-string.
- preset `provides.templates[].name`/`.file` → `TypeError` from `re.match`
  and `os.path.normpath` respectively.

The third manifest twin, `IntegrationDescriptor`, is already hardened: it
type-checks the same four fields and catches `TypeError` alongside
`InvalidVersion`. This brings the other two in line with it.

Tests: 68 added across both suites, covering each field against float, int,
None, list, dict, and bool, plus an end-to-end guard per manifest type
asserting a healthy entry still lists while the bad one degrades. All 68
fail with the source change reverted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens extension and preset manifest validation against non-string YAML values.

Changes:

  • Adds string-type validation for manifest metadata and entries.
  • Adds 68 regression cases, including end-to-end listing behavior.
  • One preset template-type validation gap remains.
Show a summary per file
File Description
src/specify_cli/extensions/__init__.py Validates extension metadata and command names.
src/specify_cli/presets/__init__.py Validates preset metadata and template fields.
tests/test_extensions.py Adds extension validation regressions.
tests/test_presets.py Adds preset validation regressions.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 4/4 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread src/specify_cli/presets/__init__.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Suppressed comments (1)

tests/test_presets.py:253

  • The validator now explicitly hardens all three required template fields, but this matrix omits type. In particular, list/dict type values previously leaked a raw TypeError from set membership, so that newly fixed path has no regression coverage. Include type in this parameterization.
    @pytest.mark.parametrize("field", ["name", "file"])
  • Files reviewed: 4/4 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@mnriem
mnriem merged commit 2d8904a into github:main Aug 3, 2026
14 checks passed
@mnriem

mnriem commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Thank you!

mnriem pushed a commit that referenced this pull request Aug 4, 2026
`requires.speckit_version` was presence-checked but never type-checked in
both the extension and preset manifest validators, so an unquoted YAML
`speckit_version: 1.0` (a float) passed validation and reached
`SpecifierSet(required)` in `check_compatibility()`.

That call is guarded by `except InvalidSpecifier` alone, which a
non-string escapes two different ways:

- a float/int/bool/None raises `TypeError: 'float' object is not
  iterable` from the `SpecifierSet` constructor;
- a list or dict is an *iterable*, so `SpecifierSet` accepts it and the
  failure surfaces much later as `AttributeError: 'str' object has no
  attribute 'filter'` from inside `.contains()`.

Neither is a `CompatibilityError`/`PresetCompatibilityError`, so both
bypass the CLI's "Compatibility Error" handler in `_commands.py` and exit
1 with a raw traceback that names no field, leaving the author with no
hint which manifest key is wrong.

Type-check the field in both validators, requiring a non-empty string,
and additionally guard `check_compatibility()` in both managers since
each is public and reachable with a hand-built or mutated manifest. This
mirrors the sibling `IntegrationDescriptor`, which already requires a
non-empty string for the same key, and completes the type-checking pass
started in #3943 for the neighbouring `extension`/`preset` fields.

Adds 33 regression tests across both modules covering every escape path;
26 of them fail without this change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Assisted-by: Claude Code (model: Claude Opus 5, supervised)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants