Skip to content

Add RankSEG integration tutorial - #2068

Open
statmlben wants to merge 4 commits into
Project-MONAI:mainfrom
rankseg:add-rankseg-integration-tutorial
Open

Add RankSEG integration tutorial#2068
statmlben wants to merge 4 commits into
Project-MONAI:mainfrom
rankseg:add-rankseg-integration-tutorial

Conversation

@statmlben

@statmlben statmlben commented Jul 27, 2026

Copy link
Copy Markdown

Related to MONAI-#8908

Description

This PR adds a runnable tutorial demonstrating how to use RankSEG as an optional third-party post-processing transform in a MONAI workflow.

The tutorial:

  • uses the pretrained pancreas_ct_dints_segmentation MONAI Bundle and a real MSD Task07 Pancreas case;
  • compares conventional argmax decoding with RankSEG using per-class Dice scores and a visualization;
  • demonstrates the array-based RankSEG transform;
  • demonstrates the dictionary-based RankSEGd transform in a MONAI Compose post-processing pipeline;
  • documents the public dataset source, CC BY-SA 4.0 license, and research-use context.

This PR also adds the tutorial to the repository README and registers it in doesnt_contain_max_epochs because it is an inference-only notebook.

Checks

  • Avoid including large-size files in the PR.
  • Clean up long text outputs from code cells in the notebook.
  • Check the contents and remove sensitive information such as user names and private keys.
  • Verify hyperlinks, markdown content, and relative repository paths.
  • Notebook runs automatically with:
./runner.sh -t modules/rankseg_integration.ipynb

Results

Argmax macro Dice: 0.7747
RankSEG macro Dice: 0.8502
Paired Dice change: +0.0755
Pancreas Dice: 0.8764 -> 0.8898
Tumor Dice: 0.6730 -> 0.8106
image

Summary by CodeRabbit

  • New Features

    • Added a tutorial for optional RankSEG/RankSEGd post-processing in MONAI segmentation workflows.
    • Compares RankSEG results with standard argmax predictions using a pretrained pancreas segmentation model.
    • Includes Dice score evaluation, visual comparisons, and tensor- and dictionary-based workflow examples.
    • Demonstrates dataset preparation and integration validation.
  • Documentation

    • Added the RankSEG integration notebook to the list of available modules and examples.

Signed-off-by: statmlben <bdai.hk@protonmail.com>
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3bba29b3-1247-4481-9f8b-29482c4ed743

📥 Commits

Reviewing files that changed from the base of the PR and between e245db0 and 1efa798.

📒 Files selected for processing (1)
  • modules/rankseg_integration.ipynb

Walkthrough

Adds a MONAI tutorial with RankSEG and RankSEGd transforms. The notebook runs pretrained pancreas inference, compares RankSEG with argmax using Dice scores, and documents the integration. README and runner support are updated.

Changes

RankSEG tutorial integration

Layer / File(s) Summary
RankSEG transform contracts
modules/rankseg_integration.ipynb
Adds RankSEG and RankSEGd wrappers for multiclass probability decoding and keyed MONAI pipelines.
Verified pancreas data and model setup
modules/rankseg_integration.ipynb
Downloads and verifies MSD Pancreas data, preprocesses inputs, and loads the pretrained Pancreas DiNTS Bundle.
Inference comparison and validation
modules/rankseg_integration.ipynb
Runs sliding-window inference, compares argmax and RankSEG predictions with Dice metrics, and visualizes tumor differences.
Notebook publication and execution support
README.md, runner.sh, modules/rankseg_integration.ipynb
Adds the notebook to the README, exempts it from the runner’s max_epochs check, and adds cleanup and metadata.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Notebook
  participant MSD Pancreas Archive
  participant Pancreas DiNTS Bundle
  participant RankSEG
  participant Dice Evaluation
  Notebook->>MSD Pancreas Archive: download and verify case members
  Notebook->>Pancreas DiNTS Bundle: load model and run sliding-window inference
  Pancreas DiNTS Bundle-->>Notebook: return logits
  Notebook->>RankSEG: decode softmax probabilities
  RankSEG-->>Notebook: return prediction
  Notebook->>Dice Evaluation: compare argmax and RankSEG predictions
  Dice Evaluation-->>Notebook: return Dice scores
Loading

Possibly related issues

Possibly related PRs

Suggested reviewers: ericspod

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely identifies the main change: adding a RankSEG integration tutorial.
Description check ✅ Passed The description explains the tutorial, implementation scope, checks, execution command, results, and reproducibility details.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@modules/rankseg_integration.ipynb`:
- Line 47: Update the dependency-check command in the notebook setup cell to
validate both MONAI and nibabel imports before skipping installation. Ensure any
installation runs through the active kernel interpreter, and retain the existing
MONAI extras including nibabel and tqdm so the later LoadImaged call can read
.nii.gz files.
- Around line 630-653: Update both torch.load calls for the architecture and
checkpoint artifacts in the model-loading flow to use weights_only=True instead
of explicitly enabling full pickle deserialization. Preserve the existing paths
and map_location settings, and only retain weights_only=False if the loaded
files require trusted non-tensor objects.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e68feaab-cd4b-4ade-99e2-419988bdcf7c

📥 Commits

Reviewing files that changed from the base of the PR and between 81dcf0f and 13eed5c.

📒 Files selected for processing (3)
  • README.md
  • modules/rankseg_integration.ipynb
  • runner.sh

Comment thread modules/rankseg_integration.ipynb Outdated
Comment thread modules/rankseg_integration.ipynb

@ericspod ericspod left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @statmlben thanks for this tutorial as we had discussed. I think it's fine overall for the existing content, but we need a discussion on what Rankseg is and how the algorithm works. At the top you should describe what it is, how it differs from other components, what it's used for, what the implications of using it are, etc. This by no means needs the same details as in your paper but should be motivating enough so readers know what the notebook is demonstrating and whether they'd want to use it themselves or not. I think the Coderabbit comments should be looked at but I had responses here as well.

Comment thread modules/rankseg_integration.ipynb Outdated
Comment thread modules/rankseg_integration.ipynb Outdated
Signed-off-by: statmlben <bdai.hk@protonmail.com>
@statmlben

Copy link
Copy Markdown
Author

Thanks @ericspod for the feedback. I added an introductory section near the top of the notebook explaining what RankSEG is, how it differs from argmax, fixed thresholding, and morphology-based post-processing, and how the RMA workflow operates at a high level. The section also discusses suitable use cases, probability requirements, computational cost, and the need to validate improvements on representative data.

I also addressed your inline suggestions by consolidating the dependency installation command and using the Bundle ConfigParser to construct the model.

@statmlben
statmlben requested a review from ericspod July 29, 2026 06:25
Comment thread modules/rankseg_integration.ipynb Outdated
"evaluation_file = {}\n",
"for key, subdirectory in ((\"image\", \"imagesTr\"), (\"label\", \"labelsTr\")):\n",
" evaluation_file[key] = hf_hub_download(\n",
" repo_id=\"MedOtter/msd-pancreas\",\n",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This downloads the MSD Task07 Pancreas case from repo_id="MedOtter/msd-pancreas", a personal/community Hugging Face dataset repo with no MONAI or MSD organizational affiliation and no checksum verification. Every other MSD download in this repo goes through a MONAI-controlled source instead: the msd-for-monai S3 bucket (e.g. spleen_segmentation_3d.ipynb), monai.apps.DecathlonDataset, or an official MONAI-org HF repo (hugging_face/finetune_vista3d_for_hugging_face_pipeline.ipynb uses MONAI/VISTA3D-HF). Since this tutorial is meant to stay in the repo long-term, relying on one individual's unmanaged mirror is a real breakage/provenance risk if that repo is edited or removed.

I checked, and there isn't yet a first-party equivalent that offers a single downloadable Task07 Pancreas case: neither the MONAI nor Project-MONAI Hugging Face orgs host any MSD Task07 dataset, and monai.apps.DecathlonDataset only supports downloading and extracting the full archive (no single-case mode). That said, MONAI's own official mirror at https://msd-for-monai.s3-us-west-2.amazonaws.com/Task07_Pancreas.tar (the same URL DecathlonDataset uses internally) does support HTTP range requests — I confirmed Accept-Ranges: bytes and a 206 Partial Content response to a ranged GET against it — so it is technically possible to fetch just the pancreas_102 case from this first-party source via a targeted tar-header walk, without downloading the full archive. Note the full tar is about 11.4 GB (Content-Length: 12289971712), not ~2 GB as the PR description implies, so a full-archive download via DecathlonDataset would be a much larger tradeoff than expected.

Suggested change: Prefer MONAI's own S3 mirror (msd-for-monai.s3-us-west-2.amazonaws.com/Task07_Pancreas.tar) over the personal HF mirror — either via a small range-request helper that reads the tar's header table to extract only pancreas_102, or by accepting the larger DecathlonDataset download if that's simpler to maintain. If neither is workable for this PR, at minimum please document the provenance/maintenance risk of the current mirror in the notebook and consider it a stopgap rather than a long-term solution.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing out the provenance and maintenance risk. I replaced the personal Hugging Face mirror with HTTP Range requests against a fixed object version of MONAI's official msd-for-monai S3 archive.

The helper downloads only the tar headers and contents for pancreas_102 and its label (about 29.7 MB total). It verifies the object ETag, tar member checksum/name/size/type, and SHA-256 digests. I also reran the complete notebook successfully with this source.

Comment thread modules/rankseg_integration.ipynb Outdated
},
"outputs": [],
"source": [
"%pip install -q --upgrade-strategy only-if-needed \"monai-weekly[nibabel, tqdm,matplotlib,huggingface_hub]\" rankseg\n",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rankseg is installed unpinned here, but it's the tutorial's central third-party dependency and is currently pre-1.0 (0.0.5 on PyPI), so its API (e.g. the pruning_prob and output_mode kwargs this notebook relies on) is more likely to change than a stable library. Other notebooks in this repo pin non-MONAI dependencies for this reason (e.g. torchio==0.18.73, pytorch-lightning==1.5.10, scikit-learn==1.0.2).

Suggested change: Pin to the version this notebook was verified against, e.g. rankseg==0.0.5.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I pinned the dependency to rankseg==0.0.5, which is the exact version used to execute and verify the notebook.

Comment thread modules/rankseg_integration.ipynb Outdated
"- [RankSEG repository and documentation](https://github.com/rankseg/rankseg)\n",
"- [RankSEG: A Consistent Ranking-based Framework for Segmentation](https://www.jmlr.org/papers/v24/22-0712.html)\n",
"- [RankSEG-RMA: An Efficient Segmentation Algorithm via Reciprocal Moment Approximation](https://openreview.net/forum?id=4tRMm1JJhw)\n",
"- [Reproducible MONAI Pancreas comparison](https://github.com/rankseg/rankseg/tree/main/experiments/monai-pr-8908-reproducibility)\n",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Practical notes" cell states a specific external benchmark (+0.0209 mean samplewise Dice, 18/20 cases) and links to rankseg/rankseg on the main branch. Since main can move, this claim isn't reproducibly tied to a fixed artifact.

Suggested change: Link to a tagged release or fixed commit SHA of that experiments directory instead of main.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I replaced the moving main-branch link with a link to the fixed commit that introduced the reproducibility experiment:

https://github.com/rankseg/rankseg/tree/2f9e7e9e36145d960d3b568f340c4ccb60b397d4/experiments/monai-pr-8908-reproducibility

@garciadias garciadias left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this tutorial — I built a GPU Docker container (monai:blackwell, RTX 5090), installed the notebook's declared dependencies exactly as specified, and ran the full ./runner.sh -t modules/rankseg_integration.ipynb end-to-end. It passed cleanly and reproduced the PR description's numbers exactly (Argmax macro Dice 0.7747, RankSEG macro Dice 0.8502, Pancreas 0.8764 -> 0.8898, Tumor 0.6730 -> 0.8106). The RankSEG/RankSEGd transform code, shape handling, and Dice evaluation are all correct.

I left three inline comments. The main one (major) is that the MSD Task07 Pancreas case is downloaded from an unofficial personal Hugging Face account (MedOtter/msd-pancreas) rather than a MONAI-controlled source, unlike every other MSD download in this repo — I've included a concrete alternative (MONAI's own S3 mirror, which supports HTTP range requests) in that comment. The other two are minor (pinning the rankseg dependency, and linking to a fixed commit rather than a moving branch for an external benchmark claim).

@statmlben

statmlben commented Jul 31, 2026

Copy link
Copy Markdown
Author

Hi @garciadias , thank you for the detailed review. I addressed all three comments by switching to MONAI's official MSD S3 source with integrity checks, pinning rankseg==0.0.5, and linking the benchmark to a fixed commit.

Thanks again for helping improve the tutorial's provenance and reproducibility

Signed-off-by: statmlben <bdai.hk@protonmail.com>
Signed-off-by: statmlben <bdai.hk@protonmail.com>
@statmlben
statmlben force-pushed the add-rankseg-integration-tutorial branch from e245db0 to 1efa798 Compare July 31, 2026 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants