Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General code update #50

Merged
merged 68 commits into from
Dec 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
95f8b1e
--pre is required to install alpha version of itkwidgets
dzenanz Aug 28, 2024
c705609
Avoid ModuleNotFoundError: No module named 'zarr.storage'
dzenanz Aug 28, 2024
15229b2
WIP: running locally
dzenanz Aug 28, 2024
75a6eee
WIP: Turn FullDemo.ipynb into a script form (pipeline skeleton)
dzenanz Sep 4, 2024
78351b9
BUG: make it run
dzenanz Sep 6, 2024
c8a2a85
STYLE: Remove unused function cast_image_as
dzenanz Sep 6, 2024
8f2b182
BUG: Compute thickness before transforming to atlas space
dzenanz Sep 6, 2024
80c0ed3
WIP: mesh is a vtkPolyData, transform is an ITK class
dzenanz Sep 6, 2024
befc0a2
ENH: Convert VTK mesh into ITK format and vice versa via files
dzenanz Sep 6, 2024
ce5471b
ENH: Allow keeping intermediate outputs for inspection
dzenanz Sep 6, 2024
64ec372
Use uniGradICON instead of OAI_knees_registration_model (ITK 5.4)
dzenanz Sep 10, 2024
612eb54
STYLE: remove redundant image saving operation
dzenanz Sep 6, 2024
20c1141
ENH: Pre-process the image intensities upon reading
dzenanz Sep 10, 2024
5da6eb9
ENH: Orient all images into RAS to match the atlas
dzenanz Sep 10, 2024
24ff40c
ENH: Support reading .ply meshes
dzenanz Sep 10, 2024
b36ec8b
ENH: Atlas and subject thickness computation are the same
dzenanz Sep 10, 2024
5222d74
ENH: Segmentation works correctly only in DICOM PIR orientation
dzenanz Sep 10, 2024
d478b4a
ENH: Use manually crafted atlas meshes
dzenanz Sep 10, 2024
339ecb7
ENH: Change 3D to 2D projection to work in LPS space
dzenanz Sep 10, 2024
041c785
feat: add pipeline_cli.py
PaulHax Sep 10, 2024
753ebd4
Merge pull request #1 from PaulHax/cli
dzenanz Sep 11, 2024
daf0e3a
ENH: Fix 3D -> 2D projection
dzenanz Sep 11, 2024
26eac4e
ENH: Use smooth meshes from the atlas
dzenanz Sep 17, 2024
6904d4d
ENH: Use fitted circle's radius to convert from radians to millimeters
dzenanz Sep 17, 2024
67c5783
ENH: Match the old orientation of femoral cartilage 2D projection
dzenanz Sep 17, 2024
d6f0390
ENH: Use smaller circles for projecting vertices to 2D surface
dzenanz Sep 17, 2024
6d1db9b
ENH: Save outer thickness meshes as intermediate results
dzenanz Sep 17, 2024
ce24462
ENH: Use old 3D to 2D projection
dzenanz Oct 1, 2024
718fdf3
BUG: Improve mesh splitting
yehan0223 Oct 18, 2024
d656025
BUG: Do mesh splitting division along Y coordinate
dzenanz Oct 18, 2024
9be0396
ENH: Smoothing before splitting mesh.
yehan0223 Oct 22, 2024
32ac259
ENH: Keep the largest island(s)
yehan0223 Oct 24, 2024
453198c
BUG: Fix TC labels
yehan0223 Oct 24, 2024
f9f3f45
ENH: Normalize connection_direction to improve splitting
yehan0223 Oct 24, 2024
f1e26d4
ENH: Handle knee laterality by L/R mirroring right knees
dzenanz Oct 28, 2024
b015ec1
ENH: New thickness computation approach - version 1
dzenanz Oct 1, 2024
c933989
ENH: Transform patient distance image to atlas space
dzenanz Oct 2, 2024
ef053c2
WIP: Sample from thickness voxels directly
dzenanz Oct 2, 2024
6fa09c3
ENH: Simple sampling of thickness via mesh
dzenanz Oct 22, 2024
1ee6d40
Trying to fix negative thickness values
dzenanz Oct 24, 2024
c0fe64c
BUG: Fix 3D to 2D projection having a hole at highest intensity level
dzenanz Oct 29, 2024
a2c6a64
Choose mesh splitting or distance-based thickness computation via flag
dzenanz Oct 29, 2024
57912ad
Reorganize intermediate files
dzenanz Oct 29, 2024
63dc69c
Run all the images from patient 9000798
dzenanz Oct 29, 2024
755f8ae
Move computation specific to image-based thickness into its if branch
dzenanz Oct 29, 2024
496bb50
ENH: Add laterality arg to pipeline_cli
PaulHax Oct 30, 2024
1980d63
Use the original (non-smooth) atlas mesh
dzenanz Nov 1, 2024
134a793
Use 5 instance optimization steps in patient-atlas registration
dzenanz Nov 1, 2024
23c9828
Use the correct filename for atlas image
dzenanz Nov 4, 2024
27b72db
Add the missing atlas_image.nii.gz to correct directory
dzenanz Nov 4, 2024
b6176ce
Remove hard-coded paths from my computer
dzenanz Nov 6, 2024
d447171
Reduce instance optimization steps in registration from 5 to 2
dzenanz Nov 6, 2024
829070f
Change the test cases to use the currently problematic ones
dzenanz Nov 12, 2024
2710cfa
Avoid holes in 3D -> 2D projection for thickness == 0.0
dzenanz Nov 12, 2024
20c05af
ENH: Transform to altas space for better mesh splitting
yehan0223 Nov 19, 2024
6901470
BUG: Use the exact same image normalization method as before
dzenanz Nov 20, 2024
b0a160f
Use image plot instead of contour plot when saving to PNG format
dzenanz Nov 21, 2024
1bc19e4
ENH: Add the currently problematic cases in local testing
dzenanz Dec 5, 2024
6b788bb
ENH: Better island filtering
dzenanz Dec 4, 2024
7245a78
ENH: Use currently problematic cases for testing
dzenanz Dec 12, 2024
ad80968
ENH: Output mesh before splitting
dzenanz Dec 12, 2024
e22350f
ENH: Record pip package versions
dzenanz Dec 13, 2024
ef2d2df
Update the README.md
dzenanz Dec 17, 2024
77d6ffa
ENH: Update GitHub actions versions in CI
dzenanz Dec 19, 2024
30133e0
ENH: Use Python 3.11 in CI
dzenanz Dec 19, 2024
92c88d4
ENH: Do not specify torch version in pyproject.toml
dzenanz Dec 19, 2024
f6f4b7b
COMP: Avoid circular import dependency
dzenanz Dec 19, 2024
ec365fc
ENH: Update test_MeshThicknessCPU to comply with thickness refactoring
dzenanz Dec 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/github-hosted-action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ jobs:
max-parallel: 5

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Set up Python 3.8
uses: actions/setup-python@v3
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.11
- name: Install dependencies
run: |
pip install -e .
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/self-hosted-action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ jobs:
max-parallel: 5

steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/setup-python@v2
- uses: actions/checkout@v4
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.11
- name: Install dependencies
run: |
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
Expand Down
67 changes: 35 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,34 @@ pip install oai-analysis

This repository contains open-source analysis approaches for the [Osteoarthritis Initiative (OAI)](https://nda.nih.gov/oai/) magnetic resonance image (MRI) data.
The analysis code is largely written in Python with the help of [ITK](http://itk.org) and [VTK](http://vtk.org) for data I/O and mesh processing
as well as [PyTorch](http://pytorch.org) for the deep learning approaches for segmentation and registration. The intial development of this work
as well as [PyTorch](http://pytorch.org) for the deep learning approaches for segmentation and registration. The initial development of this work
was led by [UNC Chapel Hill](http://biag.cs.unc.edu) as well as [Kitware](http://kitware.com). This work is also an outgrowth of conversations within the
[Community for Open and Reproducible Musculoskeletal Imaging Research (JC|MSK)](https://jcmsk.github.io/). Going forward, contributions by the
broader community are, of course, not only welcome but encouraged.

The following functionality is currently supported:
### Main functionality
1. **Deep Learning Segmentation**: Automatic cartilage segmentation (femoral and tibial cartilage) using a 3D UNet.
2. **Cartilage thickness**: Extraction of femoral and tibial cartilage meshes and measuring of cartilage thickness based on a closest-point thickness estimation.
3. **Deep Learning Atlas registration**: Registration of the carilage meshes with associated cartilage thickness to a knee atlas space via a deep registration network.
2. **Deep Learning Atlas registration**: Registration of the cartilage meshes with associated cartilage thickness to a knee atlas space via a deep registration network for 3D images.
3. **Cartilage thickness**: Extraction of femoral and tibial cartilage meshes and measuring of cartilage thickness based on a closest-point thickness estimation.
4. **2D thickness mapping**: Mapping of the thickness maps to a common 2D atlas space which provides full spatial correspondence. This is achieved via unrolling (based on a cylindrical coordinate system) for the femoral cartilage and a planar projection for the tibial cartilage.
5. **Statistical analysis**: [Longitudinal statistical analysis approaches will be added shortly]
5. **Statistical analysis**: [Longitudinal statistical analysis approaches are planned]

### Pipeline steps:
1. Read image
a. Read
b. Put into canonical orientation
c. Intensity preprocess (rescale to 0-1)
2. Segment cartilage using deep learning
3. Register patient image to atlas image.
4. Transform patient mesh into atlas space using registration transform
5. Split the mesh into inner and outer surfaces in atlas space
a. Island filtering (keep largest + islands close to atlas mesh)
b. Mesh smoothing
c. Clustering into inner and outer surface
6. Transform the inner and outer meshes back to patient space
7. Measuring the thickness by computing distances between inner and outer surfaces
8. Transfer thicknesses from patient mesh to atlas mesh by iterating over atlas vertices and finding the closest patient vertex
9. Project atlas mesh to 2D using precomputed mapping

![OAI analysis workflow](doc_imgs/OAI_workflow.png)

Expand All @@ -42,13 +59,13 @@ The following functionality is currently supported:
The analysis approaches in this repository are based on our initial [OAI Analysis](https://github.com/uncbiag/OAI_analysis) work.
Much of the functionality of the original code-base has been ported to the *OAI Analysis 2* repository. The main differences are
1. **Refactoring**: A significant refactoring of the code so that it makes better use of [ITK](http://itk.org) conventions as well as [VTK](http://vtk.org) for all the mesh processing needs.
2. **ICON registration**: We switched to the new [ICON](https://github.com/uncbiag/ICON) registration approach (see manuscripts below).
2. **uniGradICON registration**: We switched to the new [uniGradICON](https://github.com/uncbiag/uniGradICON) registration approach (see manuscripts below).
3. **Data processing**: We improved the data processing by better handling of data objects. Whereas the previous *OAI Analysis* pipeline largely depended on reading and writing various different files, the *OAI Aanalysis 2* refactoring uses ITK and VTK objects.
4. **Jupyter notebooks**: Better support of analysis in Jupyter notebooks.

We are currently working on the following features which should be available in the near future:
1. **Distributed processing**: Whereas *OAI Analysis* was set up for cluster computing via a simple SLURM script OAI Analysis 2 is moving toward using [Dask](https://dask.org/) to allow for parallel processing on clusters and the cloud.
2. **Workflow management**: Whereas *OAI Analysis* used custom code to avoid recomputing results, we are switching to [Dagster](https://dagster.io/) to manage data dependencies in *OAI Analysis 2*.
2. **Workflow management**: Whereas *OAI Analysis* used custom code to avoid recomputing results, we are switching to [Dagster](https://dagster.io/) to manage data dependencies in *OAI Analysis 2*. Current progress is [here](https://github.com/PaulHax/knee-sarg).
3. **Distribution of analysis results**: As we are planning on not only distributing code, but also analysis results (such as segmentations, meshes, thickness maps) we are planning on supporting data access via the [Interplanetary File System (IPFS)](https://ipfs.io/).

## Development
Expand All @@ -73,35 +90,21 @@ upload test data to https://data.kitware.com/#collection/6205586c4acac99f42957ac

## Citation

While we used the following stationary velocity field registration approach available in [easyReg](https://github.com/uncbiag/easyreg) for *OAI Analysis*
[[paper]](https://biag.cs.unc.edu/publication/dblp-confcvpr-shen-hxn-19/)
We now use registration approach available in [uniGradICON](https://github.com/uncbiag/uniGradICON) for *OAI Analysis*
[paper](https://arxiv.org/abs/2403.05780)
```
@InProceedings{Shen_2019_CVPR,
title={Networks for joint affine and non-parametric image registration},
author={Shen, Zhengyang and Han, Xu and Xu, Zhenlin and Niethammer, Marc},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={4224--4233},
year={2019}
@article{tian2024unigradicon,
title={uniGradICON: A Foundation Model for Medical Image Registration},
author={Tian, Lin and Greer, Hastings and Kwitt, Roland and Vialard, Francois-Xavier and Estepar, Raul San Jose and Bouix, Sylvain and Rushmore, Richard and Niethammer, Marc},
journal={arXiv preprint arXiv:2403.05780},
year={2024}
}
```

we now support the [ICON](https://github.com/uncbiag/ICON) registration approach which provides a very simple registration interface and makes use of ITK images and transforms:
[[paper]](https://biag.cs.unc.edu/publication/dblp-journalscorrabs-2105-04459/)
```
@InProceedings{Greer_2021_ICCV,
author = {Greer, Hastings and Kwitt, Roland and Vialard, Francois-Xavier and Niethammer, Marc},
title = {ICON: Learning Regular Maps Through Inverse Consistency},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {3396-3405}
}
```

An overview of the current analysis framework will be available as a [QMSKI](https://qmski.org/) abstract
An overview of the analysis framework is available as a [QMSKI](https://qmski.org/) conference [abstract](https://www.researchgate.net/publication/361754685_Reproducible_Workflow_for_Visualization_and_Analysis_of_OsteoArthritis_Abnormality_Progression)
```
@InProceedings{Sahu_2022_QMSKI,
author = {Sahu, Pranjal and Greer, Hastings and Xu, Zhenlin and Shen, Zhengyang and Bonaretti, Serena and McCormick, Matt and Niethammer, Marc},3
author = {Sahu, Pranjal and Greer, Hastings and Xu, Zhenlin and Shen, Zhengyang and Bonaretti, Serena and McCormick, Matt and Niethammer, Marc},
title = {Reproducible Workflow for Visualization and Analysis of OsteoArthritis Abnormality Progression},
booktitle = {Proceedings of the International Workshop on Quantitative Musculoskeletal Imaging (QMSKI)},
year = {2022}
Expand All @@ -124,8 +127,8 @@ Results obtained by the *OAI Analysis* pipeline can be found in this manuscript
## Acknowledgements

This work was developed with support in part from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS)
under award numbers [1R44AR074375](https://reporter.nih.gov/search/Naf5qSR3eUStFkMfGm6KpQ/project-details/9777582) and [1R01AR072013](https://reporter.nih.gov/search/eE7eB34dVUGoY1nLF3kZNA/project-details/9368542).
under award numbers [1R44AR074375](https://reporter.nih.gov/search/Naf5qSR3eUStFkMfGm6KpQ/project-details/9777582), [1R01AR072013](https://reporter.nih.gov/search/eE7eB34dVUGoY1nLF3kZNA/project-details/9368542), and [1R01AR082684](https://reporter.nih.gov/search/TdbftRplwU-HPeF3spFejw/project-details/10822142).

## License

`oai-analysis` is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license.
This is distributed under the terms of the [Apache-2.0](https://spdx.org/licenses/Apache-2.0.html) license.
Binary file modified doc_imgs/OAI_workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading