Fixes to the sphinx doc

- Rename API pages (remove API and References) - Add warning to toplevel API doc -- in development - Reorder/sort overview and API docs - move folder images to doc - Add/remove duplicate documents and instead include the main document - Docker.md - Singularity.md - README.md: One README file is re-used in doc in multiple places by adding anchors and "include" blocks in doc/START.md and doc/overview/INTRO.md - doc/overview/README.md - doc/index.rst - doc/START.md - doc/overview/index.rst - doc/overview/INTRO.md - add sphinx extension to rewrite links - add doc to dockerignore - update sphinx configuration - some fixes to sphinx doc
Deep-MI · Apr 17, 2024 · 5c5f688 · 5c5f688
1 parent e39feeb
commit 5c5f688
Show file tree

Hide file tree

Showing 31 changed files with 248 additions and 235 deletions.
diff --git a/.dockerignore b/.dockerignore
@@ -2,7 +2,7 @@
 **/.github
 checkpoints
 Singularity
-images
+doc
 venv
 Tutorial
 **/*.md

diff --git a/FastSurferCNN/README.md b/FastSurferCNN/README.md
@@ -1,7 +1,7 @@
 # Overview
 
 This directory contains all information needed to run inference with the readily trained FastSurferVINN or train it from scratch. FastSurferCNN is capable of whole brain segmentation into 95 classes in under 1 minute, mimicking FreeSurfer's anatomical segmentation and cortical parcellation (DKTatlas). The network architecture incorporates local and global competition via competitive dense blocks and competitive skip pathways, as well as multi-slice information aggregation that specifically tailor network performance towards accurate segmentation of both cortical and sub-cortical structures. 
-![](/images/detailed_network.png)
+![](/doc/images/detailed_network.png)
 
 The network was trained with conformed images (UCHAR, 1-0.7 mm voxels and standard slice orientation). These specifications are checked in the run_prediction.py script and the image is automatically conformed if it does not comply.
 

diff --git a/FastSurferCNN/models/networks.py b/FastSurferCNN/models/networks.py
@@ -14,13 +14,14 @@
 
 # IMPORTS
 import numpy as np
-import yacs
+import yacs.config
 import FastSurferCNN.models.interpolation_layer as il
 import FastSurferCNN.models.sub_module as sm
 
 from typing import Dict, Optional, Union
 from torch import Tensor, nn
 
+
 class FastSurferCNNBase(nn.Module):
     """
     Network Definition of Fully Competitive Network network.

diff --git a/README.md b/README.md
@@ -3,7 +3,9 @@
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Deep-MI/FastSurfer/blob/stable/Tutorial/Tutorial_FastSurferCNN_QuickSeg.ipynb)
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Deep-MI/FastSurfer/blob/stable/Tutorial/Complete_FastSurfer_Tutorial.ipynb)
 
-# Overview
+<!-- start of content -->
+# Welcome to FastSurfer!
+##  Overview
 
 This README contains all information needed to run FastSurfer - a fast and accurate deep-learning based neuroimaging pipeline. FastSurfer provides a fully compatible [FreeSurfer](https://freesurfer.net/) alternative for volumetric analysis (within minutes) and surface-based thickness analysis (within only around 1h run time). 
 FastSurfer is transitioning to sub-millimeter resolution support throughout the pipeline.
@@ -15,40 +17,43 @@ The FastSurfer pipeline consists of two main parts for segmentation and surface
 
 
 ### Segmentation Modules 
-- approximately 5 minutes (GPU), `--seg_only` only runs this part
-- Modules (all by default):
+- approximately 5 minutes (GPU), `--seg_only` only runs this part. 
+
+Modules (all run by default):
   1. `asegdkt:` FastSurferVINN for whole brain segmentation (deactivate with `--no_asegdkt`)
      - the core, outputs anatomical segmentation and cortical parcellation and statistics of 95 classes, mimics FreeSurfer’s DKTatlas.
-     - requires a T1w image ([notes on input images](#requirements-to-input-images)), supports high-res (up to 0.7mm, experimental beyond that).
+     - requires a T1w image ([notes on input images](README.md#requirements-to-input-images)), supports high-res (up to 0.7mm, experimental beyond that).
      - performs bias-field correction and calculates volume statistics corrected for partial volume effects (skipped if `--no_biasfield` is passed).
   2. `cereb:` CerebNet for cerebellum sub-segmentation (deactivate with `--no_cereb`)
      - requires `asegdkt_segfile`, outputs cerebellar sub-segmentation with detailed WM/GM delineation.
-     - requires a T1w image ([notes on input images](#requirements-to-input-images)), which will be resampled to 1mm isotropic images (no native high-res support).
+     - requires a T1w image ([notes on input images](README.md#requirements-to-input-images)), which will be resampled to 1mm isotropic images (no native high-res support).
      - calculates volume statistics corrected for partial volume effects (skipped if `--no_biasfield` is passed).
 
 ### Surface reconstruction
-- approximately 60-90 minutes, `--surf_only` runs only the surface part
-- supports high-resolution images (up to 0.7mm, experimental beyond that)
+- approximately 60-90 minutes, `--surf_only` runs only the surface part.
+- supports high-resolution images (up to 0.7mm, experimental beyond that).
 
+<!-- start of image requirements -->
 ### Requirements to input images
 All pipeline parts and modules require good quality MRI images, preferably from a 3T MR scanner.
 FastSurfer expects a similar image quality as FreeSurfer, so what works with FreeSurfer should also work with FastSurfer. 
 Notwithstanding module-specific limitations, resolution should be between 1mm and 0.7mm isotropic (slice thickness should not exceed 1.5mm). Preferred sequence is Siemens MPRAGE or multi-echo MPRAGE. GE SPGR should also work. See `--vox_size` flag for high-res behaviour.
+<!-- end of image requirements -->
 
-![](/images/teaser.png)
-
+![](doc/images/teaser.png)
 
-# Getting started
+<!-- start of getting started -->
+## Getting started
 
-## Installation 
+### Installation 
 There are two ways to run FastSurfer (links are to installation instructions):
 
 1. In a container ([Singularity](doc/overview/INSTALL.md#singularity) or [Docker](doc/overview/INSTALL.md#docker)) (OS: [Linux](doc/overview/INSTALL.md#linux), [Windows](doc/overview/INSTALL.md#windows), [MacOS on Intel](doc/overview/INSTALL.md#docker--currently-only-supported-for-intel-cpus-)),
 2. As a [native install](doc/overview/INSTALL.md#native--ubuntu-2004-) (all OS for segmentation part). 
 
-We recommended you use Singularity or Docker, especially if either is already installed on your system, because the images we provide on [DockerHub](https://hub.docker.com/r/deepmi/fastsurfer) conveniently include everything needed for FastSurfer, expect a [FreeSurfer license file](https://surfer.nmr.mgh.harvard.edu/fswiki/License). We have detailed, per-OS Installation instructions in the [INSTALL.md file](doc/overview/INSTALL.md).
+We recommended you use Singularity or Docker, especially if either is already installed on your system, because the images we provide on [DockerHub](https://hub.docker.com/r/deepmi/fastsurfer) conveniently include everything needed for FastSurfer. You will also need a [FreeSurfer license file](https://surfer.nmr.mgh.harvard.edu/fswiki/License) for the [Surface pipeline](#surface-reconstruction). We have detailed per-OS Installation instructions in the [INSTALL.md file](doc/overview/INSTALL.md).
 
-## Usage
+### Usage
 
 All installation methods use the `run_fastsurfer.sh` call interface (replace `*fastsurfer-flags*` with [FastSurfer flags](doc/overview/FLAGS.md#required-arguments)), which is the general starting point for FastSurfer. However, there are different ways to call this script depending on the installation, which we explain here:
 
@@ -86,16 +91,16 @@ All installation methods use the `run_fastsurfer.sh` call interface (replace `*f
 
    The `-v` flag is used to tell docker, which folders FastSurfer can read and write to.
 
-   See also __[Example 1](doc/overview/EXAMPLES.md#example-1--fastSurfer-docker)__ for a full FastSurfer run inside a Docker container and [the Docker README](Docker/README.md#docker-flags-) for more details on the docker flags including `--rm` and `--user`.
+   See also __[Example 1](doc/overview/EXAMPLES.md#example-1--fastSurfer-docker)__ for a full FastSurfer run inside a Docker container and [the Docker README](Docker/README.md#docker-flags) for more details on the docker flags including `--rm` and `--user`.
 
 2. For a __native install__, you need to activate your FastSurfer environment (e.g. `conda activate fastsurfer_gpu`) and make sure you have added the FastSurfer path to your `PYTHONPATH` variable, e.g. `export PYTHONPATH=$(pwd)`. 
 
    You will then be able to run fastsurfer with `./run_fastsurfer.sh *fastsurfer-flags*`.
 
    See also [Example 3](doc/overview/EXAMPLES.md#example-3--native-fastsurfer-on-subjectx--with-parallel-processing-of-hemis-) for an illustration of the commands to run the entire FastSurfer pipeline (FastSurferCNN + recon-surf) natively.
 
-
-## FastSurfer_Flags
+<!-- start of flags -->
+### FastSurfer_Flags
 Please refer to [FASTSURFER_FLAGS](doc/overview/FLAGS.md).
 
 
@@ -116,7 +121,7 @@ Modules output can be found here: [FastSurfer_Output_Files](doc/overview/OUTPUT_
 - [Cerebnet module](doc/overview/OUTPUT_FILES.md#cerebnet-module)
 - [Surface module](doc/overview/OUTPUT_FILES.md#surface-module)
 
-
+<!-- start of system requirements -->
 ## System Requirements
 
 Recommendation: At least 8 GB system memory and 8 GB NVIDIA graphics memory ``--viewagg_device gpu``  
@@ -146,7 +151,7 @@ Specifically, the segmentation modules feature options for optimized paralleliza
 
 ## FreeSurfer Downstream Modules
 
-FreeSurfer provides several Add-on modules for downstream processing, such as subfield segmentation ( [hippocampus/amygdala](https://surfer.nmr.mgh.harvard.edu/fswiki/HippocampalSubfieldsAndNucleiOfAmygdala), [brainstrem](https://surfer.nmr.mgh.harvard.edu/fswiki/BrainstemSubstructures), [thalamus](https://freesurfer.net/fswiki/ThalamicNuclei) and [hypothalamus](https://surfer.nmr.mgh.harvard.edu/fswiki/HypothalamicSubunits) ) as well as [TRACULA](https://surfer.nmr.mgh.harvard.edu/fswiki/Tracula). We now provide symlinks to the required files, as FastSurfer creates them with a different name (e.g. using "mapped" or "DKT" to make clear that these file are from our segmentation using the DKT Atlas protocol, and mapped to the surface). Most subfield segmentations require `wmparc.mgz` and work very well with FastSurfer,  so feel free to run those pipelines after FastSurfer. TRACULA requires `aparc+aseg.mgz` which we now link, but have not tested if it works, given that [DKT-atlas](https://mindboggle.readthedocs.io/en/latest/labels.html) merged a few labels. You should source FreeSurfer 7.3.2 to run these modules. 
+FreeSurfer provides several Add-on modules for downstream processing, such as subfield segmentation ( [hippocampus/amygdala](https://surfer.nmr.mgh.harvard.edu/fswiki/HippocampalSubfieldsAndNucleiOfAmygdala), [brainstem](https://surfer.nmr.mgh.harvard.edu/fswiki/BrainstemSubstructures), [thalamus](https://freesurfer.net/fswiki/ThalamicNuclei) and [hypothalamus](https://surfer.nmr.mgh.harvard.edu/fswiki/HypothalamicSubunits) ) as well as [TRACULA](https://surfer.nmr.mgh.harvard.edu/fswiki/Tracula). We now provide symlinks to the required files, as FastSurfer creates them with a different name (e.g. using "mapped" or "DKT" to make clear that these file are from our segmentation using the DKT Atlas protocol, and mapped to the surface). Most subfield segmentations require `wmparc.mgz` and work very well with FastSurfer,  so feel free to run those pipelines after FastSurfer. TRACULA requires `aparc+aseg.mgz` which we now link, but have not tested if it works, given that [DKT-atlas](https://mindboggle.readthedocs.io/en/latest/labels.html) merged a few labels. You should source FreeSurfer 7.3.2 to run these modules. 
 
 
 ## Intended Use
@@ -164,14 +169,13 @@ _Henschel L*, Kuegler D*, Reuter M. (*co-first). FastSurferVINN: Building Resolu
 
 _Faber J*, Kuegler D*, Bahrami E*, et al. (*co-first). CerebNet: A fast and reliable deep-learning pipeline for detailed cerebellum sub-segmentation. NeuroImage 264 (2022), 119703. https://doi.org/10.1016/j.neuroimage.2022.119703_
 
-Stay tuned for updates and follow us on Twitter: https://twitter.com/deepmilab
-
+Stay tuned for updates and follow us on [X/Twitter](https://twitter.com/deepmilab).
 
+<!-- start of acknowledgements -->
 ## Acknowledgements
 
 This project is partially funded by:
 - [Chan Zuckerberg Initiative](https://chanzuckerberg.com/eoss/proposals/fastsurfer-ai-based-neuroimage-analysis-package/)
 - [German Federal Ministry of Education and Research](https://www.gesundheitsforschung-bmbf.de/de/deepni-innovative-deep-learning-methoden-fur-die-rechnergestutzte-neuro-bildgebung-10897.php)
 
-The recon-surf pipeline is largely based on FreeSurfer 
-https://surfer.nmr.mgh.harvard.edu/fswiki/FreeSurferMethodsCitation
+The recon-surf pipeline is largely based on [FreeSurfer](https://surfer.nmr.mgh.harvard.edu/fswiki/FreeSurferMethodsCitation).
diff --git a/doc/START.md b/doc/START.md
@@ -0,0 +1,6 @@
+```{include} ../README.md
+:relative-docs: doc/
+:relative-images:
+:start-after: <!-- start of content -->
+:end-before: <!-- start of getting started -->
+```
diff --git a/doc/api/CerebNet.rst b/doc/api/CerebNet.rst
@@ -1,5 +1,5 @@
-API CerebNet References
-=======================
+CerebNet
+========
 
 
 .. currentmodule:: CerebNet

diff --git a/doc/api/CerebNet_dataloader.rst b/doc/api/CerebNet_dataloader.rst
@@ -1,5 +1,5 @@
-API CerebNet_dataloader References
-==================================
+CerebNet.dataloader
+===================
 
 
 .. currentmodule:: CerebNet.data_loader

diff --git a/doc/api/CerebNet_datasets.rst b/doc/api/CerebNet_datasets.rst
@@ -1,5 +1,5 @@
-API CerebNet_datasets References
-================================
+CerebNet.datasets
+=================
 
 
 .. currentmodule:: CerebNet.datasets

diff --git a/doc/api/CerebNet_models.rst b/doc/api/CerebNet_models.rst
@@ -1,5 +1,5 @@
-API CerebNet_models References
-==============================
+CerebNet_models
+===============
 
 
 .. currentmodule:: CerebNet.models

diff --git a/doc/api/CerebNet_utils.rst b/doc/api/CerebNet_utils.rst
@@ -1,5 +1,5 @@
-API CerebNet_utils References
-=============================
+CerebNet.utils
+==============
 
 
 .. currentmodule:: CerebNet.utils

diff --git a/doc/api/FastSurferCNN.data_loader.rst b/doc/api/FastSurferCNN.data_loader.rst
@@ -1,5 +1,5 @@
-API data_loader References
-==========================
+FastSurferCNN.data_loader
+=========================
 
 
 .. currentmodule:: FastSurferCNN.data_loader

diff --git a/doc/api/FastSurferCNN.models.rst b/doc/api/FastSurferCNN.models.rst
@@ -1,5 +1,5 @@
-API models References
-=====================
+FastSurferCNN.models
+====================
 
 
 .. currentmodule:: FastSurferCNN.models

diff --git a/doc/api/FastSurferCNN.rst b/doc/api/FastSurferCNN.rst
@@ -1,5 +1,5 @@
-API References
-==============
+FastSurferCNN
+=============
 
 
 .. currentmodule:: FastSurferCNN

diff --git a/doc/api/FastSurferCNN.utils.rst b/doc/api/FastSurferCNN.utils.rst
@@ -1,5 +1,5 @@
-API utils References
-====================
+FastSurferCNN.utils
+===================
 
 
 .. currentmodule:: FastSurferCNN.utils

diff --git a/doc/api/index.rst b/doc/api/index.rst
@@ -1,17 +1,20 @@
 FastSurfer API
 ==============
 
+.. note:: Warning
+    The FastSurfer API is in development and will change without warning. Please consider no internal module, class or function final at this point!
+
 .. toctree::
     :maxdepth: 2
 
     FastSurferCNN.rst
+    FastSurferCNN.data_loader.rst
     FastSurferCNN.models.rst
     FastSurferCNN.utils.rst
-    FastSurferCNN.data_loader.rst
-    recon_surf.rst
     CerebNet.rst
     CerebNet_dataloader.rst
     CerebNet_datasets.rst
     CerebNet_models.rst
     CerebNet_utils.rst
+    recon_surf.rst
 
diff --git a/doc/api/recon_surf.rst b/doc/api/recon_surf.rst
@@ -1,5 +1,5 @@
-API recon_surf References
-=========================
+recon_surf
+==========
 
 
 .. currentmodule:: recon_surf

diff --git a/doc/conf.py b/doc/conf.py
@@ -7,18 +7,14 @@
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
 
-import inspect
-from datetime import date
-from importlib import import_module
-from typing import Dict, Optional
-
 import sys
 import os
 
 # here i added the relative path because sphinx was not able
 # to locate FastSurferCNN module directly for autosummary
 sys.path.append(os.path.dirname(__file__) + "/..")
 sys.path.append(os.path.dirname(__file__) + "/../recon_surf")
+sys.path.append(os.path.dirname(__file__) + "/sphinx_ext")
 
 project = "FastSurfer"
 author = "FastSurfer Developers"
@@ -54,13 +50,16 @@
     "nbsphinx",
     "IPython.sphinxext.ipython_console_highlighting",
     "myst_parser",
+    "re_reference",
 ]
 
 # Suppress myst.xref_missing warning and  i.e A target was
 # not found for a cross-reference
 # Reference: https://myst-parser.readthedocs.io/en/latest/configuration.html#build-warnings
-suppress_warnings = ["myst.xref_missing",
-                     "myst.duplicate_def",]
+suppress_warnings = [
+    # "myst.xref_missing",
+    "myst.duplicate_def",
+]
 
 templates_path = ["_templates"]
 exclude_patterns = [
@@ -217,3 +216,17 @@ def linkcode_resolve(domain, info):
         anchor = ""
     result = f"{gh_url}/blob/stable/{filename}.py{anchor}"
     return result
+
+myst_ref_domains = ["myst", "std", "py"]
+
+
+# re-reftarget=(regex) => used in missing-reference
+# re-refuri/refid=(regex) => used in doctree-
+re_reference = {
+    "re-refid=^((../)*)(Singularity|Docker)\\/README\\.md#": "/overview/\\3.md#",
+    "re-reftarget=^\\/overview\\/intro\\.md#": "/index.rst#",
+    "re-refid=^README.md#requirements-": "/index.rst#requirements-",
+    "re-refid=^../README.md": "/index.rst",
+}
+
+
diff --git a/images/FastSurfer_v5.pdf → doc/images/FastSurfer_v5.pdf b/images/FastSurfer_v5.pdf → doc/images/FastSurfer_v5.pdf
diff --git a/images/FastSurfer_v5.png → doc/images/FastSurfer_v5.png b/images/FastSurfer_v5.png → doc/images/FastSurfer_v5.png
diff --git a/images/detailed_network.pdf → doc/images/detailed_network.pdf b/images/detailed_network.pdf → doc/images/detailed_network.pdf
diff --git a/images/detailed_network.png → doc/images/detailed_network.png b/images/detailed_network.png → doc/images/detailed_network.png
diff --git a/images/teaser.png → doc/images/teaser.png b/images/teaser.png → doc/images/teaser.png
diff --git a/images/teaser_white.pdf → doc/images/teaser_white.pdf b/images/teaser_white.pdf → doc/images/teaser_white.pdf
diff --git a/doc/index.rst b/doc/index.rst
@@ -3,16 +3,16 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-Welcome to FastSurfer's documentation!
-======================================
-
-.. image:: ../images/teaser.png
+.. image:: images/teaser.png
    :alt: FastSurfer Teaser Image
    :align: center
 
+.. include:: START.md
+   :parser: myst_parser.sphinx_
+
 .. toctree::
    :hidden:
 
    overview/index
    scripts/index
-   api/index
+   api/index
diff --git a/doc/overview/DOCKER.md b/doc/overview/DOCKER.md
@@ -0,0 +1,7 @@
+# Docker Support
+
+```{include} ../../Docker/README.md
+:relative-docs: doc/
+:relative-images:
+:start-line: 1
+```
diff --git a/doc/overview/FLAGS.md b/doc/overview/FLAGS.md
@@ -1,4 +1,4 @@
-# FastSurfer Flags
+# Flags
 Next, you will need to select the `*fastsurfer-flags*` and replace `*fastsurfer-flags*` with your options. Please see the Examples below for some example flags.
 
 The `*fastsurfer-flags*` will usually include the subject directory (`--sd`; Note, this will be the mounted path - `/output` - for containers), the subject name/id (`--sid`) and the path to the input image (`--t1`). For example:

diff --git a/doc/overview/INTRO.md b/doc/overview/INTRO.md
@@ -0,0 +1,25 @@
+Introduction to FastSurfer
+==========================
+
+there is a test here: {{ test }}
+
+```{include} ../../README.md
+:relative-docs: doc/
+:relative-images:
+:start-after: <!-- start of getting started -->
+:end-before: <!-- start of flags -->
+```
+
+```{include} ../../README.md
+:relative-docs: doc/
+:relative-images:
+:start-after: <!-- start of image requirements -->
+:end-before: <!-- end of image requirements -->
+```
+
+```{include} ../../README.md
+:relative-docs: doc/
+:relative-images:
+:start-after: <!-- start of system requirements -->
+:end-before: <!-- start of acknowledgements -->
+```