Deep MIB - Network Panel¶
Configuration of workflows and network architectures for deep learning segmentation in Microscopy Image Browser.
Overview¶
The Network panel in Deep MIB occupies the upper part of the interface and is used to select the workflow and convolutional network architecture for training deep learning models.
Workflows¶
Start a new project by selecting a workflow:
- 2D Semantic: clusters 2D image pixels of the same material together
- 2.5D Semantic: uses 2D network architectures to process small subvolumes (3-9 stacks), segmenting only the central slice
- 3D Semantic: clusters 3D image voxels of the same material together
- 2D Patch-wise: predicts 2D images in blocks (patches), producing a downsampled image indicating object positions
- 2D Instance: detects each object individually (each object gets a distinct label), using the SOLOv2 network. See 2D Instance Segmentation (SOLOv2)
2D Semantic workflow¶
Application of DeepMIB for 2D semantic segmentation of mitochondria on TEM images
list of available network architectures for 2D semantic segmentation
The following architectures are available for 2D semantic segmentation:
-
2D U-net: a convolutional neural network developed for biomedical image segmentation at the University of Freiburg, Germany. it segments a 512x512 image in under a second on a modern GPU (Wikipedia).
References:- Ronneberger, O., et al. "U-Net: Convolutional Networks for Biomedical Image Segmentation." MICCAI, 2015 (arXiv).
- MATLAB U-Net layers.
-
2D SegNet (MATLAB R2025b and older only): a convolutional network from the University of Cambridge, UK, designed for general image segmentation, less suited for microscopy data compared to U-net. MATLAB removed the function that builds it in R2026a, so the architecture is not offered there; projects that used it open with DeepLab v3+ instead and need retraining.
References:- Badrinarayanan, V., et al. "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation." arXiv, 2015 (arXiv).
- MATLAB SegNet layers.
-
2D DeepLabV3 (recommended): an efficient DeepLab v3+ network with selectable base networks (Resnet18, Resnet50, Xception, Inception-ResNet-v2), suitable for diverse segmentation tasks with grayscale or RGB inputs.
- Resnet18: 18-layer network, lightest option, quickest with low GPU needs, 224x224 input size, initialized with a pretrained EM/pathology template (IEEE).
- Resnet50: 50-layer network, balanced performance, 224x224 input size, initialized with a pretrained EM/pathology template (IEEE).
- Xception (MATLAB version only): 71-layer network, 299x299 input size (arXiv).
- Inception-ResNet-v2 (MATLAB version only): 164-layer network, high GPU demands, 299x299 input size (AAAI).
Reference: - Chen, L., et al. "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation." ECCV, 2018 (arXiv).
2.5D Semantic workflow¶
Architectures available in 2.5D workflows
Comparison of 2.5D vs. 2D results:
Available architectures (see 2D semantic section for details):
- Z2C + DLv3: DeepLab v3 with Resnet18 or Resnet50 encoders
- Z2C + U-net: standard U-net as the template
- Z2C + U-net + Encoder: U-net with Resnet18 or Resnet50 encoders
Notes on application of 2.5D workflows
Work with 2.5D workflows mirrors 2D workflows, with these differences:
- Input images and labels: use small substacks (3-9 sections), segmenting only the central slice, saved as separate files
- Generation of patches for training: auto-generate subvolumes via:
- From models:
Ribbon → Model (or Masks) → Model (Mask) statistics → detect objects → right-click → Crop to a file
- From annotations:
Ribbon → Model → Annotations → right-click over selected annotations → Crop out patches around selected annotations
See Generation of patches for deep learning segmentation
3D Semantic workflow¶
The 3D Semantic workflow suits anisotropic or slightly anisotropic 3D datasets, leveraging multiple sections to train networks for enhanced 3D structure prediction.
List of available network architectures for 3D semantic segmentation
- 3D U-net: a U-net variant for volumetric image segmentation.
References:- Cicek, Ö., et al. "3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation." MICCAI, 2016 (arXiv).
- MATLAB 3D U-Net layers.
- 3D U-net anisotropic: a hybrid of 2D and 3D U-nets, with 2D convolutions and max pooling at the top layer and 3D operations elsewhere, ideal for anisotropic voxels.
architecture of 3D U-net anisotropic
2D Patch-wise workflow¶
In the 2D Patch-wise workflow, training uses image patches representing specific classes. prediction processes images in blocks (with or without overlap), assigning each block a class. this is useful for quickly locating objects or targeting specific areas for semantic segmentation.
Detection of nuclei using the 2D patch-wise workflow
Examples of patches for nuclei detection:
Prediction results:
- Green: predicted nuclei locations
- Red: predicted background
- Uncolored: skipped patches (dynamic masking)
Detection of spots using the 2D patch-wise workflow
Training patches for "spots" and "background":
Synthetic spot detection example:
List of available networks for the 2D patch-wise workflow
Comparison of different network architectures
Indicative speed comparison (credit: MathWorks Inc.):
Reference: - He, K., et al. "Deep Residual Learning for Image Recognition." IEEE CVPR, 2016.
References:
- He, K., et al. "Deep Residual Learning for Image Recognition." IEEE CVPR, 2016.
- Keras Resnet50.
References:
- He, K., et al. "Deep Residual Learning for Image Recognition." IEEE CVPR, 2016.
- GitHub deep-residual-networks.
Reference: - Chollet, F. "Xception: Deep Learning with Depthwise Separable Convolutions." arXiv, 2017 (arXiv).
2D Instance workflow¶
Unlike the semantic workflows, which assign every pixel to a class (all objects of the same type share one label), the 2D Instance workflow detects each object individually — every object receives its own distinct label, so touching or overlapping objects of the same type are separated. Deep MIB implements this using the MATLAB SOLOv2 network (Resnet18 or Resnet50 backbone).
This workflow requires a dedicated preprocessing step that converts the ground-truth models into the SOLOv2 annotation format before training, and it needs the Computer Vision Toolbox Model for SOLO V2 Instance Segmentation support package.
For full details on source data requirements, preprocessing, training and prediction, see 2D Instance Segmentation (SOLOv2).
Network filename¶
The button selects a file for saving or loading a network:
- In Directories and Preprocessing or Train tabs: defines the save file
- In Predict tab: loads a pretrained network for prediction
Info
The button is color-coded to match the active tab for ease of use.
GPU dropdown¶
The dropdown sets the execution environment:
- Name of a GPU: lists available GPUs; select one for deep learning
- Multi-GPU: uses multiple GPUs on one machine via a local parallel pool (shown if multiple GPUs are present)
- CPU only: uses a single CPU
- Parallel (under development): uses a local or remote parallel pool, prioritizing GPU workers if available
Click the button for GPU details.
GPU information dialog
Eye button¶
The button loads and checks the network specified in Network filename, differing from the Check network button in the Train tab, which generates a network from parameters.