MibDeep¶

class controllers.MibDeep¶

Bases: handle

MIBDEEP - @type MibDeep class is a template class for using with.

GUI developed using appdesigner of Matlab

obj.mibController.startController('controllers.MibDeep', obj.mibController);; // as GUI tool

// a code below was used for mibImageArithmeticController
BatchOpt.Parameter = 'test'mib;  // fill edit boxes as strings
BatchOpt.Checkbox = true;     // fill checkboxes with logicals: true/false
BatchOpt.Popup = {'value'};        // value for the popups as a cell
BatchOpt.Radio = {'Radio1'};          // selection of radio buttons, as cell with the handle of the target radio button
BatchOpt.showWaitbar = true;  // show or not the waitbar
obj.startController('MibDeep', [], BatchOpt); // start MibDeep in the batch mode

// trigger return of the possible Options using returnBatchOpt function
// using notify syncBatch event
obj.startController('MibDeep', [], NaN);

Constructor Summary

MibDeep(mibModel, varargin)¶

Property Summary

ActivationLayerOpt¶: .Fraction = .6; % augment 60% of patches .FillValue = 0; .RandXReflection = true; .RandYReflection = true; .RandZReflection = true; .Rotation90 = true; .ReflectedRotation90 = true;

Aug2DFuncNames¶: a structure with augumentation options for 2D unets, default obtained from obj.mibModel.preferences.Deep.AugOpt2D, see getDefaultParameters.m .FillValue = 0; .RandXReflection = true; .RandYReflection = true; .RandRotation = [-10, 10]; .RandScale = [.95 1.05]; .RandXScale = [.95 1.05]; .RandYScale = [.95 1.05]; .RandXShear = [-5 5]; .RandYShear = [-5 5];

Aug2DFuncProbability¶: cell array with names of 2D augmenter functions

Aug3DFuncNames¶: probabilities of each 2D augmentation action to be triggered

Aug3DFuncProbability¶: cell array with names of 2D augmenter functions

AugOpt2D¶: a structure compatible with batch operation name of each field should be displayed in a tooltip of GUI it is recommended that the Tags of widgets match the name of the fields in this structure .Parameter - [editbox], char/string .Checkbox - [checkbox], logical value true or false .Dropdown{1} - [dropdown], cell string for the dropdown .Dropdown{2} - [optional], an array with possible options .Radio - [radiobuttons], cell string ‘Radio1’ or ‘Radio2’… .ParameterNumeric{1} - [numeric editbox], cell with a number .ParameterNumeric{2} - [optional], vector with limits [min, max] .ParameterNumeric{3} - [optional], string ‘on’ - to round the value, ‘off’ to do not round the value

AugOpt3D¶: a handle for GPU info window

BatchOpt¶: containers.Map with available encoders keySet - is a mixture of workflow -space- architecture {‘2D Semantic DeepLab v3+’, ‘2D Semantic U-net +Encoder’} encoders for each “workflow -space- architecture” combination, the last value shows the selected encoder encodersList{1} = {‘Resnet18’, ‘Resnet50’, ‘Xception’, ‘InceptionResnetv2’, 1}; % for ‘2D Semantic DeepLab v3+’ encodersList{2} = {‘Classic’, ‘Resnet18’, ‘Resnet50’, 2}; % for ‘2D Semantic U-net +Encoder’ encodersList{3} = {‘Resnet18’, ‘Resnet50’, 1}; % for ‘2D Instance SOLOv2’’

DynamicMaskOpt¶: options for the activation layer

InputLayerOpt¶: options for the segmentation layer

OverlapInstancesOpt¶: options for the “COCO, frozen then trainable” two-phase schedule of the 2D Instance workflow (see controllers.MibDeep/startTrainingInstances) .MinFrozenFraction = 0.07; % smallest share of MaxEpochs before a switch is allowed .MaxFrozenFraction = 0.15; % largest share of MaxEpochs the frozen phase may use .PlateauWindowEpochs = 25; % averaging window used to call the loss flat .PlateauTolerance = 0.01; % relative improvement below which the loss is flat .TrainableLearnRate = 1e-4; % absolute learn rate of phase 2, capped at InitialLearnRate

PatchPreviewOpt¶: colors of materials

ScoreExportOpt¶: structure with preview patch options .noImages = 9, number of images in montage .imageSize = 160, patch size for preview .labelShow = true, display overlay labels with details .labelSize = 9, font size for the label .labelColor = ‘black’, color of the label .labelBgColor = ‘yellow’, color of the label background .labelBgOpacity = 0.6; % opacity of the background

SegmentationLayerOpt¶: options for stitching of instances across tiles during 2D Instance prediction (the stitching mode itself is in BatchOpt.P_OverlapInstancesMode) .DetectionThreshold = 0.5; % confidence threshold of segmentObjects [both overlap modes] .MergeIoU = 0.5; % in-band intersection-over-union to merge detections [‘IoU merge’ mode] .MergeIoA = 0.8; % in-band intersection-over-smaller-area to merge detections [‘IoU merge’ mode]

SendReports¶: structure with settings to export score files .Precision = 8; % define precision for the output scores, ‘8’ or ‘16’ bit .IncludeExterior = false; % when false the scores will also have probability for the background material

StartingWeightsOpt¶: options for calculation of dynamic masks for prediction using blocked image mode .Method = ‘Keep above threshold’; % ‘Keep above threshold’ or ‘Keep below threshold’ .ThresholdValue = 60; .InclusionThreshold = 0.1; % Inclusion threshold for mask blocks

TrainEngine¶: structure for the session settings .countLabelsDir - directory with labels to count, used in count labels function

TrainingOpt¶: send email reports with progress of the training process .T_SendReports = false; .FROM_email = ‘user@gmail.com’; .SMTP_server = ‘smtp-relay.brevo.com’; .SMTP_port = ‘587’; .SMTP_auth = true; .SMTP_starttls = true; .SMTP_sername = ‘user@gmail.com’; .SMTP_password = ‘’; .sendWhenFinished = false; .sendDuringRun = false;

TrainingProgress¶: a structure with training options, the default ones are obtained from obj.mibModel.preferences.Deep.TrainingOpt, see getDefaultParameters.m .solverName = ‘adam’; .MaxEpochs = 50; .Shuffle = ‘once’; .InitialLearnRate = 0.0005; .LearnRateSchedule = ‘piecewise’; .LearnRateDropPeriod = 10; .LearnRateDropFactor = 0.1; .L2Regularization = 0.0001; .Momentum = 0.9; .ValidationFrequency = 400; .Plots = ‘training-progress’;

availableArchitectures¶: a cell array with names of initialized child controllers

availableEncoders¶: containers.Map with available architectures keySet = {‘2D Semantic’, ‘2.5D Semantic’, ‘3D Semantic’, ‘2D Patch-wise’, ‘2D Instance’}; valueSet{1} = {‘DeepLab v3+’, ‘SegNet’, ‘U-net’, ‘U-net +Encoder’}; old: {‘U-net’, ‘SegNet’, ‘DLv3 Resnet18’, ‘DLv3 Resnet50’, ‘DLv3 Xception’, ‘DLv3 Inception-ResNet-v2’} valueSet{2} = {‘Z2C + DLv3’, ‘Z2C + DLv3’, ‘Z2C + U-net’, ‘Z2C + U-net +Encoder’}; % 3DC + DLv3 Resnet18’ valueSet{3} = {‘U-net’, ‘U-net Anisotropic’} valueSet{4} = {‘Resnet18’, ‘Resnet50’, ‘Resnet101’, ‘Xception’} valueSet{5} = {‘SOLOv2’} old: {‘SOLOv2 Resnet18’, ‘SOLOv2 Resnet50’};

childControllers¶: a cell array with handles to listeners

childControllersIds¶: list of opened subcontrollers

colormap20¶: colormap for 6 colors

colormap255¶: colormap for 20 colors

colormap6¶: handle to waitbar

gpuInfoFig¶: probabilities of each 3D augmentation action to be triggered

listener¶: handle to the view / mibDeepGUI

mibController¶: handles to mibModel

mibModel¶

modelMaterialColors¶: a structure with settings for the input layer .Normalization = ‘zerocenter’; .Mean = []; .StandardDeviation = []; .Min = []; .Max = [];

sessionSettings¶: colormap for 255 colors

view¶: handle to mib controller

wb¶: a structure to be used for the training progress plot for the compiled version of MIB .maxNoIter = max number of iteractions for during training .iterPerEpoch - iterations per epoch .stopTraining - logical switch that forces to stop training and save all progress

Method Summary

activationLayerChangeCallback()¶

ACTIVATIONLAYERCHANGECALLBACK - callback for modification of the Activation Layer dropdown.

Syntax:

obj.activationLayerChangeCallback()

balanceClasses()¶

BALANCECLASSES - balance classes before training.

Syntax:

obj.balanceClasses()

see example from here: https://se.mathworks.com/help/vision/ref/balancepixellabels.html

bioformatsCallback(event)¶

BIOFORMATSCALLBACK - update available filename extensions upon press of the BioFormats.

Syntax:

obj.bioformatsCallback(event)

checkbox

Input Arguments:

event - an event structure of appdesigner

channelWisePreProcess(imgIn)¶

CHANNELWISEPREPROCESS - function imgOut = channelWisePreProcess(obj, imgIn).

Syntax:

imgOut = obj.channelWisePreProcess(imgIn)

Normalize images As input has 4 channels (modalities), remove the mean and divide by the standard deviation of each modality independently.

Input Arguments:

imgIn - input image, as matrix [heigth, width, color, depth]

Output Arguments:

imgOut - resulting image, stretched between 0 and 1

checkNetwork(fn)¶

CHECKNETWORK - generate and check network using settings in the Train tab.

Syntax:

obj.checkNetwork(fn)

Input Arguments:

fn - optional string with filename (*.mibDeep) to preview its configuration

In MATLAB the network is opened in analyzeNetwork. The deployed version has no analyzeNetwork, so the layers are listed in a table (index, name, type, patch size, details) and the layer graph is plotted in a separate figure. The patch size is the size of the activations a layer outputs, height x width (x depth) x channels. It comes from deep.internal.sdk.forwardDataAttributes, which propagates the size of the input layer through the network without computing any activations, so it is fast and needs no memory for the data. It accepts a layerGraph, DAGNetwork or dlnetwork. The batch dimension is not shown, and for a layer with several outputs only its first output is listed. Custom layers without an OutputNames property (e.g. dicePixelCustomClassificationLayer) are addressed by their name. Being an internal function it may change between MATLAB releases; if it fails the column shows - and, in DeveloperMode, the error is printed to the command window.

closeWindow()¶

CLOSEWINDOW - callback on closing of DeepMIB window.

Syntax:

obj.closeWindow()

correctBatchOpt(res)¶

CORRECTBATCHOPT - correct loaded BatchOpt structure if it is not compatible.

Syntax:

res = obj.correctBatchOpt(res)

with the current version of DeepMIB

Input Arguments:

res - BatchOpt structure loaded from a file

countLabels()¶

COUNTLABELS - count occurrences of labels in model files.

Syntax:

obj.countLabels()

callback for press of the “Count labels” in the Options panel define directory with label files

createNetwork(previewSwitch)¶

CREATENETWORK - generate network.

Syntax:

[lgraph, outputPatchSize] = obj.createNetwork(previewSwitch)

Input Arguments:

previewSwitch - logical switch, when 1 - the generated network is only for preview, i.e. weights of classes won’t be calculated

Output Arguments:

lgraph - network object
outputPatchSize - output patch size as [height, width, depth, color]

customTrainingProgressWindow_Callback(event)¶

CUSTOMTRAININGPROGRESSWINDOW_CALLBACK - callback for click on.

Syntax:

obj.customTrainingProgressWindow_Callback(event)

obj.view.handles.O_CustomTrainingProgressWindow checkbox

duplicateConfigAndNetwork()¶

DUPLICATECONFIGANDNETWORK - copy the network file and its config to a new filename.

Syntax:

obj.duplicateConfigAndNetwork()

evaluateSegmentation()¶

EVALUATESEGMENTATION - evaluate segmentation results by comparing predicted models.

Syntax:

obj.evaluateSegmentation()

with the ground truth models check for evaluation of patches in the patch-wise mode

evaluateSegmentationPatches()¶

EVALUATESEGMENTATIONPATCHES - evaluate segmentation results for the patches in the.

Syntax:

obj.evaluateSegmentationPatches()

patch-wise mode

exploreActivations()¶

EXPLOREACTIVATIONS - explore activations within the trained network.

Syntax:

obj.exploreActivations()

exportNetwork()¶

EXPORTNETWORK - convert and export network to ONNX or TensorFlow formats.

Syntax:

obj.exportNetwork()

findBestMinibatchSize()¶

FINDBESTMINIBATCHSIZE - measure which mini-batch size gives the best throughput on this GPU.

Syntax:

obj.findBestMinibatchSize()

Input Arguments:

(none)

Output Arguments:

(none) - the measured table is shown to the user, who chooses whether to apply the recommended value to BatchOpt.T_MiniBatchSize

Trains the configured network on synthetic patches for a few iterations at each candidate mini-batch size and reports patches per second. The project images are never read and nothing is ever written to the project; the 2D Instance workflow does read a sample of the label maps, to match the object count of the synthetic patches to the real ones (see localSampleInstancesPerPatch()).

Why throughput and not a memory reading. MATLAB’s gpuDevice().AvailableMemory saturates: in a calibration sweep on a 12 GB card it read 10.91 GB used both in the last healthy configuration and in every oversubscribed one, so it cannot tell them apart. Under the Windows WDDM driver an oversized batch does not raise an error either - the driver pages GPU memory to host RAM and the run silently continues many times slower. Throughput is what actually changes, and comparing two sizes of the same network on the same card needs no calibrated threshold at all:

below the limit, a larger mini-batch processes patches faster (fixed per-iteration overhead is spread over more patches);
above it, throughput collapses as the driver starts paging.

So the best size is simply where patches per second peaks.

Iterations are timed individually, through the trainer’s own OutputFcn, and the median is taken after discarding the first two. Timing the call as a whole does not work: network setup costs around 10 seconds against per-iteration costs of a fraction of a second, so the measurement is all overhead. Subtracting two runs of different lengths does not work either - the overhead varies by more than the signal, which made short runs come out slower than long ones when this was tried.

The result is a mild upper bound. The probe runs the network alone; a real run also holds the validation set and the augmentation buffers. Reading and augmenting a patch was measured at 0.085 s against a 3.70 s iteration, so on this workflow that margin is around 9% and the timing below matches a recorded run to 1%. If the recommended value is the largest one tested, the true optimum may be higher still.

For 2D Instance the probe deliberately measures with FreezeSubNetwork set to 'none', the most memory-hungry case, because the two-phase schedule ends up there and a size chosen against the cheaper frozen phase would fail once the backbone is unfrozen.

How many instances a synthetic patch carries is measured, not assumed. SOLOv2’s cost is driven by the ground-truth instance count: the loss assigns every instance across the FPN levels and the target is an [h w K] logical stack. This used to generate a fixed 5 objects per patch, and on a mitochondria project whose patches actually hold a median of 31 the probe came out 2.1x too fast (17.4 epochs/hour predicted against 8.3 recorded) while modelling a mask tensor six times smaller than the real one - 11.8 MB per iteration against 74 MB. Understating the memory is the worse half of that, because it lets the probe recommend a size that then pages in the real run. localSampleInstancesPerPatch() therefore reads a sample of the project’s own label maps first. Only the label maps are read, never the images, and a project that is not preprocessed yet falls back to the old constant.

Measured against that same recorded run - Resnet50, 768x768, mini-batch 4, whose real iteration took 3.700 s:

objects per synthetic patch	sec/iteration	epochs/hour
5 (the old constant)	1.618	19.0
31 (measured from labels)	3.732	8.2
real run	3.700	8.3

A 2D U-net at 256x256 on a 12 GB card, repeated twice and agreeing to 0.1%:

size	sec/iteration	patches/second
1	0.0254	39.40
2	0.0353	56.72
4	0.0539	74.16
8	0.1001	79.95
16	0.2346	68.21
32	0.5983	53.48

The decline past the peak here is ordinary diminishing returns. Running out of memory looks entirely different: with Resnet50 at 768x768 on the same card, an iteration went from around 2 s at mini-batch 4 to 63 s at mini-batch 8.

generate3DDeepLabV3Network(imageSize, numClasses, targetNetwork)¶

GENERATE3DDEEPLABV3NETWORK - generate a hybrid 2.5D DeepLabv3+ convolutional neural network for semantic image.

Syntax:

net = obj.generate3DDeepLabV3Network(imageSize, numClasses, targetNetwork)

segmentation. The training data should be a small substack of 3,5,7 etc slices, where only the middle slice is segmented. As the update, a 2 blocks of 3D convolutions were added before the standard DLv3

Input Arguments:

imageSize - vector [height, width, colors] defining input patch size, should be larger than [224 224] for resnet18, colors should be 3
numClasses - number of output classes (including exterior) for the output results
targetNetwork - string defining the base architecture for the initialization ‘resnet18’ - resnet18 network ‘resnet50’ - resnet50 network ‘xception’ - xception network ‘inceptionresnetv2’ - resnet50 network

generateDeepLabV3Network(imageSize, numClasses, targetNetwork)¶

GENERATEDEEPLABV3NETWORK - generate DeepLab v3+ convolutional neural network for semantic image.

Syntax:

net = obj.generateDeepLabV3Network(imageSize, numClasses, targetNetwork)

segmentation of 2D RGB images

Input Arguments:

imageSize - vector [height, width, colors] defining input patch size, should be larger than [224 224] for resnet18, colors should be 3
numClasses - number of output classes (including exterior) for the output results
targetNetwork - string defining the base architecture for the initialization ‘resnet18’ - resnet18 network ‘resnet50’ - resnet50 network ‘xception’ - xception network (requires matlab) ‘inceptionresnetv2’ - inceptionresnetv2 network (required matlab)

generateDynamicMaskingBlocks(vol, blockSize, noColors)¶

GENERATEDYNAMICMASKINGBLOCKS - generate blocks using dynamic masking parameters acquired in obj.DynamicMaskOpt.

Syntax:

bls = obj.generateDynamicMaskingBlocks(vol, blockSize, noColors)

Input Arguments:

vol - blocked image to process
blockSize - block size
noColors - number of color channels in the blocked image

Output Arguments:

bls - calculated blockLocationSet .ImageNumber .BlockOrigin .BlockSize .Levels

generateUnet2DwithEncoder(imageSize, encoderNetwork)¶

GENERATEUNET2DWITHENCODER - generate Unet convolutional neural network for semantic image.

Syntax:

[net, outputSize] = obj.generateUnet2DwithEncoder(imageSize, encoderNetwork)

segmentation of 2D RGB images using a specified encoder

Input Arguments:

imageSize - vector [height, width, colors] defining input patch size, should be larger than [224 224] for Resnet18, colors should be 3
encoderNetwork - string defining the base architecture for the initialization ‘Classic’ - classic unet architecture ‘Resnet18’ - Resnet18 network ‘Resnet50’ - Resnet50 network

Output Arguments:

net - Unet dlnetwork, with softmax (Name: ‘FinalNetworkSoftmax-Layer’) as the final layer
outputSize - output size of the network returned as [height, width, number of classes]

gpuInfo()¶

GPUINFO - display information about the selected GPU.

Syntax:

obj.gpuInfo()

helpButton_callback()¶

HELPBUTTON_CALLBACK - show Help sections.

Syntax:

obj.helpButton_callback()

importNetwork()¶

IMPORTNETWORK - import an externally trained or designed network to be used.

Syntax:

obj.importNetwork()

with DeepMIB

Example: % generate a network: net = deeplabv3plusLayers([512 512 3], 5, ‘resnet18’); % save network to a file save(‘myNewNetwork.mat’, ‘net’, ‘-mat’); % use Import opetation to load and adapt the network for use with DeepMIB

loadConfig(configName)¶

LOADCONFIG - load config file with Deep MIB settings.

Syntax:

obj.loadConfig(configName)

Input Arguments:

configName - full filename for the config file to load

mergeInstancesTo3D()¶

MERGEINSTANCESTO3D - merge predicted 2D instance models into a 3D instance model.

Syntax:

obj.mergeInstancesTo3D()

Takes the instance models produced by 2D instance prediction (startPredictionInstances(), saved under ResultingImagesDir/PredictionImages/ResultsModels) and stitches their slices into 3D instance models: objects overlapping between neighbouring slices are linked into one 3D instance with a consistent index through the whole stack (via utils.instances.stitch2Dto3D()).

Two layouts of the prediction results are recognized automatically from the depth of the first model file:

2D models (one Z-slice per file) - all files form a single stack, taken in alphabetical order of their filenames, so the prediction images must be named in their correct Z-order. One merged 3D model is written.

3D models (a z-stack per file, produced when the prediction images were themselves z-stacks) - every file is stitched independently and one merged 3D model is written per input file.

The user is asked for the stitching settings, then for the destination: a filename and format for a single output, or a folder and format when several stacks are stitched. The merged model can be written as a single 3D file or as a sequence of 2D files depending on the selected format/policy.

Input Arguments:: (none)
Output Arguments:: (none)

preprareTrainingOptions(valDS)¶

PREPRARETRAININGOPTIONS - prepare trainig options for the network training.

Syntax:

TrainingOptions = obj.preprareTrainingOptions(valDS)

Input Arguments:

valDS - datastore with images for validation

preprareTrainingOptionsInstances(valDS, trainingOptOverrides)¶

PREPRARETRAININGOPTIONSINSTANCES - prepare trainig options for training of the instance segmentation network.

Syntax:

TrainingOptions = obj.preprareTrainingOptionsInstances(valDS)
TrainingOptions = obj.preprareTrainingOptionsInstances(valDS, trainingOptOverrides)

Input Arguments:

valDS - datastore with images for validation
trainingOptOverrides - optional struct whose fields replace the matching fields of obj.TrainingOpt for this call only. Used by the two-phase “frozen then trainable” schedule, where each phase needs its own MaxEpochs and InitialLearnRate without disturbing what the user configured (see startTrainingInstances()). obj.TrainingOpt is never modified.

Output Arguments:

TrainingOptions - options object accepted by trainSOLOV2

previewDynamicMask()¶

PREVIEWDYNAMICMASK - preview results for the dynamic mode.

Syntax:

obj.previewDynamicMask()

previewImagePatches_Callback(event)¶

PREVIEWIMAGEPATCHES_CALLBACK - callback for value change of obj.view.handles.O_PreviewImagePatches.

Syntax:

obj.previewImagePatches_Callback(event)

previewModels(loadImagesSwitch)¶

PREVIEWMODELS - load images for predictions and the resulting modelsinto MIB.

Syntax:

obj.previewModels(loadImagesSwitch)

Input Arguments:

loadImagesSwitch - [logical], load or not (assuming that images have already been preloaded) images. When true, both images and models are loaded, when false - only models are loaded

previewPredictions()¶

PREVIEWPREDICTIONS - load images of prediction scores into MIB.

Syntax:

obj.previewPredictions()

previewValidationPatches()¶

PREVIEWVALIDATIONPATCHES - show or export the patches that will be used for validation.

Syntax:

obj.previewValidationPatches()

The validation patches are cropped at random positions, and which positions are used is decided by BatchOpt.T_RandomGeneratorValSeed. This inspects the patches that seed produces, so a seed can be judged before spending a training run on it - a draw that happens to land mostly on background makes the validation loss a poor guide, and OutputNetwork: best-validation-loss then selects against a set that does not represent the data.

A dialog offers two ways to look:

collage - a montage of the first PatchPreviewOpt.noImages patches, scaled down to PatchPreviewOpt.imageSize. The appearance settings are shared with the augmentation preview, so changing them there changes both.
export - every validation patch written to ScoreNetwork/ValidationPatches at full resolution, each as an image plus a MIB model holding its labels, so the pair can be opened and inspected in MIB.

Behaviour per workflow, matching what training actually does:

2D / 2.5D / 3D Semantic - patches from a randomPatchExtractionDatastore over the validation images and labels, drawn under the validation seed.
2D Instance - patches from deepmib.readInstancePatch() using the same per-observation seeds as startTrainingInstances(), so these are literally the patches that will be validated on.
2D Patch-wise - validation uses whole images from a fixed file list and never moves, so there is nothing for a seed to change.

Input Arguments:

(none)

Output Arguments:

(none)

processBlocksBlockedImage(vol, zValue, net, inputPatchSize, outputPatchSize, blockSize, padShift, dataDimension, patchwiseWorkflowSwitch, patchwisePatchesPredictSwitch, classNames, generateScoreFiles, executionEnvironment, fn, pwb)¶

PROCESSBLOCKSBLOCKEDIMAGE - Segment one image volume using the blockedImage overlap-tile strategy.

Syntax:

[outputLabels, scoreImg, cancelled] = obj.processBlocksBlockedImage(vol, zValue, net, inputPatchSize, outputPatchSize, blockSize, padShift, dataDimension, patchwiseWorkflowSwitch, patchwisePatchesPredictSwitch, classNames, generateScoreFiles, executionEnvironment, fn, pwb)

Converts the input volume to a blockedImage, divides it into tiles of blockSize (with optional border padding for overlap or valid-padding networks), calls deepmib.segmentBlockedImage on every tile via blockedImage/apply, then gathers and crops the results back to the original image extent. Supports 2D, 2.5D, and 3D networks, patch-wise classification, dynamic masking, and optional score-map generation.

Input Arguments:

obj - MibDeep controller instance
vol - input image volume - 2D - [height, width, colors] - 2.5D/3D - [height, width, depth, colors]
zValue - z-slice index used when building output filenames for the 2.5D per-slice loop; pass NaN for full-volume (2D / 3D) calls
net - trained deep learning network loaded from the network file
inputPatchSize - patch size expected by the network [height, width, colors] or [height, width, depth, colors]
outputPatchSize - network output patch size; equals inputPatchSize for ‘same’ padding, smaller for ‘valid’ padding
blockSize - effective tile footprint passed to blockedImage/apply; already reduced by padShift when overlap-tile mode is active
padShift - border overlap in pixels [height, width] or [height, width, depth]; zero when overlap-tile mode is off
dataDimension - numeric, dataset/network dimensionality - 2 - 2D network - 2.5 - 2.5D (Z-context) network - 3 - 3D network
patchwiseWorkflowSwitch - logical; true for the ‘2D Patch-wise’ workflow where the Exterior class is not removed and per-patch CSV files are written
patchwisePatchesPredictSwitch - logical; true when prediction images are stored in class-named subfolders (patch classification mode) - skips the gather/crop post-processing
classNames - cell array of class name strings loaded from the network file
generateScoreFiles - score-file format selector - 0 - do not generate score files - 1 - AmiraMesh (.am) - 2 - MATLAB non-compressed (.mibImg) - 3 - MATLAB compressed (.mibImg) - 4 - MATLAB non-compressed, range 0-1 (.mat)
executionEnvironment - string passed to segmentBlockedImage - ‘cpu’ - CPU only - ‘gpu’ - single GPU - ‘multi-gpu’ - multiple GPUs (patch-wise only) - ‘parallel’ - parallel pool
fn - base filename (no extension) of the current image; used when writing per-patch CSV score/label files
pwb - (optional) uiprogressdlg handle used to check for user cancellation between the pre-apply and post-apply stages; pass [] when no progress dialog is active

Output Arguments:

outputLabels - predicted label matrix (uint8); [] when cancelled
scoreImg - probability/score map array; [] when cancelled, 0 when generateScoreFiles == 0
cancelled - logical; true when the user pressed Cancel on pwb

Usage:

Example 1 - Typical call from startPredictionBlockedImage (2D full-volume path):

% Typical call from startPredictionBlockedImage (2D full-volume path):
[outputLabels, scoreImg, cancelled] = obj.processBlocksBlockedImage( ...
    vol, NaN, net, inputPatchSize, outputPatchSize, blockSize, padShift, ...
    2, false, false, classNames, 0, 'gpu', 'myImage', pwb);
if cancelled; close(pwb); return; end

processImages(preprocessFor)¶

PROCESSIMAGES - Preprocess images for training and prediction.

Syntax:

obj.processImages(preprocessFor)

Input Arguments:

preprocessFor - a string with target, ‘training’, ‘prediction’

processImagesForInstanceSegmentation(preprocessFor)¶

PROCESSIMAGESFORINSTANCESEGMENTATION - Preprocess labels for 2D instance segmentation for training and prediction.

Syntax:

obj.processImagesForInstanceSegmentation(preprocessFor)

as result, mat-files with the following variables are created: - instanceBoxes, matrix [N×4 double] containing bounding box coordinates of objects, where N is a number of objects on the image - instanceNames, array [N×1 categorical] containing names of objects, currently the same name should be used for all objects, can be any string converted to categorical. - instanceMasks, matrix [720×1280×N logical] binary masks where each slice represents individual object that should match the corresponding entry in instanceBoxes and instanceNames

Input Arguments:

preprocessFor - a string with target, ‘training’, ‘prediction’

returnBatchOpt(BatchOptOut)¶

RETURNBATCHOPT - return structure with Batch Options and possible configurations.

Syntax:

obj.returnBatchOpt(BatchOptOut)

via the notify ‘SyncBatch’ event

Input Arguments:

BatchOptOut - a local structure with Batch Options generated during Continue callback. It may contain more fields than obj.BatchOpt structure

saveCheckpointNetworkCheck()¶

SAVECHECKPOINTNETWORKCHECK - callback for press of Save checkpoint networks (obj.view.handles.T_SaveProgress).

Syntax:

obj.saveCheckpointNetworkCheck()

saveConfig(configName)¶

SAVECONFIG - save Deep MIB configuration to a file.

Syntax:

obj.saveConfig(configName)

Input Arguments:

configName - [optional] string, full filename to the config file

selectArchitecture(event)¶

SELECTARCHITECTURE - select the target architecture.

Syntax:

obj.selectArchitecture(event)

selectDirerctories(event)¶

SELECTDIRERCTORIES - select directories containing images for training and prediction.

Syntax:

obj.selectDirerctories(event)

selectGPUDevice()¶

SELECTGPUDEVICE - select environment for computations.

Syntax:

obj.selectGPUDevice()

selectNetwork(networkName)¶

SELECTNETWORK - select a filename for a new network in the Train mode, or.

Syntax:

net = obj.selectNetwork(networkName)

select a network to use for the Predict mode

Input Arguments:

networkName - optional parameter with the network full filename

Output Arguments:

net - trained network

selectWorkflow(event)¶

SELECTWORKFLOW - select deep learning workflow to perform.

Syntax:

obj.selectWorkflow(event)

sendReportsCallback()¶

SENDREPORTSCALLBACK - define parameters for sending progress report to the user’s.

Syntax:

obj.sendReportsCallback()

email address

setActivationLayerOptions()¶

SETACTIVATIONLAYEROPTIONS - update options for the activation layers.

Syntax:

obj.setActivationLayerOptions()

setAugFuncHandles(mode, augOptions)¶

SETAUGFUNCHANDLES - define list of 2D/3D augmentation functions.

Syntax:

[status, augNumber] = obj.setAugFuncHandles(mode, augOptions)

Input Arguments:

mode - string defining ‘2D’ or ‘3D’ augmentations
augOptions - a custom temporary structure with augmentation options to be used instead of obj.AugOpt2D and obj.AugOpt3D. It is used by mibDeepAugmentSettingsController to preview selected augmentations

Output Arguments:

status - a logical success switch (1-success, 0- fail)
augNumber - number of selected augmentations

setAugmentationSettings(mode)¶

SETAUGMENTATIONSETTINGS - update settings for augmentation for 2D or 3D networks.

Syntax:

obj.setAugmentationSettings(mode)

setInputLayerSettings()¶

SETINPUTLAYERSETTINGS - update init settings for the input layer of networks.

Syntax:

obj.setInputLayerSettings()

setSegmentationLayer()¶

SETSEGMENTATIONLAYER - callback for modification of the Segmentation Layer dropdown.

Syntax:

obj.setSegmentationLayer()

setSegmentationLayerOptions()¶

SETSEGMENTATIONLAYEROPTIONS - update options for the activation layers.

Syntax:

obj.setSegmentationLayerOptions()

setStartingWeightsSettings()¶

SETSTARTINGWEIGHTSSETTINGS - update settings of the two-phase “frozen then trainable” schedule.

Syntax:

obj.setStartingWeightsSettings()

The settings are stored in obj.StartingWeightsOpt and are only used by the 2D Instance workflow when BatchOpt.T_StartingWeights is 'COCO, frozen then trainable' (see startTrainingInstances()).

Input Arguments:: (none)
Output Arguments:: (none)

setTrainingSettings()¶

SETTRAININGSETTINGS - update settings for training of networks.

Syntax:

obj.setTrainingSettings()

singleModelTrainingFileValueChanged(event)¶

SINGLEMODELTRAININGFILEVALUECHANGED - callback for press of SingleModelTrainingFile.

Syntax:

obj.singleModelTrainingFileValueChanged(event)

start(event)¶

START - start calcualtions, depending on the selected tab.

Syntax:

obj.start(event)

preprocessing, training, or prediction is initialized

startController(controllerName, varargin)¶

STARTCONTROLLER - Launch a child controller by class name.

Syntax:

obj.startController(controllerName, varargin)

Delegates to utils.startController - see that function for full documentation of interactive, batch, and lifecycle behaviour.

Input Arguments:

controllerName - char - fully-qualified controller class name
varargin - additional arguments forwarded to the child constructor

Usage:

Example 1:

obj.startController('controllers.MibDeepController');

Example 2:

BatchOpt.Mode = '3D, Stack';
obj.startController('controllers.HistThres', [], BatchOpt);

startPrediction2D()¶

STARTPREDICTION2D - predict datasets for 2D taken to a separate function for.

Syntax:

obj.startPrediction2D()

better performance

startPrediction3D()¶

STARTPREDICTION3D - predict datasets for 3D networks taken to a separate function.

Syntax:

obj.startPrediction3D()

to improve performance

startPredictionBlockedImage()¶

STARTPREDICTIONBLOCKEDIMAGE - predict 2D/3D datasets using the blockedImage class.

Syntax:

obj.startPredictionBlockedImage()

requires R2021a or newer

startPredictionInstances()¶

STARTPREDICTIONINSTANCES - predict 2D instance segmentation (SOLOv2) datasets.

Syntax:

obj.startPredictionInstances()

Runs the trained SOLOv2 network over the prediction images using the blockedImage overlap-tile strategy (so large / whole-slide images are processed at native resolution instead of being downscaled to the network input). For each image, every detected object instance is saved as a unique integer index in a MIB model (background 0).

An image that is no larger than the network input (inputPatchSize) is passed to segmentObjects whole: it needs no tiles, so no stitching mode is applied and the tiling settings (P_OverlappingTiles / P_OverlappingTilesPercentage) are neither used nor validated for it.

Both 2D images and z-stacks are accepted, as in the 2D Semantic workflow:

a 2D file is predicted directly and saved as a 2D model;

a file with several z-slices is predicted slice-by-slice and saved as a single 3D model.

The instance indices are contiguous 1..N within each slice and are not consistent between slices - linking them into 3D objects is the job of the “Merge 2D to 3D” button (mergeInstancesTo3D), which ignores the input indices anyway.

Cross-tile stitching mode is selected with BatchOpt.P_OverlapInstancesMode:

‘Centroid in core’ (see deepmib.segmentBlockedImageInstances) - objects are emitted by the tile that owns their centroid, requiring the overlap (P_OverlappingTilesPercentage) to be >= the largest object;

‘IoU merge’ (see deepmib.segmentImageInstancesIoUMerge) - all per-tile detections are kept and merged across seams when their masks agree inside the shared overlap band; works for objects larger than the overlap.

Both modes finish by splitting every index into its connected components (utils.instances.splitDisconnected), so two objects that are separated in the image never share an instance index.

The detection confidence threshold, the merge IoU/IoA thresholds and the minimal object area are taken from obj.OverlapInstancesOpt (updateOverlapInstancesSettings). Instance prediction does not require preprocessing: images are read directly.

startPreprocessing()¶

STARTPREPROCESSING - preprocess imaging for training and prediction.

Syntax:

obj.startPreprocessing()

startTraining()¶

STARTTRAINING - perform training of the network.

Syntax:

obj.startTraining()

startTrainingInstances()¶

STARTTRAININGINSTANCES - perform training of instance segmentation network.

Syntax:

obj.startTrainingInstances()

static tif3DFileRead(filename)¶

TIF3DFILEREAD - data = tif3DFileRead(filename).

Syntax:: function data = tif3DFileRead(filename)

custom reading function to load tif files with stack of images used in evaluate segmentation function

toggleAugmentations()¶

TOGGLEAUGMENTATIONS - callback for press of the T_augmentation checkbox.

Syntax:

obj.toggleAugmentations()

transferLearning()¶

TRANSFERLEARNING - perform fine-tuning of the loaded network to a different.

Syntax:

obj.transferLearning()

number of classes

updateActivationLayers(lgraph)¶

UPDATEACTIVATIONLAYERS - update the activation layers depending on settings in.

Syntax:

lgraph = obj.updateActivationLayers(lgraph)

obj.BatchOpt.T_ActivationLayer and obj.ActivationLayerOpt

updateBatchOptFromGUI(event)¶

UPDATEBATCHOPTFROMGUI - update obj.BatchOpt from widgets of GUI.

Syntax:

obj.updateBatchOptFromGUI(event)

use an external function (utilsupdateBatchOptFromGUI_Shared.m) that is common for all tools compatible with the Batch mode

Input Arguments:

event - event from the callback

updateConvolutionLayers(lgraph)¶

UPDATECONVOLUTIONLAYERS - update the convolution layers by providing new set of weight.

Syntax:

lgraph = obj.updateConvolutionLayers(lgraph)

initializers

updateDynamicMaskSettings()¶

UPDATEDYNAMICMASKSETTINGS - update settings for calculation of dynamic masks during.

Syntax:

obj.updateDynamicMaskSettings()

prediction using blockedimage mode the settings are stored in obj.DynamicMaskOpt

updateImageDirectoryPath(event)¶

UPDATEIMAGEDIRECTORYPATH - update directories with images for training, prediction and.

Syntax:

obj.updateImageDirectoryPath(event)

results

updateMaxPoolAndTransConvLayers(lgraph, poolSize)¶

UPDATEMAXPOOLANDTRANSCONVLAYERS - update maxPool and TransposedConvolution layers depending.

Syntax:

lgraph = obj.updateMaxPoolAndTransConvLayers(lgraph, poolSize)

on network downsampling factor only for U-net and SegNet. This function is applied when the network downsampling factor is different from 2

updateNetworkInputLayer(lgraph, inputPatchSize)¶

UPDATENETWORKINPUTLAYER - update the input layer settings for lgraph.

Syntax:

lgraph = obj.updateNetworkInputLayer(lgraph, inputPatchSize)

paramters are taken from obj.InputLayerOpt

updateOverlapInstancesSettings()¶

UPDATEOVERLAPINSTANCESSETTINGS - update settings for stitching of instances across tiles.

Syntax:

obj.updateOverlapInstancesSettings()

Settings for prediction in the 2D Instance workflow; the stitching mode itself is selected with the “Overlap mode” dropdown (BatchOpt.P_OverlapInstancesMode) and the values below are stored in obj.OverlapInstancesOpt:

.DetectionThreshold - confidence threshold of segmentObjects [both overlap modes]

.MergeIoU - in-band intersection-over-union to merge detections [“IoU merge” mode]

.MergeIoA - in-band intersection-over-smaller-area to merge detections [“IoU merge” mode]

.MinSplitArea - min area, px, of a component kept when a stitched label is split into one index per connected object [both overlap modes]

updatePreprocessingMode()¶

UPDATEPREPROCESSINGMODE - callback for change of selection in the Preprocess for dropdown.

Syntax:

obj.updatePreprocessingMode()

updateScoreExportSettings()¶

UPDATESCOREEXPORTSETTINGS - update export settings for score files.

Syntax:

obj.updateScoreExportSettings()

updateSegmentationLayer(lgraph, classNames)¶

UPDATESEGMENTATIONLAYER - redefine the segmentation layer of lgraph based on.

Syntax:

lgraph = obj.updateSegmentationLayer(lgraph, classNames)

obj.BatchOpt settings

Input Arguments:

classNames - cell array with class names, when not provided is ‘auto’ switch is used

updateStartingWeightsList()¶

UPDATESTARTINGWEIGHTSLIST - refresh the “Starting weights” dropdown for the current network design.

Syntax:

obj.updateStartingWeightsList()

Where the initial weights of a network come from, and how much of the network is retrained, is a property of the (workflow, architecture, encoder) combination rather than a free choice. This method rebuilds BatchOpt.T_StartingWeights{2} (the list of states that are actually possible for the current design), reconciles {1} when the previous value is no longer in the list, and enables the dropdown only where a genuine choice exists.

Combinations that offer no choice still display their single truthful value in a disabled dropdown, rather than being blanked or hidden: a network that is silently ImageNet-pretrained (DeepLab v3+) or that trains with a frozen backbone (SOLOv2) should say so.

Available states:

'None (random)' - weights are initialized randomly
'Pretrained' - the network starts from an already trained template. What that template is depends on the design: DeepLab v3+ with a Resnet encoder downloads a MIB-hosted template and asks whether to fetch the Electron Microscopy (_sbem) or Light microscopy/Pathology (_pathology) variant, while the U-net Resnet encoders come from mib.helsinki.fi/web-update/encoders. The template is cached in preferences.ExternalDirs.DeepMIBDir and reused silently afterwards, so which variant is in use is not recorded in the config and cannot be reported here.
'ImageNet' - 2D Patch-wise classification networks initialized from the MathWorks ImageNet-pretrained weights (requires the matching support package)
'COCO, frozen then trainable' (default) - the two-phase schedule: train with the backbone frozen, then unfreeze it and continue at a much lower rate. Phase 1 ends as soon as the training loss goes flat, or at a cap, whichever comes first; the epochs it does not use go to phase 2. Configured through obj.StartingWeightsOpt (see setStartingWeightsSettings()) and run by startTrainingInstances()
'COCO, trainable backbone' - SOLOv2 from COCO weights, the whole network keeps training (trainSOLOV2(..., 'FreezeSubNetwork', 'none')), so the features adapt to microscopy data. The initial learning rate must come down to about 1e-4: at the rates that suit a frozen backbone (1e-3 to 1e-2) the pretrained backbone is destroyed within the first hundred iterations, the loss flatlines and the network detects nothing. The reliable way to use it is to continue training an already trained frozen network, which applies the current freeze setting to the restored weights (see startTrainingInstances())
'COCO, frozen backbone' - SOLOv2 from COCO weights, the backbone stays at its COCO values ('FreezeSubNetwork', 'backbone'); faster, tolerant of a high learning rate and less prone to overfitting on very small datasets

'ImageNet' is dropped from the list in the deployed (standalone) version, where the pretrained networks are not shipped - see createNetwork().

Input Arguments:: (none)
Output Arguments:: (none)

updateWidgets()¶

UPDATEWIDGETS - update widgets of this window.

Syntax:

obj.updateWidgets()

static viewListner_Callback(src, evnt)¶