io — Image I/O¶

The +io package implements a factory-based I/O pipeline:

ExtensionRegistryLoad  →  LoaderFactory.create()  →  loader
                                                      ├─ loadMetadata()
                                                      └─ loadImages()

SaverFactory.create()  →  saver

Factory functions¶

class io.ExtensionRegistryLoad¶

Bases: handle

EXTENSIONREGISTRYLOAD - Registry of supported image file extensions for loading by mode and reader.

Manages a dictionary (extensionSets) of allowed filename extensions for each combination of dataset mode (Standard, Virtual, BigData, Model) and file reader (Default or BioFormats). Provides methods to resolve which loader should be used for a given filename, mode, and reader combination.

Zarr needs more than the extension. Both zarr versions share the .zarr suffix but are read by different loaders, so the store itself is probed - locally by detectZarrFormatExtension(), and over the network by probeRemoteZarr(). The remote probe also covers a URL that points straight at a nested image group and therefore has no filename extension at all, which is how OME-Zarr data is usually published (for example .../<name>.zarr/recon-1/em/fibsem-uint8).

Constructor Summary

ExtensionRegistryLoad()¶

EXTENSIONREGISTRYLOAD - Constructor for ExtensionRegistryLoad.

Syntax:

registry = io.ExtensionRegistryLoad()

Initializes the registry with default file extension sets for all supported modes and readers.

Input Arguments:

(none)

Output Arguments:

obj - instance of ExtensionRegistryLoad

Method Summary

static clearRemoteProbeCache()¶

CLEARREMOTEPROBECACHE - Drop the memoised remote zarr-version probes.

Syntax:

io.ExtensionRegistryLoad.clearRemoteProbeCache()

The probe result is cached for the whole MATLAB session, so a store that was rewritten on the server keeps reporting its old version until this is called. io.RemoteStore.clearCache is separate and clears the directory listings.

static detectZarrFormatExtension(zarrPath)¶

DETECTZARRFORMATEXTENSION - Resolve a plain “.zarr” folder to ‘zarr2’ or ‘zarr3’.

Syntax:

ext = io.ExtensionRegistryLoad.detectZarrFormatExtension(zarrPath)

The two zarr versions are read by different loaders (v3 by the native zarr-matlab library, v2 by the python backend) but share the conventional “.zarr” folder suffix, so the version has to come from the store: v3 nodes carry zarr.json, v2 nodes carry .zgroup / .zattrs / .zarray.

Remote stores (http/https/s3) are probed over the network by probeRemoteZarr(); isfile cannot see them, and without this a remote v2 store would always fall through to the v3 default and be handed to a loader that cannot read it.

Input Arguments:

zarrPath - [char] path or URL of the zarr root folder

Output Arguments:

ext - [char] 'zarr2' or 'zarr3'; 'zarr3' when the store cannot be probed (missing folder, unreachable URL), preserving the previous behaviour. Callers that must tell “not a zarr store” apart from “a v3 store” should use probeRemoteZarr() instead, which returns ''.

getAllowedExtensions(mode, reader, withDot)¶

GETALLOWEDEXTENSIONS - Get registered extensions for specified mode and reader.

Syntax:

ext = obj.getAllowedExtensions(mode, reader, withDot)

Input Arguments:

mode - [char] dataset type:
- 'Standard' - dataset loaded into memory
- 'Virtual' - dataset loaded on demand
- 'BigData' - large dataset using OME-Zarr v3
- 'Model' - segmentation labels/masks
reader - [char] file reader type:
- 'Default' - MATLAB imread, custom, and other native readers
- 'BioFormats' - BioFormats library reader
withDot - (optional) logical, default: true
- true - return extensions without leading dots (e.g. {'tif', 'png'})
- false - return extensions with leading dots (e.g. {'.tif', '.png'})

Output Arguments:

ext - cell array of [char] filename extensions

Example 1 - get Standard dataset extensions for BioFormats reader with leading dots:

ext = extReg.getAllowedExtensions('Standard', 'BioFormats', false);

Example 2 - access from MibModel with Default reader:

ext = obj.mibModel.extensionRegistryLoad.getAllowedExtensions('Standard', 'Default', false);

Example 3 - access from MibModel with BioFormats reader:

ext = obj.mibModel.extensionRegistryLoad.getAllowedExtensions('Standard', 'BioFormats', false);

static probeRemoteZarr(url)¶

PROBEREMOTEZARR - Detect the zarr version of a remote store over the network.

Syntax:

ext = io.ExtensionRegistryLoad.probeRemoteZarr(url)

Looks for the marker files that identify a zarr node: zarr.json for v3, and .zgroup / .zattrs / .zarray for v2.

On an S3-compatible host this costs a single ListObjectsV2 request, which also populates the shared listing cache that the URL browser reuses. On any other host, where nothing can be listed, it falls back to up to four ranged HEAD/GET probes.

Unlike detectZarrFormatExtension() this returns '' when the location is not a zarr store at all, so a caller can tell that apart from a v3 store and leave a plain remote image on its imread route.

Results are memoised for the session; use clearRemoteProbeCache() to force a re-probe.

Input Arguments:

url - [char|string] URL of the store root or of a group inside it

Output Arguments:

ext - [char] 'zarr2', 'zarr3', or '' when the URL is not remote, is unreachable, or carries no zarr marker

resolveLoader(filename, mode, reader)¶

RESOLVELOADER - Find the appropriate loader for a file given mode, reader, and extension.

Syntax:

loaderInfo = obj.resolveLoader(filename, mode, reader)

Extracts the filename extension and returns a structure identifying which loader should be used. If the extension is not compatible with the requested mode/reader combination, returns an error message.

Input Arguments:

filename - [char] first filename in the sequence of files to load
mode - [char] dataset type:
- 'Standard' - dataset loaded into memory
- 'Virtual' - dataset loaded on demand
- 'BigData' - large dataset using OME-Zarr v3
- 'Model' - segmentation labels/masks
reader - [char] file reader type:
- 'Default' - MATLAB imread, custom, and other native readers
- 'BioFormats' - BioFormats library reader

Output Arguments:

loaderInfo - struct encoding the loader configuration:
- .mode - [char] mode from input
- .reader - [char] reader from input
- .extension - [char] filename extension without leading dot
- .loaderId - [char] identifier of the loader to use (e.g. 'BioFormatsStd', 'imread')
When the extension is incompatible, returns a [char] error message instead.

setAllowedExtensions(mode, reader, extensionList)¶

SETALLOWEDEXTENSIONS - Update filename extensions for specified mode and reader.

Syntax:

obj.setAllowedExtensions(mode, reader, extensionList)

Updates the list of allowed extensions for a given mode/reader combination. Extensions are stored internally without leading dots for consistency.

Input Arguments:

mode - [char] dataset type:
- 'Standard' - dataset loaded into memory
- 'Virtual' - dataset loaded on demand
- 'BigData' - large dataset using OME-Zarr v3
- 'Model' - segmentation labels/masks
reader - [char] file reader type:
- 'Default' - MATLAB imread, custom, and other native readers
- 'BioFormats' - BioFormats library reader
extensionList - cell array of [char], new extension list; may include or omit leading dots (e.g. both '.tif' and 'tif' are accepted)

Output Arguments:

(none)

class io.LoaderFactory¶

LOADERFACTORY - Factory for instantiating appropriate image loaders based on file format.

Creates concrete loader instances based on the loader information provided by ExtensionRegistryLoad. Each loader implements a standard interface with loadMetadata and loadImages methods.

Method Summary

static create(loaderInfo, options)¶

CREATE - Create an image loader instance for a given file format.

Syntax:

loader = io.LoaderFactory.create(loaderInfo, options)

Main factory method that instantiates the appropriate loader class based on the loaderId field in the loaderInfo structure. Routes to imread, BioFormats, HDF5, NRRD, IMOD, AmiraMesh, OME-Zarr, or other readers.

Input Arguments:

loaderInfo - struct returned by ExtensionRegistryLoad.resolveLoader:
- .loaderId - [char] identifier of the file reader to use
- .mode - [char] dataset mode ('Standard', 'Virtual', 'BigData')
- .reader - [char] reader type ('Default', 'BioFormats')
- .extension - [char] filename extension without leading dot
- .imageFormatType - [char] format type identifier
options - (optional) struct with configuration options:
- .UseBioFormats - [logical] use BioFormats library
- .waitbar - [logical] show progress bar during loading
- .mibPath - [char] path to MIB installation directory
- .virtual - [logical] virtual stacking mode
- .customSections - [logical] load custom sections only
- .bioFormatsMemoizerMemoDir - [char] location of MemoizerMemo cache for BioFormats
- Additional format-specific options passed to the loader constructor

Output Arguments:

loader - loader object instance implementing loadMetadata and loadImages methods

Example 1 - basic usage with Standard mode and imread:

extReg = io.ExtensionRegistryLoad();
loaderInfo = extReg.resolveLoader('image.tif', 'Standard', 'Default');
options.waitbar = true;
options.mibPath = 'c:\mib';
loader = io.LoaderFactory.create(loaderInfo, options);
[imginfo, files] = loader.loadMetadata({'image.tif'}, options);
[img, imginfo] = loader.loadImages(files, imginfo, options);

Example 2 - using BioFormats for complex formats:

loaderInfo = extReg.resolveLoader('image.czi', 'Standard', 'BioFormats');
loader = io.LoaderFactory.create(loaderInfo, options);

static getAvailableLoaders()¶

GETAVAILABLELOADERS - Get a list of all available loader types with descriptions.

Syntax:

loaderList = io.LoaderFactory.getAvailableLoaders()

Returns a struct array containing all supported loader identifiers, their descriptions, and typical file extensions.

Input Arguments:

(none)

Output Arguments:

loaderList - struct array with loader information:
- .loaderId - [char] loader identifier
- .description - [char] human-readable description
- .extensions - cell array of [char] typical file extensions

Example 1 - display all available loaders:

loaderList = io.LoaderFactory.getAvailableLoaders();
fprintf('Available loaders:\n');
for i = 1:numel(loaderList)
    fprintf('  %s: %s\n', loaderList(i).loaderId, loaderList(i).description);
end

class io.SaverFactory¶

SAVERFACTORY - Factory for instantiating appropriate savers based on output format.

Mirrors io.LoaderFactory for the writing side of the pipeline. Maps format strings to concrete io.savers.XxxSaver instances. Maintains separate registries for different data types (image, mask, labels) since the same format (e.g. TIF) may have different defaults and validation rules per category.

Architecture: SaverFactory.create(formatStr) instantiates the appropriate saver class. Registries are built on-demand and cached. Format strings exactly match those used in core.MibDataset.save() and models.MibModel.save() dropdown menus.

High-level usage (recommended):

obj.mibModel.saveImage('image',  filename, BatchOptIn);
obj.mibModel.saveImage('mask',   filename, BatchOptIn);
obj.mibModel.saveImage('labels', filename, BatchOptIn);

Direct factory use (advanced/scripted):

saver = io.SaverFactory.create('TIF format uncompressed (*.tif)');
fnOut = saver.save(data, metadata, '/tmp/out.tif', options);

GUI context (pass ParentFigure/mibPath so progress dialogs attach to MIB):

ctorOpts.ParentFigure = obj.mibModel.mibGUI;
ctorOpts.mibPath = obj.mibModel.mibPath;
saver = io.SaverFactory.create('Amira Mesh binary (*.am)', ctorOpts);

List all formats for a given layer type:

imageFormats = io.SaverFactory.getFormats('image');
maskFormats = io.SaverFactory.getFormats('mask');
labelFormats = io.SaverFactory.getFormats('labels');

Minimal scripted load + save (no metadata required):

imageFormats = io.SaverFactory.getFormats('image');
labelFormats = io.SaverFactory.getFormats('labels');
img = io.loadImagesWrapper('C:\data\image.tif');
saver = io.SaverFactory.create('TIF format uncompressed (*.tif)');
fnOut = saver.save(img, [], 'C:\data\image_out.tif', struct());

See also: io.LoaderFactory, io.savers.BaseSaver, core.MibDataset.save, models.MibModel.save

Method Summary

static create(formatStr, options)¶

CREATE - Instantiate the saver that handles the requested output format.

Syntax:

saver = io.SaverFactory.create(formatStr, options)

Instantiates the appropriate BaseSaver subclass based on the format string. Format strings must exactly match those from getFormats().

Input Arguments:

formatStr - [char] format descriptor as it appears in Format dropdown:
- 'TIF format uncompressed (*.tif)'
- 'Amira mesh binary (*.am)'
- 'Matlab format (*.model)'
- (see getFormats() for the complete list)
options - (optional) struct passed to saver constructor via BaseSaver.initBaseProps(). Important fields when calling from GUI context:
- .ParentFigure - [handle] to main MIB window; enables uiprogressdlg dialogs attached to the GUI (typically obj.mibGUI from MibModel). Leave empty for standalone use.
- .mibPath - [char] MIB installation directory; used for resource and icon lookup by dialogs.
- All other options are typically passed at save() time.

Output Arguments:

saver - concrete BaseSaver subclass instance

Throws:

io:SaverFactory:UnknownFormat - if formatStr is not registered

Example 1 - standalone scripted use:

saver = io.SaverFactory.create('TIF format uncompressed (*.tif)');
opts.Format = 'TIF format uncompressed (*.tif)';
opts.Saving3DPolicy = '3D stack';
opts.showWaitbar = false;
opts.silent = true;
opts.Compression = 'none';
opts.overwrite = true;
meta.filename = 'source.tif';
meta.colorType = 'grayscale';
meta.lutColors = [1 1 1];
meta.dataClass = 'uint8';
meta.maxInt = 255;
meta.sliceName = {};
meta.pixSize = struct('x',0.1,'y',0.1,'z',0.5,'units','um','t',1,'tunits','s');
data = uint8(rand(64,64,10,1,1)*255);
fnOut = saver.save(data, meta, '/tmp/out.tif', opts);

Example 2 - GUI context with progress dialogs:

ctorOpts.ParentFigure = obj.mibModel.mibGUI;
ctorOpts.mibPath = obj.mibModel.mibPath;
saver = io.SaverFactory.create('Amira Mesh binary (*.am)', ctorOpts);
opts.Format = 'Amira Mesh binary (*.am)';
opts.Saving3DPolicy = '3D stack';
opts.showWaitbar = true;
opts.silent = true;
opts.overwrite = true;
opts.ParentFigure = obj.mibModel.mibGUI;
opts.mibPath = obj.mibModel.mibPath;
opts.pixSize = struct('x',0.065,'y',0.065,'z',0.2,'units','um','t',1,'tunits','s');
meta.filename = 'source.tif';
meta.colorType = 'grayscale';
meta.lutColors = [1 0 0];
meta.dataClass = 'uint8';
meta.maxInt = 255;
meta.sliceName = {};
data = uint8(rand(128,128,20,1,1)*255);
fnOut = saver.save(data, meta, '/tmp/stack.am', opts);

Example 3 - save segmentation model in native MIB format:

saver = io.SaverFactory.create('Matlab format (*.model)');
opts.Format = 'Matlab format (*.model)';
opts.showWaitbar = false;
opts.silent = true;
opts.overwrite = true;
meta.filename = 'image.tif';
meta.materialNames = {'Nucleus'; 'Mitochondria'};
meta.materialColors = [0 0 1; 0 1 0];
meta.labelsVariable = 'mibModel';
meta.dataClass = 'uint8';
meta.pixSize = struct('x',0.1,'y',0.1,'z',0.5,'units','um','t',1,'tunits','s');
labels = uint8(rand(64,64,10,1,1)*2);
fnOut = saver.save(labels, meta, '/tmp/Labels_image.model', opts);

static getDefaultFormat(layerType, filenameOrExt)¶

GETDEFAULTFORMAT - Return the default format string for a layer type.

Syntax:

defaultFormat = io.SaverFactory.getDefaultFormat(layerType, filenameOrExt)

Returns a default format string, optionally guided by a filename or file extension. Used to initialize BatchOpt.Format{1} in MibModel.save().

Input Arguments:

layerType - [char] layer type:
- 'image' - default: 'Amira mesh binary (*.am)'
- 'mask' - default: 'Matlab format (*.mask)'
- 'labels' - default: 'Matlab format (*.model)'
- 'everything' - same as 'all', falls back to TIF
filenameOrExt - (optional) [char] full filename (e.g. 'out.tif') or bare extension (e.g. 'tif'). When supplied, function first tries to resolve format from extension; if unknown, falls back to layer-type default.

Output Arguments:

defaultFormat - [char] default format string matching getFormats() output

Example 1 - get default format by layer type and extension:

def = io.SaverFactory.getDefaultFormat('image');
% def == 'Amira mesh binary (*.am)'
def = io.SaverFactory.getDefaultFormat('image', 'tif');
% def == 'TIF format uncompressed (*.tif)'
def = io.SaverFactory.getDefaultFormat('image', 'result.png');
% def == 'Portable Network Graphics (*.png)'
def = io.SaverFactory.getDefaultFormat('labels');
% def == 'Matlab format (*.model)'

static getFormats(layerType)¶

GETFORMATS - Return available format strings for a given layer type.

Syntax:

formats = io.SaverFactory.getFormats(layerType)

Returns a sorted cell array of format strings suitable for populating Format dropdowns in MibModel.save() and MibDataset.save().

Input Arguments:

layerType - (optional) [char], default: 'all'
- 'image' - formats for pixel-data saving
- 'mask' - formats for binary mask saving
- 'labels' - formats for multi-material segmentation
- 'all' or omitted - returns all registered formats

Output Arguments:

formats - cell array of [char] sorted format strings

Example 1 - get available formats by layer type:

imageFormats = io.SaverFactory.getFormats('image');
maskFormats = io.SaverFactory.getFormats('mask');
labelFormats = io.SaverFactory.getFormats('labels');

io.loadImagesWrapper(filename, options)¶

LOADIMAGESWRAPPER - Load a single image file using MIB3 LoaderFactory; returns [H, W, Z, C, T].

Syntax:

function img = loadImagesWrapper(filename, options)

Input Arguments:

filename - [char] full path to the image file
options - (optional) struct with loading options:
- .mibBioformatsCheck - (logical) use BioFormats reader; default false
- .BioFormatsIndices - (numeric) BioFormats series index; default 1
- .verbose - (logical) show timing info; default false

Output Arguments:

img - image array [height, width, depth, color, time]

Usage:

Example 1 - Load a standard TIF image (returns [H, W, 1, C, 1] for a single slice)

img = io.loadImagesWrapper('C:\data\image.tif');

Example 2 - Load an Amira mesh file without verbose output

img = io.loadImagesWrapper('C:\data\stack.am', struct('verbose', false));

Example 3 - Load a PNG file and check output dimensions

img = io.loadImagesWrapper('C:\data\patch.png');
fprintf('Size: %d x %d x %d x %d x %d\n', size(img,1), size(img,2), size(img,3), size(img,4), size(img,5));

Example 4 - Load with BioFormats reader, selecting series index 2

opts.mibBioformatsCheck = true;
opts.BioFormatsIndices = 2;
img = io.loadImagesWrapper('C:\data\multiSeries.czi', opts);

Example 5 - Use as a ReadFcn in an imageDatastore

opts.verbose = false;
opts.mibBioformatsCheck = false;
opts.BioFormatsIndices = 1;
ds = imageDatastore('C:\data\Images', 'FileExtensions', '.tif', ...
    'ReadFcn', @(fn) io.loadImagesWrapper(fn, opts));
img = read(ds);   % returns [H, W, Z, C, T]

Example 6 - Discover supported extensions, then load a file

extReg = io.ExtensionRegistryLoad();
standardExts   = extReg.getAllowedExtensions('Standard', 'Default');
bioformatsExts = extReg.getAllowedExtensions('Standard', 'BioFormats');
img = io.loadImagesWrapper('C:\data\image.tif');

io.mibImage2mrc(O, Options)¶

MIBIMAGE2MRC - Export volume data in MRC format.

Syntax:

result = io.mibImage2mrc(O, Options)

Exports 3D volumetric image data to MRC format (Electron Microscopy Data Bank standard). Requires the MatTomo function set, available in mib/external/MatTomo.

Input Arguments:

O - [H, W, D] or [H, W, 1, D] numeric array, volumetric dataset. Grayscale format required (MRC does not support multichannel images).
Options - struct with configuration:
- .volumeFilename - [char] output filename; use '.mrc' extension
- .pixSize - struct with voxel size information:
  - .x - [numeric] physical width of voxels
  - .y - [numeric] physical height of voxels
  - .z - [numeric] physical thickness of voxels
  - .units - [char] physical units ('m', 'cm', 'mm', 'um', 'nm')
- .showWaitbar - (optional) [logical] default: true show progress bar during save
- .ParentFigure - (optional) [handle] main MIB window for uiprogressdlg attachment (recommended for GUI use). When absent, falls back to legacy waitbar.

Output Arguments:

result - [logical] 1 on success, 0 on failure

Example 1 - standalone scripted use (no GUI parent):

mrcOpts.volumeFilename = '/output/volume.mrc';
mrcOpts.pixSize = struct('x',0.065,'y',0.065,'z',0.2,'units','um');
mrcOpts.showWaitbar = false;
io.mibImage2mrc(imageData_hwd, mrcOpts);

Example 2 - GUI use with progress dialog attached to MIB window:

mrcOpts.volumeFilename = '/output/volume.mrc';
mrcOpts.pixSize = struct('x',0.065,'y',0.065,'z',0.2,'units','um');
mrcOpts.showWaitbar = true;
mrcOpts.ParentFigure = obj.mibModel.mibGUI;
io.mibImage2mrc(imageData_hwd, mrcOpts);

Remote stores¶

RemoteStore holds the URL and object-store plumbing shared by the remote OME-Zarr paths: it recognises the common S3 URL spellings, rewrites s3:// into an anonymous HTTPS URL, enumerates the children of a prefix through the S3 ListObjectsV2 API, and fetches small metadata files. It knows nothing about Zarr itself.

The amazonaws.com patterns in parse exist only to tell Amazon’s two spellings apart - virtual-host style puts the bucket in the host, path style puts it in the path. They are not what makes a store listable: any other host is read as path style with the endpoint taken from the URL, which is what reaches MinIO and Ceph deployments addressed as https://HOST/BUCKET/KEY. Such a host is flagged parse(url).flavour == 's3compatible' to mark the guess as unconfirmed; one that does not answer the API simply returns an empty listing, and callers fall back to an explicit user-supplied path.

class io.RemoteStore¶

REMOTESTORE - Helpers for object-store URLs (HTTP / HTTPS / Amazon S3).

Pure URL and listing plumbing with no knowledge of Zarr or of any other format: it recognises the common S3 URL spellings, rewrites s3:// into an anonymous HTTPS URL, lists the children of a “folder” through the S3 ListObjectsV2 REST API, and fetches small metadata files. Consumers are io.ExtensionRegistryLoad (probing a remote store for its zarr version), the OME-Zarr setup loaders, and controllers.SelectFromUrl.

Why not s3fs / AWS credentials. Public buckets answer ordinary anonymous HTTPS GET requests, and ListObjectsV2 is likewise anonymous on a public bucket. Normalising s3://bucket/key to https://bucket.s3.amazonaws.com/key therefore removes any need for credentials, region configuration or an extra Python dependency. Private and requester-pays buckets are out of scope.

Listing needs the S3 REST API, not Amazon. Any endpoint that answers ListObjectsV2 can be browsed - MinIO and Ceph deployments do, which is how https://HOST/BUCKET/KEY works. A URL that matches none of the AWS spellings is read as path style and tried; a host that turns out not to speak the API simply returns no listing, and listChildren reports that as empty rather than as an error. Callers must degrade to asking the user for an explicit path rather than failing.

Example - walk down to the image group of a Janelia OpenOrganelle store:

root = 'https://janelia-cosem-datasets.s3.amazonaws.com/jrc_mus-liver-zon-1/jrc_mus-liver-zon-1.zarr';
[childUrls, childNames] = io.RemoteStore.listChildren(root);
% childNames -> {'recon-1'}
emGroup = io.RemoteStore.join(root, 'recon-1/em/fibsem-uint8');
attrs   = io.RemoteStore.readJson([emGroup '/.zattrs']);

Method Summary

static clearCache()¶

CLEARCACHE - Drop the memoised listings.

Listings are cached for the whole MATLAB session, so a store that changed on the server keeps showing its old contents until this is called. Nothing else in MIB depends on the cache being warm.

Syntax:

io.RemoteStore.clearCache()

static exists(url)¶

EXISTS - True when a remote object can be fetched.

Tries HEAD first and falls back to a one-byte ranged GET, because some object stores and CDNs reject HEAD. Used to probe for marker files such as zarr.json or .zgroup without downloading anything meaningful.

Syntax:

tf = io.RemoteStore.exists(url)

Input Arguments:

url - [char|string] full URL of the object

Output Arguments:

tf - [logical] true when the object responded; false on any error, timeout or non-success status

static isRemote(path)¶

ISREMOTE - True when the path is an http, https or s3 URL.

Syntax:

tf = io.RemoteStore.isRemote(path)

Input Arguments:

path - [char|string] path or URL to test

Output Arguments:

tf - [logical] true for a remote URL, false for a local path and for any empty or non-text input

static join(baseUrl, relativePath)¶

JOIN - Append a relative path to a URL with forward slashes.

fullfile must never be used on a URL: on Windows it inserts a backslash and turns https://host/group into https://host\group. An already absolute relativePath is returned normalised, so a caller can accept either form from the user.

Syntax:

url = io.RemoteStore.join(baseUrl, relativePath)

Input Arguments:

baseUrl - [char|string] base URL
relativePath - [char|string] path relative to the base, or an absolute URL

Output Arguments:

url - [char] joined URL, without a trailing slash

static listChildren(url, options)¶

LISTCHILDREN - Enumerate the immediate children of a store “folder”.

One ListObjectsV2 request per call (plus one per extra page when the result is truncated), issued anonymously with delimiter='/' so only the immediate level is returned. Results are memoised for the session, so re-expanding a node in a browser costs nothing.

Returns empty for a host that cannot be listed and for any network or parse failure - it never throws, because the caller is expected to degrade to manual path entry rather than abort.

Syntax:

[childUrls, childNames, fileNames] = io.RemoteStore.listChildren(url)
[childUrls, childNames, fileNames] = io.RemoteStore.listChildren(url, options)

Input Arguments:

url - [char|string] URL of the group / prefix to list
options - (optional) [struct] with optional fields:
- .timeout - [numeric] request timeout in seconds (default: 30)
- .maxKeys - [numeric] keys per request (default: 1000)
- .useCache - [logical] reuse the session cache (default: true)
- .fetchFcn - [function_handle] listing fetcher, used by the tests to run offline; called as fetchFcn(listEndpoint, prefix, continuationToken, timeout, maxKeys) and expected to return the response body as [char]

Output Arguments:

childUrls - [1xN cell] full URLs of the child groups
childNames - [1xN cell] child group names, no trailing slash
fileNames - [1xM cell] names of the files directly in this node, e.g. {'.zattrs', '.zgroup'}

static normalise(path)¶

NORMALISE - Canonical HTTPS form of a store URL.

Rewrites s3://bucket/key into https://bucket.s3.amazonaws.com/key so the rest of MIB only ever sees an anonymous HTTPS URL, and strips any trailing slash so joins and prefix comparisons are stable. Local paths are returned unchanged.

Syntax:

url = io.RemoteStore.normalise(path)

Input Arguments:

path - [char|string] path or URL

Output Arguments:

url - [char] normalised URL, or the input path unchanged

static parse(path)¶

PARSE - Decompose a store URL into bucket, key and listing endpoint.

Recognised spellings, tried in this order (regional forms first, as they are the more specific match):

s3://BUCKET/KEY
https://BUCKET.s3.<region>.amazonaws.com/KEY     (also s3-<region>)
https://BUCKET.s3.amazonaws.com/KEY
https://s3.<region>.amazonaws.com/BUCKET/KEY     (also s3-<region>)
https://s3.amazonaws.com/BUCKET/KEY
https://storage.googleapis.com/BUCKET/KEY        (parsed, not listable)
https://ANYHOST/BUCKET/KEY                       (assumed path style)

The amazonaws.com patterns are here only to tell AWS’s two spellings apart - the bucket sits in the host in one and in the path in the other. They are not what makes a store listable: any other host falls through to the last rule and is read as path style, with the endpoint taken from the URL itself. That is what reaches a MinIO or Ceph deployment addressed as https://HOST/BUCKET/KEY.

Bucket names may contain dots, so the host patterns use a lazy match up to the .s3 label rather than “one label”.

Google Cloud Storage exposes only the marker-based v1 XML API, which this class does not implement, so it is parsed but reported as not listable, ahead of the generic rule.

Syntax:

info = io.RemoteStore.parse(path)

Input Arguments:

path - [char|string] path or URL

Output Arguments:

info - [struct] with fields:
- .url - [char] normalised URL (see normalise)
- .bucket - [char] bucket name, '' when not an object store
- .key - [char] object key of the node, without a trailing slash
- .region - [char] region when the URL carried one, else ''
- .listEndpoint - [char] bucket root to issue ListObjectsV2 against, including the trailing slash; '' when not listable
- .listable - [logical] whether listChildren may be tried
- .flavour - [char] 's3' (an AWS spelling, known to list), 's3compatible' (assumed path style, listing unconfirmed), 'gcs', 'plain' or 'local'

static readJson(url, timeoutSeconds)¶

READJSON - Fetch and decode a small JSON metadata file.

Fetched as text and decoded here rather than asking webread for JSON: object stores commonly serve .zattrs / .zgroup / zarr.json as binary/octet-stream, and a json-then-text retry would make every miss cost two requests. The group walk issues many misses, so one request per call matters.

Returns [] rather than throwing when the file is absent, which is the normal outcome when probing for a marker.

Syntax:

data = io.RemoteStore.readJson(url)
data = io.RemoteStore.readJson(url, timeoutSeconds)

Input Arguments:

url - [char|string] full URL of the JSON file
timeoutSeconds - (optional) [numeric] request timeout (default: 15)

Output Arguments:

data - [struct] decoded JSON, or [] when unavailable

static relativePath(rootUrl, url)¶

RELATIVEPATH - Path of a URL relative to a store root.

Inverse of join. When the URL does not sit under the root it is returned unchanged, so the caller can pass it on as an absolute location instead.

Syntax:

relative = io.RemoteStore.relativePath(rootUrl, url)

Input Arguments:

rootUrl - [char|string] store root URL
url - [char|string] URL at or below the root

Output Arguments:

relative - [char] path relative to the root, '' when the two are the same node

Chunk cache¶

io.zarr.ChunkCache is an in-memory LRU of decoded Zarr chunks, shared by io.loaders.Zarr2VirtualLoader and io.loaders.Zarr3VirtualLoader. It works in Zarr’s declared C-order index space and knows nothing about OME-Zarr, pyramids, MIB axis conventions or which engine produced the chunks, which is what lets one implementation serve both formats and both backends.

It exists because a chunk is the smallest unit a store will hand over, and published volumes are routinely chunked for 3D block access - [64, 128, 128] is typical, so a chunk carries 64 slices. Without the cache, showing one plane fetches all 64 and discards 63, and the next slice re-fetches the identical chunks. Missing chunks of one request are fetched in a single call over their bounding box, never one call per chunk, because the engines fetch the chunks of one request concurrently.

class io.zarr.ChunkCache¶

CHUNKCACHE - In-memory LRU cache of decoded Zarr chunks.

Sits between the virtual loaders and whichever engine actually fetches pixels, so a region already held in memory is served without touching the store. It works in Zarr’s declared C-order index space and knows nothing about OME-Zarr, pyramids or MIB axis conventions, which is what lets the same instance serve both io.loaders.Zarr2VirtualLoader (zarr-python) and io.loaders.Zarr3VirtualLoader (native zarrMex).

Why this exists. A chunk is the smallest unit a store will give you, and published OME-Zarr is routinely chunked for 3D block access rather than for browsing one plane at a time. The Janelia C. elegans volume uses [64, 128, 128]: showing a single 1137x610 screenful at full resolution touches 60 chunks, and each of those carries 64 z-slices. Without a cache that is ~50 MB fetched to display 0.7 MB - 1.1% useful - and stepping one slice re-fetches all of it, because the store has no idea it just sent you the neighbouring slice. Measured on that volume: reading 64 slices costs the same 2.6 s as reading 1, so every z-step inside a chunk block was being paid for 64 times over.

What is cached. Whole chunks, decoded, keyed by array identity plus chunk index. The identity comes from storeKey() - the level path plus the version of its metadata file - so a store rewritten at the same path starts cold instead of being assembled from the replaced store’s chunks. Caching whole chunks rather than whole requests is what makes panning cheap: a viewport shifted by less than a chunk re-uses everything but the new edge. Chunks at the far edge of an array are stored at their true (clipped) size, not padded.

How misses are fetched. All missing chunks of one request are fetched in a single call covering their bounding box, never one call per chunk. The engines fetch the chunks of one request concurrently, so one call for N chunks is far quicker than N calls for one chunk each - on the volume above, ~6x. The bounding box may pull in a few chunks that were already cached; they are cheap next to a second round trip, and they get refreshed rather than wasted.

Eviction is least-recently-used against a byte budget (setBudgetMB(), default 512 MB). The cache is process-wide and survives closing a dataset, so re-opening the same store is warm.

Example - wrap an engine read:

raw = io.zarr.ChunkCache.read(fullPath, bbox, meta.chunkShape, meta.shape, ...
    @(alignedBbox) io.zarr.PyBackend.readArray(pyArray, alignedBbox, meta));

Method Summary

static clear()¶

CLEAR - Drop every cached chunk and reset the counters.

Syntax:

io.zarr.ChunkCache.clear()

static isEnabled()¶

ISENABLED - True when chunks are being cached.

Syntax:

tf = io.zarr.ChunkCache.isEnabled()

static read(cacheKey, bbox, chunkShape, arrayShape, readFcn)¶

READ - Serve a region from cached chunks, fetching only what is missing.

Syntax:

raw = io.zarr.ChunkCache.read(cacheKey, bbox, chunkShape, arrayShape, readFcn)

Input Arguments:

cacheKey - [char|string] identity of the array; use storeKey() of the pyramid level path, so a store rewritten at the same path is not served from the old one’s chunks
bbox - [nDims x 2] requested region in Zarr C-order, as [start_1based, end_exclusive] - the same form the loaders already build for the engines
chunkShape - [1 x nDims] chunk size of the array
arrayShape - [1 x nDims] full array shape
readFcn - [function_handle] readFcn(alignedBbox) fetching a chunk-aligned region from the engine, in the same bbox form

Output Arguments:

raw - [numeric] the requested region, in Zarr C-order, exactly as readFcn would have returned it for bbox

static setBudgetMB(budgetMB)¶

SETBUDGETMB - Set the memory budget, evicting immediately if needed.

A budget of 0 disables caching, which is the escape hatch for a machine short of RAM and the way to measure the uncached cost.

Syntax:

io.zarr.ChunkCache.setBudgetMB(512)

Input Arguments:

budgetMB - [numeric] budget in megabytes

static stats()¶

STATS - Current cache occupancy and hit counters.

Syntax:

info = io.zarr.ChunkCache.stats()

Output Arguments:

info - [struct] with .chunkCount, .bytes, .budgetBytes, .hits, .misses, .enabled

static storeKey(arrayPath)¶

STOREKEY - Cache identity of one zarr array: its path plus a version of the store.

Syntax:

key = io.zarr.ChunkCache.storeKey(arrayPath)

The cache is process-wide and outlives the datasets that fill it, so a key made of the path alone serves the chunks of a store that has since been replaced at that path - e.g. exporting or converting to BigData over a store opened earlier in the session. When the chunk layout differs the old chunks are assembled into the wrong places and the level shows mostly empty or scrambled tiles.

A local array therefore also carries the modification time and size of its metadata file (zarr.json for v3, .zarray for v2), which every writer rewrites when it creates the array. Only one dir call, so callers compute it once per opened level, not per read. A remote array (URL) keeps its bare path: a published store is not rewritten under a reader, and probing it would cost a round trip.

Input Arguments:

arrayPath - [char|string] full path or URL of the level array

Output Arguments:

key - [string] arrayPath for a remote array or when no metadata file is found, otherwise "<path>@<datenum>:<bytes>"

io — Image I/O¶

Factory functions¶

Remote stores¶

Chunk cache¶

Loaders and savers¶

Format sub-packages¶