cellSAM.cellsam_pipeline.cellsam_pipeline#

cellSAM.cellsam_pipeline.cellsam_pipeline(img, chunks=256, model_path=None, bbox_threshold=0.4, low_contrast_enhancement=False, swap_channels=False, use_wsi=True, gauge_cell_size=False, block_size=400, overlap=56, iou_depth=56, iou_threshold=0.5)#

Run the cellsam inference pipeline on img.

Cellsam is capable of segmenting a variety of cells (bacteria, eukaryotic, etc.) spanning all forms of microscopy (brightfield, phase, autofluorescence, electron microscopy) and staining (H&E, PAS, etc.) / multiplexed (codex, mibi, etc.) modalities.

Parameters:
imgarray_like with shape (W, H) or (W, H, C), where C is 1 or 3

The image to be segmented. For multiple-channel images, img should have the following format:

  • Stained images (e.g H&E): (W, H, C) where C == 3 representing color channels in RGB format.

  • Multiplexed images: (W, H, C) where C == 3 and the channel ordering is: (blank, nuclear, membrane). The membrane channel is optional, in which case a nuclear segmentation is returned.

chunksint

TODO: should this be an option?

model_pathstr or pathlib.Path, optional

Path to the model weights. If None (the default), the latest released cellsam generalist model is used.

Note

Downloading the model requires internet access

bbox_thresholdfloat in range [0, 1], default=0.4

Threshold for the outputs of Cellfinder, only cells with a confidence higher than the threshold will be included. This is the main parameter to control precision/recall for CellSAM. For very out of distribution images use a value lower than 0.4 and vice versa.

low_contrast_enhancementbool, default=False

Whether to enhance low contrast images, like Livecell images as a preprocessing step to improve downstream segmentation.

swap_channelsbool, default=False

TODO: this should be removed with loading from file

use_wsibool, default=True

Whether to use tiling to support large images, default is True. Generally, tiling is not required when there are fewer than ~3000 cells in an image.

gauge_cell_sizebool, default=False

Wheter to perform one iteration of segmentation initially, and use the results to estimate the sizes of cells and then do another round of segmentation using tiling parameters with these results.

block_sizeint

Size of the tiles when use_wsi is True. In practice, should be in the range [256, 2048], with smaller tile sizes preferred for dense (i.e. many cells/FOV) images.

overlapint

Tile overlap region in which label merges are considered. Must be smaller than block_size. For reliable tiling, value should be large enough to encompass iou_threshold of the extent of a typical object.

iou_depthint

TODO: Detail effects of this parameter: is this/should this be distinct from overlap?

filter_below_minbool

TODO: Detail this parameter - is it necessary?

Returns:
segmentation_mask2D numpy.ndarray of dtype numpy.uint32

A numpy.ndarray representing the segmentation mask for img. The array is 2D with the same dimensions as img, with integer labels representing pixels corresponding to cell instances. Background is denoted by 0.

Examples

Using CellSAM to segment a slice from the cells3d dataset.

>>> import numpy as np
>>> import skimage
>>> data = skimage.data.cells3d()
>>> data.shape
(60, 2, 256, 256)

From the cells3d docstring, data is a 3D multiplexed image with dimensions (Z, C, X, Y) where the ordering of the channel dimension C is (membrane, nuclear). Start by extracting a 2D slice from the 3D volume. The middle slice is chosen arbitrarily:

>>> img = data[30, ...]

For multiplexed images, CellSAM expects the channel ordering to be (blank, nuclear, membrane):

>>> seg = np.zeros((*img.shape[1:], 3), dtype=img.dtype)
>>> seg[..., 1] = img[1, ...]  # nuclear channel
>>> seg[..., 2] = img[0, ...]  # membrane channel

Segment the image with cellsam_pipeline. Since this is a small image, we’ll set use_wsi=False. We’ll also forgo any pre/post-processing:

>>> mask = cellsam_pipeline(seg, use_wsi=False)
On this page

This Page