Paper: https://www.frontiersin.org/articles/10.3389/fcomp.2022.777728/full
Extensive documentation and tutorials provided here: https://imagej.net/plugins/labkit/.
I2K talk with a quick demo: https://youtu.be/UZjZtmm7adU
- Get familiar with Labkit interface
- Use slice-by-slice labeling to generate manual labels
-
Basic understanding of Fiji
-
Segmentation types
Semantic vs instance segmentation
Download example 3D dataset:
https://bbbc.broadinstitute.org/BBBC032
Install Labkit via update site:
Help > Update...
In the ImageJ Updater click Manage Update sites.
In the pop up menu activate Labkit - (https://sites.imagej.net/Labkit/) update site:
Install dependencies for GPU acceleration (if a good GPU is available at your machine):
Activate clij and clij2 update sites
Click Close.
In the ImageJ then select Apply changes. Restart Fiji.
NOTE: In order to benefit from the performance boost that BigDataViewer (BDV) offers particularly for rapid visualization you should re-save the data as BDV XML/HDF5: https://imagej.net/plugins/bdv/#exporting-datasets-for-the-bigdataviewer
In Fiji:
Plugins > BigDataViewer > Export Current Image as XML/HDF5
Unzip the example dataset. We will have a look at channel 3 of this dataset:
BMP4blastocystC3.tif
Open the example dataset in Fiji by drag & drop.
Load the image volume into Labkit:
Plugins > Labkit > Open Current Image in Labkit
The image gets opened in Labkit:
Adjust the contrast by pressing: auto contrast.
The Labkit interface:
Which consists of the BigDataViewer interface (1). The Labeling classes (2). The labeling tool box (3) as well as the pixel classifier setting (4). We will introduce these now in more detail.
This allows rapid interaction with the 3D volume. The interaction works via the mouse and keyboard. For all the interactions have a look here: https://imagej.net/plugins/bdv/
You see in the field-of-view a slice through the image volume. In the top left corner the green/magenta wire diagram visualizes the entire volume. The gray slices corresponds to the visualized slice in this volume
The most relevant mouse and keyboard interactions for BDV are the:
Mouse wheel: scroll through the volume
Right mouse click & hold and drag: pan the image
Left mouse click & hold and drag: rotate around the mouse pointer
Up & down arrow keys (Ctrl + Shift + Mouse scroll): zoom in and out
Right & left arrow keys: rotate around a selected axis (select axis by pressing x,y or z)
Shift & x, y or z: rotate 90 degrees around a specific axis
TIP: speed up interactions by pressing and holding SHIFT
This box allows you to define and adjust the labeling classes. You can turn on/off the labeling by pressing the “eye” symbol. You can add and remove labels by Add label and Remove all. You can change the color of the class by pressing on the color. To rename a label, click the down arrow box to enter the options and choose Rename.
Pressing the arrow cross activates the interaction mode in BDV so you can navigate the data with the mouse.
You can select the pencil (D) for labeling the images. Change the brush size on the right. You can fill a closed (!) outline using the fill function (F). You can use the eraser (E) to modify the labels. Finally you can delete entire connected components (R).
Draw the labels on the BDV canvas.
NOTE: You can switch between different visualization modes using the slice mode button on the far right side of the labeling toolbox.
With the 3D-box it is in 3D mode:
With the slice-box visible Labkit is in the slice-by-slice mode:
In the slice-by-slice mode the image volume will be locked into an xy visualization. Like a typical stack in Fiji. You can scroll through the slices using the mouse wheel. Or the slider on the right.
The slice-by-slice mode is helpful if you want to generate and curate dense manual labels
Here you can add pixel classifiers by pressing on the Labkit Pixel Classification button:
For creating the dense manual labels you need to first activate the slice-by-slice labelling mode. Press the BDV mode button by pressing on the 3D box:
The slice-by-slice labelling mode will be activated. The button will change to the slice box:
The image volume will be locked into an xy visualization. Like a typical stack in Fiji. You can scroll through the slices using the mouse wheel. Or the slider on the right:
Labkit uses the following data labeling tools:
For drawing on the canvas first select the pencil (D). For drawing dense labels you would like to draw with a larger brush (this is in contrast to what you will do with the labeling for the automatic segmentation later). The brush size can be changed with the slider to the right. For dense labels the best strategy is to first draw the outline of the object.
Then you can fill the remaining hole using the fill function (F).
The objects can be adjusted using the eraser (E). The eraser size is also adjusted with the brush size. You can easily delete entire connected components with the delete blob function (R).
Draw the labels on the BDV canvas. Go through all slices and label all relevant parts. You can verify if all slices have been labeled by activating the 3D interaction mode again:
You can then rotate the image volume (Shift + x or shift + y). A complete dense labelled volume looks like this:
Whereas with missing slices it will look like this:
An important aspect is to understand the difference between instance and semantic segmentation. A semantic segmentation will not differentiate between individual objects within one class. Thus, even if you manually label individual cell separately they will all be labeled with the same label identity. An instance segmentation will differentiate individual objects with a separate label.
By default Labkit is setup to primarily support semantic segmentation, especially in the automatic segmentation mode. You can label individual objects by using one class and then use a different tool to generate label mask afterwards. However, in this case you need to make sure the labeled objects are well separated (could be hard in 3D). Afterwards one could use other tools to generate individual labels for the well separated labeled objects.
The alternative to this is to use per object one class. This is also supported in Labkit but might be a bit awkward to use.
When you are happy with the labels save them by:
Labkit > Labeling > Save Labeling ...
You can also load labelings:
Labkit > Labeling > Open Labeling ...
You can save the labels also as a new image by selecting the correct labeling class in the labeling interface and then:
Labkit > Labeling > Export Selected Label as Bitmap
The image volume exported shows for the label class as binary mask with 1 being the exported label.
In order to use this in Fiji we need to convert the 0 1 binary image to a Fiji binary image with 0 and 255 value. You can do this using the thresholding method:
Image > Adjust > Threshold ...
The thresholding applet opens. Then press Set in the thresholding applet. A new window will pop up that allows you to set manual thresholds. Set a lower and upper threshold level to select the specific class you want to work on further. Then press ok in the Set Threshold Levels.
Then press Apply. A new window will pop up with Convert Stack to Binary. Untick Calculate threshold for each image to take in the manual settings. Click ok to apply the threshold, which results in a binary image that you can use for any further processing.
You can then further process the binary mask in Fiji. For example you could create an instance segmentation.
For an instance segmentation you need to create individual labels per object. If you did not label each individual object with a different class label you can use other tools to generate individually labeled objects from the semantic segmentation: https://imagej.net/plugins/morpholibj#connected-component-labeling
You need to install MorphoLibJ for this: https://imagej.net/plugins/morpholibj#installation
In MorphoLibJ apply then on a binary mask the connected component labeling:
Plugins > MorphoLibJ > Binary Images > Connected Components labeling
This will give each connected component a different grey value.
If your labels should have pretty colors instead of the grey values you can use:
Plugins > MorphoLibJ > Label Images > Labels to RGB
NOTE: make sure you set the background color to your liking
