Use-case:
Segmentation.
Identification of rare events.
Advantages:
Segment sample that are unsegmentable by common methods.
Segment samples without a necessity to tune the parameters of the segmentation method for every sample.
Segment.ai is trained using the NIS.ai > Train Segment.ai window and executed using the NIS.ai > Segment.ai window.
Figure 491.
Figure 492.
Segmentation created by using Threshold, Spot Detection, Auto-detection, Homogeneous Area Detection, created manually by Binary Editor or by any combination of these methods. It is usually best to start by using some common method (Threshold, Spot Detection, etc.) and then manually correct/edit the segmentation to get the desired result. The other method is to create the segmentation manually from scratch.
Guidelines for the binary preparing:
Segment.ai requires a precise drawing of binary objects (see the image examples below).
Also, it is possible to acquire a fluorescence image which can be easily segmented by one of the conventional methods mentioned above.
Binary layers should not have any intersection. If an intersection is found, it is automatically deleted before the training. This control process proceeds from the first binary layer to the last one. Therefore the most important objects should be drawn in the first binary layer which is always preserved.
Different types of objects should be drawn into separate binary layers. E.g. when segmenting neurons, the neuron body should be drawn into a different layer than the dendrites, because they differ significantly from each other.
Background should not be added to the training set because it is generated automatically.
Do not crop smaller chunks of the image as training data - always segment the whole image.
For successful training, AI needs both positive samples (where objects are located) and negative samples (where objects are not located). For example, if edges of the wells are visible on some images, then the training dataset should contain such images, so that the AI can recognize that these parts of image do not contain any objects. If the AI was trained only on images without visible edges of a well, then it might not work on images which contain it.
If the ground truth segmentation is imprecise, then the trained AI will be imprecise as well. That being said, the AI is robust and a few incorrect cells will not confuse it but try to be precise.
Keep the same rule for the entire segmentation. Be consistent.
Draw lines precisely over the cell boundaries (see example below). Draw the bright border inside the rounded object and do not draw the black border of the rounded object. Keep the same rule for entire drawing and do not mix drawings inside/outside border of the objects. Stay consistent.
Segment all objects of the image – if you miss some objects it will lead to errors in training (see images below). If particular frame does not contain any objects then this is not a problem but if it does, you need to segment all of them.
The number of required images depends on the difficulty of the sample. When the image seems easy, you may get away with segmenting only 2-3 images (of reasonable size – e.g. 1000 x 1000 pixels; see the example image “Image suitable for the Segment.ai” below). When judging whether the scene is difficult or not, use intuition. If it looks hard, it probably is and it will require more training images thus requiring more iterations to train.
The other option is to always segment only few images (e.g. 2-3 images in case of simple scene), train the AI and if the result is not sufficient then add some more images of scenes that are difficult for the AI to segment. Repeat this process until the results are sufficient.
Examples of precise binary object's drawing in DIC
Figure 493. The cells without drawn border. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]
Figure 494. Draw the bright border inside the rounded cell. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]
Figure 495. Do not draw the black border of the rounded cell. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]
Figure 496. Keep the same rule for entire drawing and do not mix drawings inside/outside border of the objects; stay consistent. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]
Missing cell segmentation leads to errors
Figure 497. Correct segmentation. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]
Figure 498. Incorrect segmentation. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]
Figure 499. Correct segmentation. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]
Figure 500. Incorrect segmentation. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]
Figure 501. Correct segmentation including border objects.
Figure 502. Incorrect segmentation.
Segment all objects of the image. If you miss some objects, it will lead to errors in training (areas marked by the red arrow in the images above). Even the object on the border should be included.
Figure 503. Image suitable for the Segment.ai. Only 2-3 images are needed for the segmentation.
To properly train the Segment.ai, perform the following steps:
Prepare the binary layers (e.g. nuclei) on the channel (e.g. DIC).
Open the sample image having the binary layers.
Select Train Segment.ai in NIS.ai menu. In the Train Segment.ai dialog window you should:
In the Training data section select file(s) from your hard drive used for training by clicking on or .
Select your Source channels (for example the DIC channel).
Select the Ground truth binaries (e.g. nuclei).
In Options:
The number of neural training Iterations (use 1000 by default).
Input the name (Name in Explorer) for your AI used in the Explorer.ai (e.g. nuclei on DIC).
Specify the Output (Save) where the AI will be stored.
Check Save graph screenshot if you want to create a screenshot of the loss chart when the training is done.
Click to begin the training. Or you can click Add to Queue to run this training using the NIS.ai > Explorer.ai later.
Wait for the training progress to automatically end when all iterations are done or wait until the curve seems to be constant and then click (see the image “Loss chart overview” below).
If the loss curve is not decreasing, try the following options:
Retrain the AI with a training dataset having a higher number of binary objects.
Add multi-points to increase the variability of the training dataset.
Add more iterations in Continue training on.
Figure 504. Train Segment.ai execution dialog and settings.
Options
If you want to teach the existing AI further, check this function and to a *.sai AI file. See Continue Training (Continue Training) for more information.
Figure 505. Loss chart overview.
Loss curve (red) should be decreasing over time, faster in the beginning, slower in the end as it approaches zero. There may be a few bumps along the way which is correct. Training time depends on the difficulty of the sample and the use case. To improve the training results, it is best to use multi-point image with higher variability, however more training images require more training time.
To test the trained AI:
Open the test image without a binary layer.
Select Segment.ai in NIS.ai menu and run its execution dialog.
Select the Source channels (e.g. DIC).
Select the Trained AI with the network trained for the particular function or pick it from the Explorer (e.g. …\nuclei_on_DIC.sai).
Select Preview and correct the binary with the binary tool if necessary, located in (see image below). If your trained network creates more than one binary, Multi-Smooth feature is shown instead. This feature is used to smooth out multiple binaries at once.
If you are satisfied with the result, click .
When the process is finished the binary appears on the image (see the resulting image below).
Figure 506. Segment.ai execution dialog containing an extension to modify the binary.
Figure 507. Segmentation of Nuclei from DIC by Segment.ai. Nuclei were detected from trained AI which was trained on DIC channel and the ground truth binary layer defined by a manual drawing. [Courtesy of Nikon Imaging Center, RIES at Hokkaido University]