All Frames

All frames in the loop are reduced into one frame (all to 1).

Rolling

For each frame in the loop reduces surrounding N frames into the current one (all to all).

Piecewise

Reduces every consecutive N frames into one frame (all to all/N).

Loop

Loop to be reduced.

Frames

Number of frames to reduce. Disabled for All Frames.

All Frames

All frames in the loop are reduced into one frame (all to 1).

Rolling

For each frame in the loop reduces surrounding N frames into the current one (all to all).

Piecewise

Reduces every consecutive N frames into one frame (all to all/N).

Loop

Loop to be reduced.

Frames

Number of frames to reduce. Disabled for All Frames.

Loop

Loop where the frame will be selected.

Select Frame

Specifies the selection from the current loop.

Start

Starting frame.

Count

Number of frames.

Step

Step size.

All Frames

All frames in the loop are reduced into one frame (all to 1).

Rolling

For each frame in the loop reduces surrounding N frames into the current one (all to all).

Piecewise

Reduces every consecutive N frames into one frame (all to all/N).

Loop

Loop to be reduced.

Frames

Number of frames to reduce. Feature is disabled for All Frames.

All Frames

All frames in the loop are reduced into one frame (all to 1).

Rolling

For each frame in the loop reduces surrounding N frames into the current one (all to all).

Piecewise

Reduces every consecutive N frames into one frame (all to all/N).

Loop

Loop to be reduced.

Frames

Number of frames to reduce. Feature is disabled for All Frames.

Background

Value added to the background (everything except the binary).

Object value

Object feature which is written to the object linked through Entity and ObjectId .

Background

Value added to the background (everything except the binary).

Object value

Object feature which is written to the object linked through Entity and Object3DId .

Class count

Number of binaries created.

Source Rect

Defines the size (in microns if calibrated) of the center portion of the frame that is cropped and rendered.

Maximum Size

Maximum Size (in pixels) is a limit to which the rendered image is stretched down if being bigger.

Rendered Size

Size of the rendered image.

Format & Quality

Specifies the Image file format (jpg or png) and image quality in [%].

Auto-contrast

Binary Opacity

Upgrading pip

Be careful and call .\python.exe from the NIS-Elements Python folder to be sure to upgrade the local NIS-Elements pip not the system pip.

CD \Program Files\NIS Elements\Python
.\python.exe -m pip install --upgrade pip

Troubleshooting pip

When pip.bat is not working:

Solution 1: instead use

python.bat -m pip install <package>

Solution 2: bootstrap pip

1. Call the bootstrappip.bat from NIS-Elements\Python folder.

2. Check pip.bat from C:\Program Files\NIS Elements\ and if successful end here. Otherwise continue with the steps below.

3. Go to NIS Elements Python folder C:\Program Files\NIS Elements\Python

4. Delete pip*.exe (pip.exe, pip3.exe, pip3.12.exe) from Scripts folder if present.

5. Delete pip and pip-XX.Y.Z.dist-info (e.g. pip-23.2.1.dist-info) from Lib\site-packages if present.

6. From C:\Program Files\NIS Elements\Python call

   .\python.exe -m ensurepip --default-pip

7. Check pip.bat from C:\Program Files\NIS Elements\

Python stdout - where print() text goes

NIS-Elements installs hook to both python stderr and stdout and routes it to the Log file. To see it go to Help > Open Log File).

Python nodes in GA3

output() function

The purpose of this function is to define all aspects of output parameters by modifying out tuple items.

Output() is typically called before the run of GA3 after some user change upstream the graph. The information provided is used by the downstream nodes in GUI and run (for example table columns defined here are listed dependent nodes).

In GA3 output parameters can have assigned input or can be create new output. When they have assigned input, they inherit name, color and the number of components from that input. The output changes as the input gets reconnected. New outputs however, must be fully defined.

For assigning all output items have assign(...) method which takes input of the same type. For creating new output there are dedicated makeNew*(...) methods. All these methods return self and can be safely concatenated.

By default the node tries to assign to every output the first input of the same type. It this is not possible a new output is made.

Here is a summary of the methods:

class OutputChannelDef:
    def assign(self, param: InputChannelDef) -> OutputChannelDef:
        ...
    def makeNewMono(self, name: str, color: AnyColorDef) -> OutputChannelDef:
        ...
    def makeNewRgb(self, name: str|None = None) -> OutputChannelDef:
        ...
    # change bitdepth
    def makeFloat(self) -> OutputChannelDef:
        ...
    def makeUInt16(self) -> OutputChannelDef:
        ...
    def makeUInt8(self) -> OutputChannelDef:
        ...

class OutputBinaryDef:
    def assign(self, param: InputBinaryDef) -> OutputBinaryDef:
        ...
    def makeNew(self, name: str, color: AnyColorDef) -> OutputBinaryDef:
        ...
    # change bitdepth
    def makeInt32(self) -> OutputBinaryDef:
        ...
    def makeUInt8(self) -> OutputBinaryDef:
        ...

class OutputTableDef:
    def assign(self, param: InputTableDef) -> OutputTableDef:
        ...
    # new empty table
    def makeEmpty(self, name: str) -> OutputTableDef:
        ...
    # optional InputTableDef to copy columns from or if None add just loopCols
    def makeNew(self, name: str, param: InputTableDef|None = None) -> OutputTableDef:
        ...
    # to add all necessary loop columns
    def withLoopCols(self) -> OutputTableDef:
        ...
    # to add Entity and ObjectID columns
    def withObjectCols(self) -> OutputTableDef:
        ...
    # and arbitrary data columns based on type
    def withIntCol(self, title: str, unit: str|None = None, feature:str|None = None, id: str|None = None) -> OutputTableDef:
        ...
    def withFloatCol(self, title: str, unit: str|None = None, feature:str|None = None, id: str|None = None) -> OutputTableDef:
        ...
    def withStringCol(self, title: str, unit: str|None = None, feature:str|None = None, id: str|None = None) -> OutputTableDef:
        ...

Channels and binaries

Example of assigning an input channel to output and changing it to float:

def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    out[0].assign(inp[0]).makeFloat()

Example of creating a new YFP (yellow) 16bit channel:

def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    out[0].makeNewMono('YFP', "#FFFF00").makeUInt16() # color can be a RGB tuple too: (255, 255, 0)

Tables

Example of adding two columns to an output table:

# by making new the table has only loop columns and the two columns added to it
def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    unit_x, unit_y, _ = inp[0].units # inp[0] is InputChannelDef or InputBinaryDef
    out[0].makeNew("Records").withFloatCol("X coord", unit_x).withFloatCol("Y coord", unit_y)

# by assigning the table gets all columns from inp[0] and a new Ratio column of type float
def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    out[0].assign(inp[0]).withFloatCol("Ratio")    

build() function

It is called before run to enquire hew to call the run method. By default it returns None to indicate that run(...) will be called for each frame/volume once.

Other two possibilities are for:

• stack reduction, and

• two-pass algorithms

For stack reduction return ReduceProgram(...) with appropriate loop overZStack() or overTimeLapse() method.

def build(loops: list[limnode.LoopDef]) -> limnode.Program|None:
    return limnode.ReduceProgram(loops).overZStack()

For two-pass algorithms return TwoPassProgram(...) with appropriate loop overZStack() or overTimeLapse() method.

def build(loops: list[limnode.LoopDef]) -> limnode.Program|None:
    return limnode.TwoPassProgram(loops).overZStack()

run() function

It is called for every frame/volume (depending on build(...) return value) to fill the data of each out item.

Channels and binaries

For OutputChannelData and OutputBinaryData the data property is a numpy ndarry of shape (z, y, x, comps) and dtype defined by output() function. By default it is filled with zeros.

# to simply multiply the input by factor of two
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
    out[0].data[:] = inp[0].data[:] * 2

Tables

For OutputTableData the data property is a table (type LimTableDataBase defined in limtabledata.py). Table columns are defined by output function(). By default the table is empty (there is no data in any column).

# for more columns at once
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
    out[0].withColData(["X coord", "Y coord"], [ [0.0, 0.5], [0.0, 0.5] ])  # list, list[list]

# for inp[0] being an assigned (has same columns) InputTableData
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
    a, b = inp[0].colArray(["MeanOfDiO", "MeanOfDiI"])
    out[0].withColDataFrom(inp[0]).withColData("Ratio", a/b)

Each item has prefilled recdata (of type LimTableDataBase) with per-frame recorded data (e.g. AcqTime, X, Y, Z, ...) which can be altered.

Note

The columns in colArray() and withColData() are regular experssions (columns are looked up using re.search(colname)). Thus, for exact match use "^colname$" (see colIndexByPattern()in limtabledata.py).

The context

The context ctx provides more information about the current run:

@dataclass(kw_only=True)
class RunContext:
    inpFilename: str                            # full input filename
    outFilename: str                            # full output filename (typically same as inpFilename except in Cluster Computing)
    inpParameterCoordinates: tuple[tuple[int]]  # loop coordinates for every input parameter 
    outCoordinates: tuple[int]                  # output loop coordinates
    reducingCoordinateIndexes: tuple[int]|None  # loop of loops being reduced or None when not reducing
    finalCall: bool                             # when out should be filled (when false out.data and recdata are set to None)

    @property
    def shouldAbort(self) -> bool:              # abort is requested
        ...

ctx.shouldAbort should be checked whenever possible. However, it is not possible to abort calls into libraries. In such cases switch out of process execution ON.

Reduce

When ReduceProgram was returned in build() the run() function must fill the out items only when ctx.lastCall == True.

Before lastCall it is expected to accumulate the result like in this maximum intensity example:

# IMPORTANT: 'limnode' must be imported like this (not from nor as)
import limnode, numpy as np

tmp = None

# defines output parameter properties
def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    out[0].makeNewMono("DiY", (0, 0, 255))

# return Program for dimension reduction or two-pass processing
def build(loops: list[limnode.LoopDef]) -> limnode.Program|None:
    return limnode.ReduceProgram(loops).overZStack()

# called for each frame/volume
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
    global tmp
    if tmp is None:
        tmp = np.zeros(inp[0].data.shape, inp[0].data.dtype)
    tmp = np.maximum(tmp, inp[0].data)
    if ctx.finalCall:
        out[0].data[:] = tmp

# child process initialization (when outproc is set)
if __name__ == '__main__':
    limnode.child_main(run, output, build)

Two-passes

When TwoPassProgram was returned in build() the run() function must fill the out items only when ctx.lastCall == True. The run can monitor the frame/volume being processed in ctx.inpParameterCoordinates and ctx.reducingCoordinateIndexes. During first pass the node analyses data and then during second pass if fills the out items.

In the example below the node accumulates a maximum intensity projection in the first pass and during the second pass it subtracts current frame from it:

import limnode, numpy as np

tmp = None

# defines output parameter properties
def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    out[0].assign(inp[0])

# return Program for dimension reduction or two-pass processing
def build(loops: list[limnode.LoopDef]) -> limnode.Program|None:
    return limnode.TwoPassProgram(loops).overZStack()

# called for each frame/volume
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
    global tmp
    if tmp is None:
        tmp = np.zeros(inp[0].data.shape, inp[0].data.dtype)
    if ctx.finalCall:
        out[0].data[:] = tmp - inp[0].data[:]
    else:
        tmp = np.maximum(tmp, inp[0].data)   

# child process initialization (when outproc is set)
if __name__ == '__main__':
    limnode.child_main(run, output, build) 

Important

All numpy.ndarray data properties are valid only during the particular run. If the algorithm has to to store it in a variable it has to make a copy of it.

tmp = numpy.copy(unp[0].data[0, :, :, 0])

Out of process run

Many times the required python modules are incompatible with NIS-Elements or lengthy operation dictate the need for out of process execution. In such scenario all three functions output, build and run are called in a separate process. Specifically the C:\Program Files\NIS Elements\Python\pythonw.exe is and the script is passed to it. All calls are forwarded to that process.

For details see limnode.py.

Examples

Using scikit-image for Otsu segmentation

This example will show how to install an additional package (skimage) and use it for basic segmentation.

Install the package using pip.bat and check that it is installed and can be imported using python.bat:

CD \Program Files\NIS Elements
pip.bat install scikit-image
python.bat
>>> import skimage

Open the C:\Program Files\NIS-Elements\Images\agnor.tif.

In NIS-Elements GA3 Editor:

1. Drag the Python node (in the ND Processing & Conversions section) into the graph.

2. Open the node settings dialog by clicking on the ... and add one color input and one binary output .

3. Connect the input pin to the blue channel.

4. Connect the output pin to the SaveBinaries node.

5. Paste the code below inside the dialog.

Figure 727. 

# IMPORTANT: 'limnode' must be imported like this (not from nor as)
import limnode, numpy
from skimage import filters

# defines output parameter properties
def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    out[0].makeNew("otsu", (0, 255, 255))

# return Program for dimension reduction or two-pass processing
def build(loops: list[limnode.LoopDef]) -> limnode.Program|None:
    return None

# called for each frame/volume
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
    image = inp[0].data[0, :, :, 0]
    threshold = filters.threshold_otsu(image)
    binary = image < threshold
    out[0].data[0, :, :, 0] = binary.astype(numpy.uint8)

# child process initialization (when outproc is set)
if __name__ == '__main__':
    limnode.child_main(run, output, build) 

The code:

1. Imports filters from the skimage package.

2. In the output() function defines the output to be a new binary named “otsu” and cyan color (0, 255, 255) or “#00FFFF”.

3. In the run() function:

   1. takes the image from the inp array,

   2. calculates the threshold calling threshold_otsu,

   3. creates the binary by directly comparing the values from the image with the threshold value (in this example objects are dark), and

   4. sets the binary data into the out array while converting to the proper format (uint8 or int32 for binary IDs).

The resulting thresholded binary image should look like this:

Figure 728. 

Using Cellpose for segmentation

Cellpose is well known package for cellular segmentation (see cellpose.org).

First install the cellpose module:

C:\Program Files\NIS Elements>pip.bat install cellpose[gui]

In NIS-Elements open an image, insert python node and set it up for segmentation (one channel input and one binary output) like so:

Figure 729. 

Because the libraries used by cellpose collide with NIS-Elements, make sure to switch ON the Run out of process.

Then edit the python code as follow:

# IMPORTANT: 'limnode' must be imported like this (not from nor as)
import limnode, numpy
from cellpose import models

model = models.Cellpose(model_type='cyto3')

# defines output parameter properties
def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    out[0].makeNew("cell", (0, 255, 255))

# return Program for dimension reduction or two-pass processing
def build(loops: list[limnode.LoopDef]) -> limnode.Program|None:
    return None

# called for each frame/volume
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
    masks, flows, styles, diams = model.eval(inp[0].data[0, :, :, 0])
    out[0].data[0, :, :, 0] = masks.astype(numpy.uint8)

# child process initialization (when outproc is set) 
if __name__ == '__main__':
    limnode.child_main(run, output, build) 

And see the results:

Figure 730. 

All imports and processing defined in global scope is often run by GA3 editor and can result in frequent freezing of GA3 editor. To make GA3 editor more responsible, try to hide these in functions.

Here is short example how to rewrite Cellpose segmentation to make editor more responsive.

# IMPORTANT: 'limnode' must be imported like this (not from nor as)
import limnode

model = None

# defines output parameter properties
def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:
    out[0].makeNew("cell", (0, 255, 255))

# return Program for dimension reduction or two-pass processing
def build(loops: list[limnode.LoopDef]) -> limnode.Program|None:
    return None

# called for each frame/volume
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
    import numpy
    from cellpose import models
    
    global model
    if model is None:
        model = models.Cellpose(model_type='cyto3')
    
    masks, flows, styles, diams = model.eval(inp[0].data[0, :, :, 0])
    out[0].data[0, :, :, 0] = masks.astype(numpy.uint8) 

# child process initialization (when outproc is set) 
if __name__ == '__main__':
    limnode.child_main(run, output, build) 

All imports except limnode are moved to function 'run' as its only place where they are needed. Global variable “model” is initially assigned to None and is reassigned in “run”.

Using the BaSiCPy for shading removal

BaSiCPy is a library "for background and shading correction of optical microscopy images" (see BaSiCPy @ github.com).

First install the basicpy module:

C:\Program Files\NIS Elements>pip.bat install basicpy

Setup the node for processing by adding Channel input and output. It is not necessary to run BaSiC in different process.

The python con be written like so:

# IMPORTANT: 'limnode' must be imported like this (not from nor as)
import limnode, numpy
from basicpy import BaSiC

basic = BaSiC(get_darkfield=True)
fitted: bool = False
imgcache: list[numpy.ndarray] = []

# defines output parameter properties  
def output(inp: tuple[limnode.AnyInDef], out: tuple[limnode.AnyOutDef]) -> None:  
   pass

# return Program for dimension reduction or two-pass processing
def build(loops: list[limnode.LoopDef]) -> limnode.Program|None:
   return limnode.TwoPassProgram(loops).overMultiPoint()  

# called for each frame/volume
def run(inp: tuple[limnode.AnyInData], out: tuple[limnode.AnyOutData], ctx: limnode.RunContext) -> None:
   global imgcache, fitted, basic
   if ctx.finalCall:
       if not fitted:
           stk = numpy.stack(imgcache, axis=0)
           basic.fit(stk)
           imgcache = []
           fitted = True
       out[0].data[0, :, :, 0] = basic.transform(inp[0].data[0, :, :, 0])[0]
   else:
       imgcache.append(numpy.copy(inp[0].data[0, :, :, 0]))
       fitted = False

# child process initialization (when outproc is set)  
if __name__ == '__main__':
   limnode.child_main(run, output, build)   

The result looks like this:

Figure 731.