Measurement on the whole image file is performed frame-by-frame (volume-by-volume). This action can be used once or multiple times to obtain records (table rows) from all frames (or volumes) in the selected loop. By default (without accumulate) for performance reasons, records are generated per frame (or per volume). In order to work (sorting, filtering, calculating statistics, ...) on a larger record, use accumulate.

Columns from the input tables are appended in the order that they appear. This is useful when joining more features measured on the same number of objects. Number of records across all input tables is expected to be the same.

Figure 891. Append Columns action results shown in the bottom table. Left table belongs to “Records 1” whereas the right table belongs to “Records 2”. EqDiameter feature is duplicated.

This node simply works as a calculator with pre-defined operators. Existing columns or their statistics (see Data manipulation > Grouping > Aggregate Rows) can be used as variables. Type of the calculated column and Unit should be set first. Once the calculation is written, confirm it by clicking Apply.

Selects which features are displayed in the resulting table, change their order using the arrows, change their name (New Title ), number format (New Format ) and Precision.

Groups records and computes the selected statistic of rows. If the table is grouped the statistic is calculated for each group. Each group of rows becomes one row in the output table.

For more information about grouping see Data manipulation > Grouping > Group Records, for more information about statistics see Data manipulation > Grouping > Aggregate Rows.

This node is used for changing units, skipping values and similar tasks. Select the Column which is about to be changed, define the new Unit (NewVal example is shown below) and set the value Offset or Gain. No new column is created, only the values in the source column are skipped.

Aggregate Rows computes the selected statistic of rows. If the table is grouped the statistic is calculated for each group. Each group of rows becomes one row in the output table.

* Does not include empty (null) values.

Filters whole groups (selected in the Column) based on the selected Statistics per group. Define the filtering using the Comparator drop-down menu and the Value edit box.

This node uses aggregation statistics. Please see Data manipulation > Grouping > Aggregate Rows.

All rows are in one group by default. Rows having an equal given column (selected in this node) form a group which is visualized in the table. Select the feature(s) in the Group by drop-down menu. To remove a grouping feature, switch it back to “---”.

Grouping is used with statistics (Aggregate Rows) for calculating per group (BinaryLayer, ZStack, Object...) aggregates. Only limitation is that the result of the grouping is currently not visible in the preview and in the final result.

Previously grouped records are ungrouped by this action.

Filters records associated to the current frame. This node is useful only when the connected table is accumulated.

This action can be used to filter table records (rows) using a selected column. Resulting table contains only the rows satisfying the filter condition applied to the given column.

Example 10. Filtering out Spots not having any Cell as a parent.

Figure 892. 

Example 11. Filter Records action used in series to act as an intersection. Both conditions (10 <= EqDiameter AND 0.9 <= Circularity) must be met at the same time.

Figure 893. 

Example 12. Filter Records action used n parallel to act as an union. One or both conditions (10 <= EqDiameter OR 0.9 <= Circularity) must be true.

Figure 894. 

This node “pivots” the input table by the specified Pivot Column (“ObjectId” in the example below), thus it creates a new column for each “ObjectId” value. The new column will contain values of columns having a column Role in the bottom definition table (“Entity” and “EqDiameter” in the example below). E.g. in the second column of the results table, the “EqDiameter” values for objects with ID==1 are arranged in rows by the “ZStackIndex”. The number of rows depends on the possible combinations of all values of columns marked with the role Row. Each column will be present for the first number of values specified in the Column Count edit box. The order of the columns is specified by the Column Order switch. In our example below, Append would add 10x Entity whereas Mix orders them: Entity, EqDia, Entity, EqDia, etc. Column Suffix sets the name of the created column. PIVOT_NAME (“ObjectId” in our example) and PIVOT_VALUE (“ObjectId” value) or any other text can be used here.

Figure 895. Pivot Table example

Figure 896. Pivot Table example results

Shows the first and last record from the connected table.

Select Records action takes a specified number of records (Count) from a selected starting row (First Row ). Typical use-case is to select the first row of a sorted record set.

Reorders the records based on a given column and then shows the defined number of records (either smallest or biggest). This node is a combination of Data manipulation > Sort & Select > Sort Records and Data manipulation > Sort & Select > Select Records.

Sort action reorders records so that the selected column is sorted in an ascending order.

Appends records (rows) from input tables one by one into the output table. If an input table contains a column already present (has same column ID) in the output table (from the preceeding input tables) it will use it (i.e. not append a new column).

The resulting table is ordered by the identification columns.

Column ID 
Every column has an implicit ID (invisible to the user) given to it by the node that creates the column. The ID is used internally to reference columns. Therefore, even after a column is renamed it is still correctly pointed to by subsequent nodes. Consequently, columns are considered “same” if they have the same ID. Special identification columns like Loop indexes, Object Entity, Object IDs are given the same ID for each special column. Therefore the ObjectID column will be considered the same from all tables. This behavior is usually expected. If not, it can be altered with these nodes: Data manipulation > Table Manipulation > Copy Column ID, Data manipulation > Table Manipulation > New Column ID and Data manipulation > Table Manipulation > Compact Columns.

Appending records is useful when joining two tables with the same columns. For example for joining two disjunct filtrations or aggregations.

Figure 897. 

Figure 898. 

Figure 899. 

Figure 900. 

Aggregates all columns with the same title (or title different just in numerical suffix) to the first column using the First Valid rule.

This is useful when Data manipulation > Table Manipulation > Join Records or Data manipulation > Table Manipulation > Append Records produces duplicate colums because of their different IDs (please see Data manipulation > Table Manipulation > Append Records for more on Column IDs).

For more control on which columns will merge use Copy Column ID.

The following example demonstrates how to force two different columns to merge into single one using Data manipulation > Table Manipulation > Append Records.

Figure 901. 

After append records there are two columns “MeanObjectIntensity” and “MeanObjectIntensity2” that need to be merged into one.

Figure 902. 

Figure 903. 

The resulting table contains all columns from table A with specified columns IDs replaced with IDs from reference table B.

This is useful when tables A and B are intended to be merged using Data manipulation > Table Manipulation > Join Records or Data manipulation > Table Manipulation > Append Records node and they contain columns with different IDs (please see Data manipulation > Table Manipulation > Append Records for more on Column IDs) that should be treated as same column.

There is a simpler automatic way using Data manipulation > Table Manipulation > Compact Columns.

The following example demonstrates how to force two different columns to merge into a single one using Data manipulation > Table Manipulation > Append Records.

Figure 904. 

Figure 905. 

Figure 906. 

Duplicates the selected column.

This action is useful for merging incompatible tables:

It joins records from two or more unrelated (with different number of rows and columns) record sets. Resulting table contains a union of all distinct input table columns (loop indexes, entity and Object Ids are considered the same).

Select the type of join (see below) and click to add a table row where you specify the relation between the joined features taken from two different tables.

Figure 907. Join Records action results shown in the bottom table. Left table belongs to “Objects” whereas the right table belongs to “Frame”.

Makes the specified columns IDs unique (please see Data manipulation > Table Manipulation > Append Records for more on Column IDs).

It is useful when nodes like Data manipulation > Basic > Append Columns, Data manipulation > Table Manipulation > Join Records and others treat a column coming from two different tables as the same one and does not include it twice. This is typically because both columns were made by one node or the is a “system” column like “Loop Index Columns”, “Entity” or “Object ID”.

Figure 908. 

Figure 909. 

Figure 910. 

Figure 911. 

Without the Data manipulation > Table Manipulation > New Column ID node both source columns are merged (overwritten) into one.

Figure 912. 

Shifts data in selected columns by given number of rows.

Aggregate Columns action creates a new column with the given name (Title) in which the result of a calculation is placed (Aggregation). The calculation is based on the checked parameters.

This node uses aggregation statistics. Please see Data manipulation > Grouping > Aggregate Rows.

Creates new bin column and assigns each row to one bin based on the value of the selected column. Then the rows can be grouped based on the bins.

Creates a new column and fills it with bin labels to which the value in the source column falls into. Bins are equidistant intervals between min and max.

This table can be used to classify the results data to see the number of elements in each defined class. Select the Source column, name the New column and add units (Unit). Fill in the table so that From and To values represent the bin size (range of source values) which will be substituted by the new value specified in the third column.

Other tools are similar to the Data manipulation > Statistics > Binning node.

(requires: Local Option)

Creates a table with points of Probability density function or Cumulative distribution function. Select the Distribution, its Parameters, left and right interval endpoints and Step. If checked, y- values of the Critical Region are generated to a separate column. Tested Value is generated to a separated column as well.

Creates a summary statistics table, as seen in the example below.

Figure 915. 

This node uses aggregation statistics. Please see Data manipulation > Grouping > Aggregate Rows.

(requires: Local Option)

Performs One-way ANOVA (analysis of variance). Please see one-way ANOVA.

Select a column with Sample Data and parameters of the test. The table must be grouped. Every group is one factor (i.e. treatment group). The output results of the analysis are displayed in a one row table.

Working example from a Wikipedia page:

(requires: Local Option)

This test can be used to test:

Select Sample A from table A, Sample B from table B and parameters of test. If tables are grouped, then count of the groups in each table must be same. F-test is done for each pair of the groups (i-th group in table A and i-th group in table B). Output is a table with one row of data for each group pair.

(requires: Local Option)

This is a location test of whether the mean of a population has a value specified in a null hypothesis (please see t-test).

Select Sample and parameters of test. If table is grouped, then the t-test is done for each group. Output is a table with one row of data for each group.

(requires: Local Option)

Paired samples t-tests typically consist of a sample of matched pairs of similar units, or one group of units that has been tested twice (a "repeated measures" t-test). A typical example of the repeated measures t-test would be where subjects are tested prior to a treatment, say for high blood pressure, and the same subjects are tested again after treatment with a blood-pressure-lowering medication. By comparing the same patient's numbers before and after treatment, we are effectively using each patient as their own control (please see t-test).

Select Sample A, Sample B and parameters of test. If table is grouped, then the t-test is done for each group. Output is a table with one row of data for each group.

(requires: Local Option)

The independent (unpaired) samples t-test is used when two separate sets of independent and identically distributed samples are obtained, one from each of the two populations being compared. For example, suppose we are evaluating the effect of a medical treatment, and we enroll 100 subjects into our study, then randomly assign 50 subjects to the treatment group and 50 subjects to the control group. In this case, we have two independent samples and would use the unpaired form of the t-test (please see t-test).

Select Sample A from table A, Sample B from table B and parameters of test. If tables are grouped, then count of the groups in each table must be the same. t-test is done for each pair of groups (i-th group in table A and i-th group in table B). Output is a table with one row of data for each group pair.

Calculates Z-factor for positive and negative control groups.

Vectorally adds two positions. Z positions are optional. The result is two or three new columns (X, Y, (Z)).

ResX = X0 + X1, ...

Vectorally subtracts two positions in time. Z positions are optional. The result is two or three new columns (X, Y, (Z)).

ResX = X0 - X1, ...

Calculates the distance between two points in time. Z positions are optional. The result is two or three new columns (X, Y, (Z)).

Dist = SQRT( (X0 - X1)^2 + (Y0 - Y1)^2 + (Z0 - Z1)^2 )

Sorts given points represented by their positions in the most effective order for scanning.

Calculates the length of the vector. If Z (Vector Z) column is left blank 2D length is calculated:

Figure 921. Vector Length in 2D

Figure 922. Vector Length in 3D

Calculates the heading of the vector. Heading is the angle in degrees between the positive X axis and the counterclockwise in range <0; 360>. Elevation is calculated as well when the Z (Vector Z) column is given. Elevation is the angle in degrees between the XY plane and the in range <-90; 90>.

Transforms (recalculates) the position coordinates between the stage (absolute) and the image (relative) system.

Only the local extreme values found in the source Column are copied into the new column. Name the New Column and specify its units (Unit). Select which Extrema are taken into account and adjust the Threshold to filter out subtle changes.

Calculates the rolling average value as an average of three consecutive values (current value and the value before and after the current one). Select the Column from which the rolling average will be calculated, name the New Column and specify its units (Unit). Use Window Radius to expand the number of consecutive values from which the average is calculated.

Calculates the rolling median value as a median of three consecutive values (current value and the value before and after the current one).

Calculates the rolling minimum value as a minimum of three consecutive values (current value and the value before and after the current one).

Calculates the rolling maximum value as a maximum of three consecutive values (current value and the value before and after the current one).

Fits a curve to data using the method of least squares. Fit values are placed into a new column. Select the Dependent Column and Independent Column, choose the Curve type and specify the Outputs.

Fits a Dose-Response curve to data using the method of non-linear least squares. Fitted values are placed into a new column.

Fits multiple gaussian curves to data using the method of nonlinear least squares.

This node is similar to the Data manipulation > Curve Fitting > Dose Response node. It fits a growth curve to data and is used by the Results & Graphs > Graphs > Fitplot and Results & Graphs > Reports > HTML Report.

This Density-Based Spatial Clustering of Applications with Noise (DBSCAN ) action clusters 2D points. Cluster IDs are inserted into a new column. Name the new column (New Column Name ), select the Position X Column and Position Y Column , set the Max. Distance in Cluster and the Min. Number of Neighbours . Two points are neighbors if their distance from each other is not greater than the Maximal Distance in Cluster. If any point has at least Minimal Number of Neighbours, this point and all his neighbors are added to the same cluster. If any point is part of a cluster, all of his neighbors are also part of this cluster.

For details about this action, please see the DBSCAN method.

Detects a square grid and appends rows with coordinates of the missing points. Select columns containing coordinates of the grid points and select what to ignore during the detecting of the missing points.

Recognizes and parses well names to row and column.

The node classifies objects in a table based on their measured features.

Figure 924. 

Inserts a new column containing new indexes of binary objects of detected TMA (Tissue microarray) cores. Choose the Position X Column and Position Y Column with position of core centers, select the Orientation and then set the indexing start (Index from) or choose the Meander indexing. The settings are shown in the preview scheme below. Select Continuous indexing to not include numbers of missing cores in indexing (so binaries will be numbered from 1 to N, where N is the count of binary objects).

This node automatically detects columns and rows of grid of TMA cores. Then it assigns new indexes so that cores in the upper row have smaller indexes than cores in the lower row. In each row, cores are indexed from left to right. Numbers of missing cores are not used (for example, if you have 10 cores in a 4x3 grid, core in the bottom right corner (if not missing) will always have an index of 12).

For good results cores should be laying in a square grid which is not rotated.

(requires: Local Option)

Detects run of the same values and returns the column with IDs. Same ID indicates the same values in the Column for consecutive indexes in the Index Column. New Column adjusts the name of the new column.

Calculates a new column as a difference (subtraction) between the second and first value, third and second, etc. Select the Column from which the difference will be calculated, choose the Loop, name the New Column and specify its units (Unit). First record is blank when First item is selected.

Calculates a new column as a sum of the first and the second value, second and the third value, etc. First row stays the same. Select the Column from which the integration will be calculated, choose the Loop, name the New Column and specify its units (Unit). Enter a Constant if you want to add a number to each sum of the two values.

Computes difference (vector) X, Y, Z columns in every two consecutive rows.

Figure 925. 

Figure 926. 

Figure 927. 

Adds up X, Y, Z columns in every two consecutive rows.

Figure 928. 

Figure 929. 

Figure 930. 

Figure 931. Alpha calculation

ΔT is a time interval [s] and fc is a cut-off frequency [Hz].

Please see High Pass Filter for more details.

Figure 932. Alpha calculation

ΔT is a time interval [s] and fc is a cut-off frequency [Hz].

Please see Low Pass Filter for more details.

Generates a new column with an integer sequence. Name the New Column and set the Start and Step value of the sequence. Optionally the sequence can respect a selected loop over column value if switched from the default “<rows>” value. New Column adjusts the name of the new column.

Generates a new column with an exponential sequence. Name the New Column and set the Start and Factor (exponent) value of the sequence. To create an inverted exponential sequence, click Invert factor and the Factor value is automatically changed. Optionally the sequence can respect a selected Loop over column value if switched from the default “<rows>” value.

Define new columns in the upper table by specifying their name, type, and unit.

import pandas as pd
import numpy as np
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

# take only the float columns
df_float64 = df.select_dtypes(include=["Float64"])

# scale the input
df_float64[ ["BinMeanOfBlue", "BinMeanOfGreen", "BinMeanOfRed" ] ] *= 5.0

print(df_float64.head())

X = df_float64.values

# Scale after weighting
scaler = StandardScaler()
scaled_features = scaler.fit_transform(X)

# Apply K-means clustering
kmeans = KMeans(n_clusters=4, random_state=42, n_init=10)

#set the output dataset
out["Pollen_Class"] = kmeans.fit_predict(X)

When the following question is added to the generated prompt:

Task (provided by the user):
Classify the objects into two classes.

The ChatGPT 4o outputs following column definition and code:

[
  {"name": "ClassLabel", "type": "int", "unit": null}
]

import pandas as pd
import numpy as np
from sklearn.cluster import KMeans

# Start from an empty frame with the same index
out = pd.DataFrame(index=df.index)

# Select numeric features for clustering (excluding IDs and categorical fields)
features = [
    "Area", "EqDiameter", "Perimeter", "Length", "Width", "MaxFeret", "MinFeret",
    "Circularity", "Elongation", "MeanIntensityEGFP", "MeanIntensityTD",
    "SumIntensityEGFP", "SumIntensityTD"
]

# Prepare data: drop rows with missing values in the selected features
valid_mask = df[features].notna().all(axis=1)
X = df.loc[valid_mask, features].astype("float64")

# Perform KMeans clustering into 2 classes
kmeans = KMeans(n_clusters=2, n_init="auto", random_state=42)
labels = pd.Series(kmeans.fit_predict(X), index=X.index).astype("Int64")

# Create the output column and fill with cluster labels, keeping <NA> where data was insufficient
out["ClassLabel"] = pd.Series(pd.NA, index=df.index, dtype="Int64")
out.loc[labels.index, "ClassLabel"] = labels

Define new columns or copy them from the source tables in the upper table by specifying their name, type, and unit. Copying columns keeps their ID and Metadata.

import pandas as pd
import numpy as np

# predefined globals:
# df: pd.DataFrame with the input table of input "a"
# mdf: pd.DataFrame with the input column metadata table of input "a"
# out: pd.DataFrame with the new columns to be filled

print("Table A:", df[0].columns)
print("Table B:", df[1].columns)

out = pd.merge(*df, on=("TimeLapseIndex", "Entity", "ObjectId"), suffixes=("", None))

Task: I want to find out if there is one or more classes in the input table based on the coefficient of diffusion (CD). Calculate the CD from MSDs. Note that one TrackId has more than one row. Can you help me with that?

Figure 933. 

ChatGPT 5 produced following answer:

[
  {"name": "Entity", "type": "text", "unit": null},
  {"name": "TrackId", "type": "int", "unit": null},
  {"name": "PointsUsed", "type": "int", "unit": null},
  {"name": "CD", "type": "number", "unit": "µm²/s"},
  {"name": "CD_SE", "type": "number", "unit": "µm²/s"},
  {"name": "R2_Origin", "type": "number", "unit": null},
  {"name": "NumClasses", "type": "int", "unit": null},
  {"name": "ClassLabel", "type": "int", "unit": null},
  {"name": "ClassMeanCD", "type": "number", "unit": "µm²/s"},
  {"name": "ClassCount", "type": "int", "unit": null},
  {"name": "BIC_Best", "type": "number", "unit": null}
]

Python code is too long (150 lines) to reproduce here.

Figure 934. Python Create table node with ChatGPT generated code.

From the CD per track it is clear that there is only one class:

Figure 935. Resulting table with per track CD fits and classes.

Define one or more new columns and calculate its data using JavaScript.

Figure 936. 

In the upper table, you can define new columns by specifying their name, type, and unit.

To reference columns from the input table in your JavaScript code, simply click the button with the column name. This will insert a variable declaration with the appropriate column index. For example, clicking on the ZStackIndex column will insert:

var ZStackIndexIndex = tableA.colIndexById("_loopZStackIndex"); //ZStackIndex

This variable holds the index of the input column and can be used to access its values in your script, e.g.:

var tableA = new Table(context.tableForParameter(node.child("A")));
var a = tableA.data;
var ig = tableA.rowsToGroupRows;

var ZStackIndexIndex = tableA.colIndexById("_loopZStackIndex"); //ZStackIndex

function newItemForRow(i) {
   return a[ZStackIndexIndex][i];
}

Input table is already accessible as tableA variable of Table class.

The Table class wraps the input table data and metadata. It provides methods to access columns, rows, groupings, and statistics.

Methods and properties of the Table class 

For each table row function newItemForRow(i) is called. It expects to return either single value or array. If you have defined multiple columns in upper table it expects to return array only.

Creating column with indices of rows 

function newItemForRow(i) {
   return i;
}

Creating column with indices of rows per group 

var tableA = new Table(context.tableForParameter(node.child("A")));
var ig = tableA.rowsToGroupRows;

function newItemForRow(i) {
   return ig[i];
}

Creating column with time delta 

var tableA = new Table(context.tableForParameter(node.child("A")));

//insert here your own time column index (by clicking column button above code editor)
var TimeIndex = tableA.colIndexById("ColumnId");

var times = tableA.data[TimeIndex]

function newItemForRow(i) {
   return (i==0) ? null : times[i] - times[i-1];
}

Creating column with cumulative sum 

var tableA = new Table(context.tableForParameter(node.child("A")));

//insert here your own column index (by clicking column button above code editor)
var ColumnIndex = tableA.colIndexById("ColumnId");

var columnData = tableA.data[ColumnIndex];
var sum = 0;

function newItemForRow(i) {
   sum += columnData[i];
   return sum;
}

Creating column with mean of column data 

var tableA = new Table(context.tableForParameter(node.child("A")));

//insert here your own time column index (by clicking column button above code editor)
var ColumnIndex = tableA.colIndexById("ColumnId");

var columnData = tableA.data[ObjectIdIndex];
var mean = MathApi.mean(columnData);

function newItemForRow(i) {
   return mean;
}

Creating column with group mean of column data 

var tableA = new Table(context.tableForParameter(node.child("A")));
var rowsToGroupIds = tableA.rowsToGroupIds;
var groups = tableA.groups;

//insert here your own time column index (by clicking column button above code editor)
var ColumnIndex  = tableA.colIndexById("ColumnId");

var columnData = tableA.data[ColumnIndex ];

var means = [];
for(let i=0; i<groups.length; i++) {
    let groupData = groups[i].map(idx => columnData[idx]);
    means.push(MathApi.mean(groupData));
}

function newItemForRow(i) {
   return means[rowsToGroupIds[i]];
}

Creates a new table by combining one or more existing tables (table inputs A, B, …). All output table columns must be defined in the node dialog column list table:

Figure 937. 

The code contains a quick reference of predefined variables:

GA3 JavaScript Editor 

// a, b, ... - input tables (use a[col][row])
// ag, bg, ... - grouped input tables (use ag[group][col][rowingroup])

// _?T - time-lapse loop column index
// _?Z - z-stack loop column index
// _?M - multi-point loop column index
// _?0..._?N - column indexes after loop columns
// where ? should be replaced by a, b, ... for input table

The expected output is the data of the new table (an array of arrays columns x rows). The number of columns must match the number of defined columns. And the number of rows must be equal for all columns.