Package napari_blender

napari-blender is a plugin for napari that allows you to render 3D scenes using Blender.

This is a system that combines optical validation of model predictions by having different 3D visualizations and quantitative evaluation. Utilizing Blender’s rendering capabilities for deepening the understanding of nuclei segmentation for users with different levels of expertise. Examples are time-lapse animations (aimed to be generated from label data), opaque ground truth visualizations with solid prediction objects to compare prediction quality optically and 3D images where nuclei are coloured according to their prediction quality. To facilitate meaningful quantitative evaluation, different metrics are calculated for these predictions and displayed within the animation, such as the Jaccard index, Intersection over Union and the F1-score. The emphasis in this system is on operating it with limited technical knowledge, allowing for bridging between researchers and developers; but includes metrics that allow for a deeper understanding of performance for expert users.

Modes

Transparant

This mode is a multiple comparison mode, it works on the premise of having a manual annotation and a prediction. This manual annotation is presumed to be correct, and the prediction is tested against this. These two segmentations are used by the plugin, next to some user-specified parameters, and a visualization is made.


Input

ground-truth : Image Data (.tif file)
Data to which to compare prediction to.
prediction : Image Data (.tif file)
Segmentation to visualize.
start-frame : int
Index of first frame to visualize. (inclusive)
end-frame : int
Index of last frame to visualize. (inclusive)
rotation-speed : Enum (ROTATION)
Frames per full organoid rotation:
  • No_Rotation = 1 (picture)
  • Very_Slow = 300
  • Slow = 200
  • Normal = 150
  • Quick = 100
  • Very_Quick = 50
video-quality : Enum (SAMPLES)
Render samples done by Blender rendering, adds more details and subtler lighting to frame:
  • Low = 1
  • Medium = 26
  • High = 52
output-location : path
Desired location for visualization to be saved.
filename : string (ends with .mp4)
File name of video after saving.
display-text : Checkmark (checked is True, unchecked False)
Whether to render any text or render clean visualization.
scene_name : string
Filename of video, only evaluated if save-scene is checked.
save-scene : Checkmark (checked is True, unchecked False)
Whether to save the Blender Scene used for rendering for later interaction.

Output

visualization : Video (.mp4 file)
Visualization, immediately displayed interactively in napari frame. Also saved in specified location.
3d-objects : 3D objects (.obj files)
Objects used in visualization, able to be interacted with. Saved in ../3d folder.
scene : Blender Scene (optional)
Interactive Blender Scene containing all animations and objects. Saved in ../scenes folder.
Gradient

This mode is a multiple comparison mode, it works on the premise of having a manual annotation and a prediction. This manual annotation is presumed to be correct, and the prediction is tested against this. While it still uses full organoid comparison methods, it focuses on visualizing a single metric. By default, this is the Jaccard-index, but this is customizable by the user.


Input

ground-truth : Image Data (.tif file)
Data to which to compare prediction to.
prediction : Image Data (.tif file)
Segmentation to visualize.
start-frame : int
Index of first frame to visualize. (inclusive)
end-frame : int
Index of last frame to visualize. (inclusive)
rotation-speed : Enum (ROTATION)
Frames per full organoid rotation:
  • No_Rotation = 1 (picture)
  • Very_Slow = 300
  • Slow = 200
  • Normal = 150
  • Quick = 100
  • Very_Quick = 50
video-quality : Enum (SAMPLES)
Render samples done by Blender rendering, adds more details and subtler lighting to frame:
  • Low = 1
  • Medium = 26
  • High = 52
output-location : path
Desired location for visualization to be saved.
filename : string (ends with .mp4)
File name of video after saving.
display-text : Checkmark (checked is True, unchecked False)
Whether to render any text or render clean visualization.
scene_name : string
Filename of video, only evaluated if save-scene is checked.
save-scene : Checkmark (checked is True, unchecked False)
Whether to save the Blender Scene used for rendering for later interaction.

Output

visualization : Video (.mp4 file)
Visualization, immediately displayed interactively in napari frame. Also saved in specified location.
3d-objects : 3D objects (.obj files)
Objects used in visualization, able to be interacted with. Saved in ../3d folder.
scene : Blender Scene (optional)
Interactive Blender Scene containing all animations and objects. Saved in ../scenes folder.
Microscopy

This mode focuses on generating animations that display the quality of nuclei segmentation over time, using time-lapse data. It allows for a deeper understanding of performance and visual comparison. Additionally, it provides a platform for visualizing time-lapse predictions.


Input

ground-truth : Image Data (.tif file)
Data to which to compare prediction to.
prediction : Image Data (.tif file)
Segmentation to visualize.
start-frame : int
Index of first frame to visualize. (inclusive)
end-frame : int
Index of last frame to visualize. (inclusive)
rotation-speed : Enum (ROTATION)
Frames per full organoid rotation:
  • No_Rotation = 1 (picture)
  • Very_Slow = 300
  • Slow = 200
  • Normal = 150
  • Quick = 100
  • Very_Quick = 50
video-quality : Enum (SAMPLES)
Render samples done by Blender rendering, adds more details and subtler lighting to frame:
  • Low = 1
  • Medium = 26
  • High = 52
output-location : path
Desired location for visualization to be saved.
filename : string (ends with .mp4)
File name of video after saving.
display-text : Checkmark (checked is True, unchecked False)
Whether to render any text or render clean visualization.
scene_name : string
Filename of video, only evaluated if save-scene is checked.
save-scene : Checkmark (checked is True, unchecked False)
Whether to save the Blender Scene used for rendering for later interaction.

Output

visualization : Video (.mp4 file)
Visualization, immediately displayed interactively in napari frame. Also saved in specified location.
3d-objects : 3D objects (.obj files)
Objects used in visualization, able to be interacted with. Saved in ../3d folder.
scene : Blender Scene (optional)
Interactive Blender Scene containing all animations and objects. Saved in ../scenes folder.
Linked Nuclei

This mode has a function that is tailored towards the quantification of the quality in the prediction of individual nuclei. It creates a one-to-one assignment of nuclei of the prediction and the ground truth by maximizing the overall metric score of the entire organoid. The Jaccard index is used for this color, and teal and purple indicate False Negatives and False Positives respectively.


Input

ground-truth : Image Data (.tif file)
Data to which to compare prediction to.
prediction : Image Data (.tif file)
Segmentation to visualize.
start-frame : int
Index of first frame to visualize. (inclusive)
end-frame : int
Index of last frame to visualize. (inclusive)
rotation-speed : Enum (ROTATION)
Frames per full organoid rotation:
  • No_Rotation = 1 (picture)
  • Very_Slow = 300
  • Slow = 200
  • Normal = 150
  • Quick = 100
  • Very_Quick = 50
video-quality : Enum (SAMPLES)
Render samples done by Blender rendering, adds more details and subtler lighting to frame:
  • Low = 1
  • Medium = 26
  • High = 52
tracking-file : path
Location of tracking file in .xml or .tmxml format.
output-location : path
Desired location for visualization to be saved.
filename : string (ends with .mp4)
File name of video after saving.
display-text : Checkmark (checked is True, unchecked False)
Whether to render any text or render clean visualization.
scene_name : string
Filename of video, only evaluated if save-scene is checked.
save-scene : Checkmark (checked is True, unchecked False)
Whether to save the Blender Scene used for rendering for later interaction.

Output

visualization : Video (.mp4 file)
Visualization, immediately displayed interactively in napari frame. Also saved in specified location.
3d-objects : 3D objects (.obj files)
Objects used in visualization, able to be interacted with. Saved in ../3d folder.
scene : Blender Scene (optional)
Interactive Blender Scene containing all animations and objects. Saved in ../scenes folder.
Timelapse

This mode gives insight in the development and orientation of the organoid. It uses a tracking file to link nuclei over time, and visualizes this in a 3D animation. It provides intuitiveness and a clear overview of the organoid development.


Input

stack-layer : Image Data (.tif file)
Organoid segmentation to animate.
start-frame : int
Index of first frame to visualize. (inclusive)
end-frame : int
Index of last frame to visualize. (inclusive) If this is the same as start-frame, no animation will be performed.
rotation-speed : Enum (ROTATION)
Frames per full organoid rotation:
  • No_Rotation = 1 (picture)
  • Very_Slow = 300
  • Slow = 200
  • Normal = 150
  • Quick = 100
  • Very_Quick = 50
video-quality : Enum (SAMPLES)
Render samples done by Blender rendering, adds more details and subtler lighting to frame:
  • Low = 1
  • Medium = 26
  • High = 52
tracking-file : path
Location of tracking file in .xml or .tmxml format.
output-location : path
Desired location for visualization to be saved.
filename : string (ends with .mp4)
File name of video after saving.
display-text : Checkmark (checked is True, unchecked False)
Whether to render any text or render clean visualization.
scene_name : string
Filename of video, only evaluated if save-scene is checked.
save-scene : Checkmark (checked is True, unchecked False)
Whether to save the Blender Scene used for rendering for later interaction.

Output

visualization : Video (.mp4 file)
Visualization, immediately displayed interactively in napari frame. Also saved in specified location.
3d-objects : 3D objects (.obj files)
Objects used in visualization, able to be interacted with. Saved in ../3d folder.
scene : Blender Scene (optional)
Interactive Blender Scene containing all animations and objects. Saved in ../scenes folder.
Video Loader

This mode is a complementary addition to ease loading previous videos into napari. It allows for the loading of videos into napari, and the ability to interact with them as an image stack. It utilizes napari_video for this.


Input

video-file : path
Location of video file to be opened.

Output

None