Deep Learning Nuclei Detection

Nuclei (Deep Learning) sits in a layer's Detection section, beside the classical NucleiLoading... engine. It takes a grayscale image and works only where nuclei are the bright structures on a dark background, as in a DAPI channel. A trained model performs the detection and segmentation, and the result is a set of events in a coded imageLoading.... All pixels of one nucleus share a unique ID, which keeps each nucleus's shape even where nuclei touch.

A 16-bit image is first converted to 8 bits over the 16-bit conversion range you set. The other controls stay hidden until you right-click the engine's window and enable Show Advanced Parameters. Magnification tells the engine how the images were acquired, and it is filled in automatically when that information is available. Use Merging Rules can join touching events, and Use Border Input borrows pixels from neighboring fields of viewLoading....

The engine has no confidence threshold and no area limits. To drop events by size or intensity after detection, add a Post-Processing engine such as Remove objects v2Loading....

From written rules to learned ones. In image analysis generally, classical nuclear segmentation is built from steps someone designed: a threshold, a blob or shape criterion, a way to split touching objects. Those steps work well when their assumptions hold. A deep learning model instead learns, from images with the nuclei outlined, which patterns of edges, texture and context mark a nucleus. The patterns can be hard to put into words, which is often why such models cope with crowded or unevenly stained nuclei.

Convolutional networks, in general. Many image segmentation models are convolutional neural networks. Their early layers respond to simple features such as edges, much like a Sobel filterLoading..., and deeper layers combine those into larger shapes. That is the textbook picture. The Engines Reference says only "trained deep learning model," so don't read it as a description of this engine's network.

Training data sets the limits. A model generally does best on images like the ones it learned from, and less predictably on images unlike them: an unusual stain, magnification or tissue. That is one plausible reason the engine requires the Magnification, though the reference doesn't give one. Check the output overlay on a few fields of each new sample type before you trust a batch.

Parameters marked advanced appear only after you enable Show Advanced Parameters from the right-click menu on the engine's window.

Source: StrataQuest 8.1 Engines Reference, ch. 25 (pp. 444–448), ch. 26 (pp. 449–452), ch. 37 (p. 504), ch. 46 (pp. 557–559); User Manual §3 (p. 15).

StrataQuest offers three Detection engines that turn a nuclear image into events. All three need nuclei bright on a dark background, and all three return a coded image.

• NucleiLoading... has the most controls: a Nuclei size hint, the background threshold with Otsu's method, percentage removal of small or weak objects, fixed Remove Labels limits and a Post Processing Order. The Engines Reference lists the Python environment as a requirement only for the other two.
• Nuclei (Deep Learning) runs TissueGnostics' trained model. You give up the threshold and removal controls, and keep merging rules and border input.
• Nuclear Segmentation Plug-in runs third-party models, CellPose or InstanSeg, through the same Python environment. For CellPose you can set an approximate Diameter (0 runs it in auto mode) and an optional Mask image, such as a cytoplasmic marker.

The documentation doesn't compare their accuracy or speed, so the test is yours. Run two of them on the same few fields and compare the overlays where nuclei are crowded, dim or oddly shaped. Whichever you choose, you can prune the result with Remove objects v2Loading... and edit what's left with Manual CorrectionLoading....