Can FAST currently achieve patch-level segmentation?

[liuyichong2250]

I am currently working on segmentation for patches from WSI images. Specifically, after dividing the WSI into patches, I want to perform inference on each patch and display the results. Each patch will have multiple segmentation categories, so I need to show the segmentation results for each patch individually. I would like to know if the current version of FAST has any related functions that can achieve this: performing inference on each patch, drawing the masks, and displaying the results. If this is possible, could you provide a sample code or example? Thank you!

I wrote the following code based on the official website example, but it doesn't work at all:

auto importer = WholeSlideImageImporter::create("/Users/liuyichong/Downloads/CMU-1-JP2K-33005.svs");

 auto tissueSegmentation = TissueSegmentation::create()->connect(importer);

 auto generator = PatchGenerator::create(512, 512, 1, 0)
         ->connect(importer)
         ->connect(1, tissueSegmentation);

 InferenceEngine::pointer engine = InferenceEngineManager::loadEngine("ONNXRuntime");

 std::string modelPath = "/Users/liuyichong/cc_code/PsoriasisSystem/show_fast/best_model.onnx";

 auto network = NeuralNetwork::create(modelPath, {}, {}, engine->getName())
         ->connect(generator);
 network->setScaleFactor(1.0f/255.0f); // Add a preprocessing step: multiply each pixel with 1/255, thus normalizing the intensity

 auto tensor2seg = TensorToSegmentation::create()
         ->connect(network);

 auto stitcher = PatchStitcher::create()
         ->connect(tensor2seg);

 auto renderer = ImageRenderer::create()
         ->connect(importer);

 auto segRenderer = SegmentationRenderer::create()
         ->connect(stitcher);
 segRenderer->setColor(1, Color::Red()); // Set color for class 1
 segRenderer->setColor(2, Color::Blue()); // Set color for class 2
 segRenderer->setColor(3, Color::Yellow()); // Set color for class 3
 auto window = SimpleWindow2D::create()
         ->connect({renderer, segRenderer});
 window->run();

[smistad]

Yes, FAST has the functionality to do this.
I see one mistake in your code; instead of ImageRenderer you should use ImagePyramidRenderer

[smistad]

Two other comments regarding your code:
The InferenceEngine::pointer engine = InferenceEngineManager::loadEngine("ONNXRuntime"); line is unnecessary.
Replace engine->getName() with "ONNXRuntime" instead.
Also, NeuralNetwork followed by TensorToSegmentation can be replaced by a single SegmentationNetwork object instead: https://fast.eriksmistad.no/classfast_1_1SegmentationNetwork.html

[liuyichong2250]

Thank you very much for your careful answer! I believe my current problem has been solved!☺️