StarDist3D

[stefanhahmann]

I tried running StarDist3D with a small dummy dataset:

import java.io.IOException;
import java.net.URISyntaxException;

import net.imglib2.img.Img;
import net.imglib2.img.array.ArrayImgs;
import net.imglib2.type.numeric.real.FloatType;

import org.apache.commons.compress.archivers.ArchiveException;

import io.bioimage.modelrunner.apposed.appose.MambaInstallException;
import io.bioimage.modelrunner.bioimageio.description.exceptions.ModelSpecsException;
import io.bioimage.modelrunner.exceptions.LoadEngineException;
import io.bioimage.modelrunner.exceptions.LoadModelException;
import io.bioimage.modelrunner.exceptions.RunModelException;
import io.bioimage.modelrunner.model.Stardist3D;

public class Stardist3DDummyPrediction
{
	public static void main( String[] args )
			throws MambaInstallException, IOException, URISyntaxException, ArchiveException, InterruptedException, ModelSpecsException,
			LoadEngineException, RunModelException, LoadModelException
	{
		Stardist3D.installRequirements();
		Stardist3D stardist3D = Stardist3D.fromPretained( "StarDist Plant Nuclei 3D ResNet", false );
		final Img< FloatType > dummyImg = ArrayImgs.floats( 10, 10, 10 );
		stardist3D.predict( dummyImg );
	}
}

When trying to run this, I get:

Exception in thread "main" Error loading a Deep Learning engine located at tensorflow-1.15.0-1.15.0-windows-x86_64-cpu-gpu.
java.lang.reflect.InvocationTargetException
	at io.bioimage.modelrunner.engine.EngineLoader.setEngineInstance(EngineLoader.java:330)
	at io.bioimage.modelrunner.engine.EngineLoader.<init>(EngineLoader.java:138)
	at io.bioimage.modelrunner.engine.EngineLoader.createEngine(EngineLoader.java:159)
	at io.bioimage.modelrunner.model.Model.setEngineClassLoader(Model.java:480)
	at io.bioimage.modelrunner.model.Model.<init>(Model.java:142)
	at io.bioimage.modelrunner.model.Model.createBioimageioModel(Model.java:319)
	at io.bioimage.modelrunner.model.Model.createBioimageioModel(Model.java:258)
	at io.bioimage.modelrunner.model.Model.createBioimageioModel(Model.java:236)
	at io.bioimage.modelrunner.model.Stardist3D.predict(Stardist3D.java:245)
	at main(Stardist3DInstallRequirements.java:28)

This behavior is unexpected. It happens on Windows machine with JDK-21 (not sure, if this OS related or Java-version related).

[stefanhahmann]

@carlosuc3m any ideas how to solve this? I tried investigating a bit, but did not get very far.

I am surprised a little bit that the above downloads dl-modelrunner-tensorflow-1-0.4.2-20250103.161616-23.jar instead of the latest release https://maven.scijava.org/service/local/artifact/maven/redirect?r=releases&g=io.bioimage&a=dl-modelrunner-tensorflow-1&v=0.4.1&e=jar

[stefanhahmann]

I found out that with the 0.5.8 release, when dl-modelrunner-tensorflow-1-0.4.1.jar was used the above mentioned example actually works. After that, it seems to be broken.

[carlosuc3m]

HEllo @stefanhahmann

First of all sorry for having forgotten about this topic for so long. Thanks to your reminder i am back on it.

What are you using to run this code snippet? I have just tried with the latest snapshot and it seems to work.

I am surprised a little bit that the above downloads dl-modelrunner-tensorflow-1-0.4.2-20250103.161616-23.jar instead of the latest release https://maven.scijava.org/service/local/artifact/maven/redirect?r=releases&g=io.bioimage&a=dl-modelrunner-tensorflow-1&v=0.4.1&e=jar

Totally normal to be surprised. I have just changed many many things on JDLL, so I ma keeping everything on SNAPSHOTS at the moment (hopefully for not too long).

[stefanhahmann]

HEllo @stefanhahmann

First of all sorry for having forgotten about this topic for so long. Thanks to your reminder i am back on it.

No worries. I also I did not work on it in the mean time.

What are you using to run this code snippet? I have just tried with the latest snapshot and it seems to work.

I am running this snippet from the IDE (IntelliJ in my case). The snippet misses a package on the first line. But there could be any package.

Totally normal to be surprised. I have just changed many many things on JDLL, so I ma keeping everything on SNAPSHOTS at the moment (hopefully for not too long).

This is also my interpretation. I think the InvocationTargetException may have the reason that something with Reflection goes wrong. E.g. a constructor that may need some parameters but is not provided any parameter when invoked using reflection. But I was not able to trace this down.

[carlosuc3m]

I am running this snippet from the IDE (IntelliJ in my case).

Is the snippet using dl-modelrunner as a dependency? Or did you directly pull it from JDLL github and are running the example in the Stardist3D class?

[stefanhahmann]

I am running this snippet from the IDE (IntelliJ in my case).

Is the snippet using dl-modelrunner as a dependency? Or did you directly pull it from JDLL github and are running the example in the Stardist3D class?

I have checked out JDLL locally and set it to this commit db3c3f7

When I do this, the snippet is working.

When I set JDLL to the current main, it is not working.

[carlosuc3m]

I have just added a commit for debugging. Could you please pull, try again and post the new error?

[stefanhahmann]

I have just added a commit for debugging. Could you please pull, try again and post the new error?

I have just pulled. Now I am getting a different error message:

[SERVICE-0] {"task": "1381c64b-21b1-4dad-880f-b4968d0ee182", "responseType": "UPDATE", "message": "extra module imported: 1.3110008239746094"}
[SERVICE-0] {"task": "1381c64b-21b1-4dad-880f-b4968d0ee182", "responseType": "UPDATE", "message": "Imports"}
[SERVICE-0] {"task": "1381c64b-21b1-4dad-880f-b4968d0ee182", "responseType": "UPDATE", "message": "method"}
[SERVICE-0] <INVALID> warning: empty list of points (returning background-only image)
[SERVICE-0] {"task": "1381c64b-21b1-4dad-880f-b4968d0ee182", "responseType": "UPDATE", "message": "Preparing outputs"}
[SERVICE-0] {"task": "1381c64b-21b1-4dad-880f-b4968d0ee182", "responseType": "COMPLETION", "outputs": {"output0": {"data": "wnsm_a8853886", "shape": [10, 12, 12], "appose_data_type__15012025_132609": "np_arr", "is_fortran": false, "dtype": "float32"}}}
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Error retrieving the image from Python
	at io.bioimage.modelrunner.runmode.RunMode.runOP(RunMode.java:176)
	at io.bioimage.modelrunner.model.Stardist3D.postProcessing(Stardist3D.java:195)
	at io.bioimage.modelrunner.model.Stardist3D.predict(Stardist3D.java:174)
	at net.stefanhahmann.stardist3d.jdll.StarDist3DDummyPrediction.main(StarDist3DDummyPrediction.java:33)
Caused by: java.lang.RuntimeException: Error retrieving the image from Python
	at io.bioimage.modelrunner.runmode.RunMode.createImgLib2ArrFromApposeOutput(RunMode.java:511)
	at io.bioimage.modelrunner.runmode.RunMode.recreateOutputObjects(RunMode.java:209)
	at io.bioimage.modelrunner.runmode.RunMode.runOP(RunMode.java:165)
	... 3 more
Caused by: java.nio.file.FileAlreadyExistsException: Shared memory segment already exists with different dimensions, data type or format. Size of existing shared memory segment: 8192, size of proposed object: 23040
	at io.bioimage.modelrunner.tensor.shm.SharedMemoryArrayWin.<init>(SharedMemoryArrayWin.java:224)
	at io.bioimage.modelrunner.tensor.shm.SharedMemoryArrayWin.readOrCreate(SharedMemoryArrayWin.java:337)
	at io.bioimage.modelrunner.tensor.shm.SharedMemoryArray.readOrCreate(SharedMemoryArray.java:189)
	at io.bioimage.modelrunner.runmode.RunMode.createImgLib2ArrFromApposeOutput(RunMode.java:508)
	... 5 more

[carlosuc3m]

Hello @stefanhahmann
It seems that Python and stardist has been run. The error might probably be because of the size of the volume.
I have tried with the following size and it works
RandomAccessibleInterval<FloatType> img = ArrayImgs.floats(new long[] {116, 120, 66})
Could you please try it?

Just so you know I am completely re-writing the API. As mentioned on the zulip thread I am trying first to create a Java wrapper for the Stardist original API, but if it is not worth it or too complex (too complex meaning that it takes more than today and tomorrow) I will just call everything in Python.
After the changes, you should be able to run a model directly without converting into the Bioimage.io format.

Does it suit you?

[stefanhahmann]

Hello @stefanhahmann It seems that Python and stardist has been run. The error might probably be because of the size of the volume. I have tried with the following size and it works RandomAccessibleInterval<FloatType> img = ArrayImgs.floats(new long[] {116, 120, 66}) Could you please try it?

I tried, but now I am getting again errors due to long classpath:

java.io.IOException: Cannot run program ""C:\Program Files\Java\jdk-21\bin\java"" (in directory "."): CreateProcess error=206, Der Dateiname oder die Erweiterung ist zu lang
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1170)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1089)
	at io.bioimage.modelrunner.apposed.appose.Service.start(Service.java:149)
	at io.bioimage.modelrunner.apposed.appose.Service.task(Service.java:180)
	at io.bioimage.modelrunner.tensorflow.v1.Tensorflow1Interface.launchModelLoadOnProcess(Tensorflow1Interface.java:218)
	at io.bioimage.modelrunner.tensorflow.v1.Tensorflow1Interface.loadModel(Tensorflow1Interface.java:192)
	at io.bioimage.modelrunner.model.Model.loadModel(Model.java:495)
	at io.bioimage.modelrunner.model.Stardist3D.predict(Stardist3D.java:247)
	at net.stefanhahmann.stardist3d.jdll.StarDist3DDummyPrediction.main(StarDist3DDummyPrediction.java:33)
Caused by: java.io.IOException: CreateProcess error=206, Der Dateiname oder die Erweiterung ist zu lang
	at java.base/java.lang.ProcessImpl.create(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:500)
	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:159)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1126)
	... 8 more

	at io.bioimage.modelrunner.tensorflow.v1.Tensorflow1Interface.loadModel(Tensorflow1Interface.java:194)
	at io.bioimage.modelrunner.model.Model.loadModel(Model.java:495)
	at io.bioimage.modelrunner.model.Stardist3D.predict(Stardist3D.java:247)
	at net.stefanhahmann.stardist3d.jdll.StarDist3DDummyPrediction.main(StarDist3DDummyPrediction.java:33)

This migh be a Windows specific problem. I am trying to workaround adding

private static String[] fixArgs( final String[] args )
	{
		try
		{
			String[] newArgs = new String[ args.length - 1 ];
			File classPathFile = Files.createTempFile( "classpath", "" ).toFile();
			classPathFile.deleteOnExit();
			String arguments = "-cp ";
			arguments += "\"";
			arguments += args[ 2 ];
			arguments += "\"";
			Files.write( classPathFile.toPath(), arguments.getBytes( StandardCharsets.UTF_8 ) );
			newArgs[ 0 ] = args[ 0 ].replaceAll( "\"", "" ) + " ";
			newArgs[ 1 ] = "@" + classPathFile.getAbsolutePath();
			newArgs[ 2 ] = args[ 3 ];
			return newArgs;
		}
		catch ( IOException e )
		{
			throw new RuntimeException( e );
		}
	}

to the Service class and the changing the Constructor to

public Service(File cwd, String... args) {
		this.cwd = cwd;
		this.args = fixArgs(args.clone());
		serviceID = serviceCount++;
	}

But, then I get

even though the class is on the classpath..

Just so you know I am completely re-writing the API. As mentioned on the zulip thread I am trying first to create a Java wrapper for the Stardist original API, but if it is not worth it or too complex (too complex meaning that it takes more than today and tomorrow) I will just call everything in Python. After the changes, you should be able to run a model directly without converting into the Bioimage.io format.

Does it suit you?

That sounds good. My feeling at the moment is that running everything in python and getting it into Java using https://github.com/imglib/imglib2-appose seems most promising as this would allow testing everything on the python site and once this is working connecting it to java, but this is just a gut feeling.

[carlosuc3m]

Hello @stefanhahmann I have completely reworked the Stardist api in JDLL.
Please can you try it?
Now it should mimic the Python one, you can proide models that are not in the bioimage.io format

REgards,
Carlos