6.13.0 dependency updates

[melissalinkert]

See https://forum.image.sc/t/plugin-maintainers-can-you-test-fiji-2-11-0/78852

Comparing the list of updated dependencies in Fiji 2.11.0 to the current state of Bio-Formats, there are several things we'll need to evaluate:

• kryo 5.4.0 (Update kryo to 5.4.0 #3967)
• guava 31.1-jre (Bump guava version to 31.1-jre ome-common-java#75)
• version conflicts with minio dependencies
• conflicting commons-io version (dependency of cisd:jhdf5)
• version conflicts with edu.car:cdm-core dependencies
• ij 1.54c (Bump imagej version to 1.54c #3974, for compiling/testing bio-formats-plugins component)
• jackson 2.14.2 (Bump jackson dependencies to 2.14.2 ome-common-java#76)
• jaxb-runtime 2.3.5
• joda-time 2.12.2 (Bump joda-time to 2.12.2 ome-common-java#77, Remove unneeded joda-time dependency #3973, Remove unneeded joda-time dependency ome-model#170)
• json 20230227 (Bump json dependency to 20220924 #3966)
• logback 1.2.11 / slf4j 1.7.36 (Bump sl4j-api to 2.x #3956)
• snakeyaml 1.33 (Bump snakeyaml from 1.32 to 2.0 in /components/formats-bsd #3965)
• version conflicts with file-leak-detector (test-suite component only)

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/plugin-maintainers-can-you-test-fiji-2-11-0/78852/16

[sbesson]

Thanks for putting this list together Melissa. Immediately, I don't see anything overly concerning but as noted on the image.sc thread, we will need to go through each group of dependencies and test their upgrade. Overall, I am expecting the process will take a handful of weeks until we are in a position to release a version of Bio-Formats compatible with the pom-scijava changes.

On the existing dependency update PRs, two questions:

• Bump snakeyaml from 1.32 to 2.0 in /components/formats-bsd #3965 automatically bumps snakeyaml to 2.0 but it looks like scijava/pom-scijava@1de5965 chose to cap it to 1.33
• Bump sl4j-api to 2.x #3956 and the related PR proposes to upgrade slf4j-api 2.x which is a requirement to use the active logback 1.3.x (Java 8+) or logback 1.4.x (Java 11+) series.https://github.com/scijava/pom-scijava/blob/38baeabd34455b8075eec29ce7285e9e1c1034d1/pom.xml#L2209 still declares slf4j-api 1.7.x which is API incompatible

There are no associated code changes for both changes, so I assume the Bio-Formats code base will remain fully compatible with older versions of each dependencies. @ctrueden could you comment on the potential impact on the Fiji ecosystem if Bio-Formats were to depend on the latest major versions of both libraries? Additionally, do you anticipate both dependencies mentioned above to be considered for upgrades in the upcoming roadmap of the Scijava ecosystem ?

[dgault]

When looking into the jaxb-runtime update, Bio-Formats is currently using jaxb-api version 2.3.0. There doesn't seem to be any need to bump this to the runtime dependency and since jaxb-api is only used in a test scope it should have no issues for FIJI users. If for sanity sake we did want to update the API dependency to match then we would instead switch to jakarta.xml.bind-api version 2.3.3 as this is the matching version for jaxb-runtime 2.3.5.

[sbesson]

Summarizing the outcome of the weekly formats meeting, there has been good progress on the upgrade of the Bio-Formats dependencies to be compatible with the BOM of pom-scijava 34.1.0. Hopefully we should be in a position of releasing Bio-Formats 6.13.0 around mid April. There are a few outstanding questions re the integration with the Fiji ecosystem:

• the target version for the two dependencies mentioned in 6.13.0 dependency updates #3970 (comment). The proposed approach would be to bump snakeyaml to 2.0 (the code should remain compatible with snakeyaml 1.33) and cautiously keep slf4-api 1.x.
• the coordination of Bio-Formats deployment onto the Java-8 update site alongside the new Fiji distribution

Feedback on both points very welcome as always @ctrueden . In particular, it would be useful to know whether there is already a timeline for the deployment of Fiji 2.11.0 onto the core update site.

[jburel]

See comment for side effect #3973 (comment)

[ctrueden]

Hopefully we should be in a position of releasing Bio-Formats 6.13.0 around mid April.

Thanks everyone for your efforts. I appreciate it.

@ctrueden could you comment on the potential impact on the Fiji ecosystem if Bio-Formats were to depend on the latest major versions of both libraries?

My thoughts aren't super organized at the moment, but here goes.

The first requirement, of course, is for everything to work, and keep working. If that means waiting to upgrade a dependency, then that's OK. That's why I hesitated to upgrade snakeyaml to 2.0: because it will require code changes to some components.

That said, another softer requirement is to keep dependencies at the newest stable library release versions whenever possible, to gain security fixes and new features, hence the large number of dependency upgrades in pom-scijava 34.0.0 and 34.1.0. Right now, CI is not testing Bio-Formats for compatibility with BOM upgrades, because Bio-Formats's build system does not conform to the requirements for doing so. To add Bio-Formats to the pom-scijava melting pot test, Bio-Formats components would need to: A) extend pom-scijava; and B) use inherited dependency versions when possible (i.e. when the pom-scijava BOM manages them); and C) for dependencies not managed by pom-scijava, use version properties rather than direct <version>x.y.z</version> hardcoding in the POMs. If we did those things, then I could update this code block to also build Bio-Formats and other OME components at the versions currently declared in pom-scijava, catching dependency skew issues earlier as they emerge, rather than relying on manual testing.

About these particular component updates (snakeyaml and slf4j-api):

• It sounds like Bio-Formats still works with snakeyaml 1.33, so that's OK? But it is potential technical debt for BF to depend on 2.0 while pom-scijava is held back at 1.33; if Bio-Formats were to inadvertently start using 2.0+ features, thus breaking its usage in Fiji, we wouldn't know until manual test time. But I think that scenario is unlikely in practice, and I also hope pom-scijava can update to snakeyaml 2.0 after not too much longer.

• As for slf4j-api, IIRC, the reason I didn't upgrade to 2.x already is just for reasons of time due to the complexity of it. A big piece these though is that Bio-Formats depends on logback-classic, which I didn't immediately figure out how to make compatible with slf4j-api 2.x. If OME components are all upgrading away from that, it might actually enable the whole SciJava component collection to upgrade to slf4j 2.x in tandem with that, which would be great. I did some research recently on how to fix the logging behavior in various scenarios such as when running Fiji as a library, or via PyImageJ, including Bio-Formats messages. The presence (or not) of slf4j bindings from logback-classic is crucial here, so we'll need to tread carefully as we upgrade this logging dependency stack, but I'm happy to try it when you folks are.

the coordination of Bio-Formats deployment onto the Java-8 update site alongside the new Fiji distribution

Yes, this is a continual gotcha, that updates get uploaded manually independently from pom-scijava updates. I still want to get the point where it's automated, but to do that, the pom-scijava updates need to be really smooth. I think pom-scijava 34.x gets us a lot closer, but for the time being, the status quo of manual uploads is working and lets us release in an agile manner. And as long as the OME team (thanks again @dgault!) keeps filing PRs against pom-scijava when OME components update, that will at least prevent me from accidentally downgrading Bio-Formats when I upload big releases to the Java-8 update site.

Anything else you need my input on? Sorry for slow replies; there's always just so much going on. But I'm happy to schedule a video call or whatever to help move this along more quickly as needed.

[sbesson]

Thanks Curtis. Agreed there are several steps required to make Bio-Formats more tightly integrated with the Scijava ecosystem. Immediately, pom-scijava gives us the BOM which we can compare directly to the Bio-Formats dependency list and as you mentioned bumping the Bio-Formats components versions is part of our release workflow so we are likely in an acceptable place.

On the slf4j-api front, my understanding is that:

• logback 1.2.x requires JDK 8 as the minimal LTS version and depends on slf4j-api 1.x
• logback 1.3.x requires JDK 8 as the minimal LTS version and depends on slf4j-api 2.x
• logback 1.4.x requires JDK 11 as the minimal LTS version and depends on slf4j-api 2.xwe'll hold off for now as noted above given the potential breaking impact on logging but we'll watch out https://github.com/scijava/pom-scijava/blob/1e41f08061c975e802eea4067d694142eee52e59/pom.xml#L2121 for updates.

We are down to a few PRs to review but expect we will be in a position to release Bio-Formats 6.13.0 with the set of compatible dependencies next week. Is the image.sc post the reference place to track for the timeline to move Fiji 2.11.0 into production?

[ctrueden]

@sbesson Great, thanks again to you and the team for working on it! 👍

For the Fiji upload timeline, yes, this forum topic is the one to track. I haven't posted any updates for a couple of weeks because I've been doing other things—e.g. this week we are doing a small TrackMate hackathon—but the ball is in my court to resolve the problems folks have reported before we upload Fiji 2.11.0 to users. Since Bio-Formats 6.13.0 is close, I am hopeful that the timing will coincide well with my availability to work on it further. It is very likely that the path forward will be a Fiji 2.12.0 with another small set of dependency updates including BF 6.13.0, followed by upload of JAR files. I will definitely ping back here again with updates as things get closer.