Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Mitochondria TOM20 artificial labelling - pix2pix #700

Merged
merged 3 commits into from
Jan 23, 2024

Conversation

bioimageiobot
Copy link
Collaborator

This is an automatic PR created by the @bioimageiobot regarding changes to the resource item 10.5281/zenodo.10366411.
The following version(s) will be added:

Please review the changes and make sure the new item or version(s) pass the following check list:

  • Passed the bioimage.io CI tests: static (and dynamic) validations
  • The meta information for the RDF item is complete
    • The tags are complete and describe the model
    • Naming is intuitive and descriptive, example: Multi-Organ Nucleus Segmentation (StarDist 2D)
    • Authors are provided
    • Documentation is complete
      • For models, include an overview, describe how the model is trained, what is the training data, how to use the model, how to validate the results and list the references. TODO: Model documentation template.
  • Approved by at least one of the bioimage.io admin team member.

Maintainers: @esgomezm

Note: If you updated or re-uploaded another version for the current item on Zenodo, this PR won't be changed automatically. To proceed, you can do the following:

  1. Block this version, but keep looking for future versions: Edit the current resource.yaml and keep the top-level status field as accepted, but change the status under the current version to blocked.
  2. Accept this version and keep looking for future versions: Merge this PR for now.
  3. Keep proposed version(s) (and this resource in general if it is new) as pending: Close this PR without merging.

Then wait for the CI on the main branch to complete. It should detect the new version(s) and create another PR for the new version(s).

Previous PRs of this resource: #695

@FynnBe
Copy link
Member

FynnBe commented Jan 12, 2024

@esgomezm why does the zenodo record also include the packaged bioimage-model.zip?
Looks like it contains the same files as are in the record. This file duplication makes it unclear which ones were tested, etc..
I'm going to ignore the bioimageio-model.zip as our convention is to upload the files individually to zenodo to allow selective download.

back to the real issue:

The error suggests that it is a pytorch bug:

terminate called after throwing an instance of 'c10::Error'
  what():  isTuple()INTERNAL ASSERT FAILED at "/opt/conda/conda-bld/pytorch_1640811723911/work/aten/src/ATen/core/ivalue_inl.h":1400, please report a bug to PyTorch. Expected Tuple but got String
Exception raised from toTuple at /opt/conda/conda-bld/pytorch_1640811723911/work/aten/src/ATen/core/ivalue_inl.h:1400
full conda env used in CI
    _libgcc_mutex           0.1           conda_forge            conda-forge
    _openmp_mutex           4.5           2_kmp_llvm             conda-forge
    aom                     3.7.1         h59595ed_0             conda-forge
    bioimageio.core         0.5.11        pyhd8ed1ab_0           conda-forge
    bioimageio.spec         0.4.9.post5   pyhd8ed1ab_0           conda-forge
    blas                    2.120         mkl                    conda-forge
    blas-devel              3.9.0         20_linux64_mkl         conda-forge
    blosc                   1.21.5        h0f2a231_0             conda-forge
    brotli                  1.1.0         hd590300_1             conda-forge
    brotli-bin              1.1.0         hd590300_1             conda-forge
    brotli-python           1.1.0         py39h3d6467e_1         conda-forge
    brunsli                 0.1           h9c3ff4c_0             conda-forge
    bzip2                   1.0.8         hd590300_5             conda-forge
    c-blosc2                2.12.0        hb4ffafa_0             conda-forge
    ca-certificates         2023.11.17    hbcca054_0             conda-forge
    certifi                 2023.11.17    pyhd8ed1ab_0           conda-forge
    charls                  2.4.2         h59595ed_0             conda-forge
    charset-normalizer      3.3.2         pyhd8ed1ab_0           conda-forge
    click                   8.1.7         unix_pyh707e725_0      conda-forge
    colorama                0.4.6         pyhd8ed1ab_0           conda-forge
    cpuonly                 2.0           0                      pytorch    
    dav1d                   1.2.1         hd590300_0             conda-forge
    freetype                2.12.1        h267a509_2             conda-forge
    giflib                  5.2.1         h0b41bf4_3             conda-forge
    icu                     73.2          h59595ed_0             conda-forge
    idna                    3.6           pyhd8ed1ab_0           conda-forge
    imagecodecs             2024.1.1      py39hf9b8f0e_0         conda-forge
    imageio                 2.33.1        pyh8c1a49c_0           conda-forge
    jxrlib                  1.1           hd590300_3             conda-forge
    lcms2                   2.16          hb7c19ff_0             conda-forge
    ld_impl_linux-64        2.40          h41732ed_0             conda-forge
    lerc                    4.0.0         h27087fc_0             conda-forge
    libaec                  1.1.2         h59595ed_1             conda-forge
    libavif16               1.0.3         hef5bec9_1             conda-forge
    libblas                 3.9.0         20_linux64_mkl         conda-forge
    libbrotlicommon         1.1.0         hd590300_1             conda-forge
    libbrotlidec            1.1.0         hd590300_1             conda-forge
    libbrotlienc            1.1.0         hd590300_1             conda-forge
    libcblas                3.9.0         20_linux64_mkl         conda-forge
    libdeflate              1.19          hd590300_0             conda-forge
    libffi                  3.4.2         h7f98852_5             conda-forge
    libgcc-ng               13.2.0        h807b86a_3             conda-forge
    libgfortran-ng          13.2.0        h69a702a_3             conda-forge
    libgfortran5            13.2.0        ha4646dd_3             conda-forge
    libhwloc                2.9.3         default_h554bfaf_1009  conda-forge
    libiconv                1.17          hd590300_2             conda-forge
    libjpeg-turbo           3.0.0         hd590300_1             conda-forge
    liblapack               3.9.0         20_linux64_mkl         conda-forge
    liblapacke              3.9.0         20_linux64_mkl         conda-forge
    libnsl                  2.0.1         hd590300_0             conda-forge
    libpng                  1.6.39        h753d276_0             conda-forge
    libsqlite               3.44.2        h2797004_0             conda-forge
    libstdcxx-ng            13.2.0        h7e041cc_3             conda-forge
    libtiff                 4.6.0         ha9c0a0a_2             conda-forge
    libuuid                 2.38.1        h0b41bf4_0             conda-forge
    libuv                   1.46.0        hd590300_0             conda-forge
    libwebp-base            1.3.2         hd590300_0             conda-forge
    libxcb                  1.15          h0b41bf4_0             conda-forge
    libxcrypt               4.4.36        hd590300_1             conda-forge
    libxml2                 2.11.6        h232c23b_0             conda-forge
    libzlib                 1.2.13        hd590300_5             conda-forge
    libzopfli               1.0.3         h9c3ff4c_0             conda-forge
    llvm-openmp             17.0.6        h4dfa4b3_0             conda-forge
    lz4-c                   1.9.4         hcb278e6_0             conda-forge
    markdown-it-py          3.0.0         pyhd8ed1ab_0           conda-forge
    marshmallow             3.20.2        pyhd8ed1ab_0           conda-forge
    marshmallow-jsonschema  0.13.0        pyhd8ed1ab_0           conda-forge
    marshmallow-union       0.1.15.post1  pyhd8ed1ab_0           conda-forge
    mdurl                   0.1.2         pyhd8ed1ab_0           conda-forge
    mkl                     2023.2.0      h84fe81f_50496         conda-forge
    mkl-devel               2023.2.0      ha770c72_50496         conda-forge
    mkl-include             2023.2.0      h84fe81f_50496         conda-forge
    ncurses                 6.4           h59595ed_2             conda-forge
    numpy                   1.26.3        py39h474f0d3_0         conda-forge
    openjpeg                2.5.0         h488ebb8_3             conda-forge
    openssl                 3.2.0         hd590300_1             conda-forge
    packaging               23.2          pyhd8ed1ab_0           conda-forge
    pandas                  2.1.4         py39hddac248_0         conda-forge
    pillow                  10.2.0        py39had0adad_0         conda-forge
    pip                     23.3.2        pyhd8ed1ab_0           conda-forge
    pthread-stubs           0.4           h36c2ea0_1001          conda-forge
    pygments                2.17.2        pyhd8ed1ab_0           conda-forge
    pysocks                 1.7.1         pyha2e5f31_6           conda-forge
    python                  3.9.18        h0755675_1_cpython     conda-forge
    python-dateutil         2.8.2         pyhd8ed1ab_0           conda-forge
    python-stdnum           1.19          pyhd8ed1ab_0           conda-forge
    python-tzdata           2023.4        pyhd8ed1ab_0           conda-forge
    python_abi              3.9           4_cp39                 conda-forge
    pytorch                 1.10.2        py3.9_cpu_0            pytorch    
    pytorch-mutex           1.0           cpu                    pytorch    
    pytz                    2023.3.post1  pyhd8ed1ab_0           conda-forge
    pyyaml                  6.0.1         py39hd1e30aa_1         conda-forge
    rav1e                   0.6.6         he8a937b_2             conda-forge
    readline                8.2           h8228510_1             conda-forge
    requests                2.31.0        pyhd8ed1ab_0           conda-forge
    rich                    13.7.0        pyhd8ed1ab_0           conda-forge
    ruamel.yaml             0.18.5        py39hd1e30aa_0         conda-forge
    ruamel.yaml.clib        0.2.7         py39hd1e30aa_2         conda-forge
    setuptools              69.0.3        pyhd8ed1ab_0           conda-forge
    shellingham             1.5.4         pyhd8ed1ab_0           conda-forge
    six                     1.16.0        pyh6c4a22f_0           conda-forge
    snappy                  1.1.10        h9fff704_0             conda-forge
    svt-av1                 1.8.0         h59595ed_0             conda-forge
    tbb                     2021.11.0     h00ab1b0_0             conda-forge
    tifffile                2023.12.9     pyhd8ed1ab_0           conda-forge
    tk                      8.6.13        noxft_h4845f30_101     conda-forge
    tqdm                    4.66.1        pyhd8ed1ab_0           conda-forge
    typer                   0.9.0         pyhd8ed1ab_0           conda-forge
    typing-extensions       4.9.0         hd8ed1ab_0             conda-forge
    typing_extensions       4.9.0         pyha770c72_0           conda-forge
    tzdata                  2023d         h0c530f3_0             conda-forge
    urllib3                 2.1.0         pyhd8ed1ab_0           conda-forge
    wheel                   0.42.0        pyhd8ed1ab_0           conda-forge
    xarray                  2023.12.0     pyhd8ed1ab_0           conda-forge
    xorg-libxau             1.0.11        hd590300_0             conda-forge
    xorg-libxdmcp           1.1.3         h7f98852_0             conda-forge
    xz                      5.2.6         h166bdaf_0             conda-forge
    yaml                    0.2.5         h7f98852_2             conda-forge
    zfp                     1.0.1         h59595ed_0             conda-forge
    zlib-ng                 2.0.7         h0b41bf4_0             conda-forge
    zstd                    1.5.5         hfc55251_0             conda-forge

@esgomezm could you please post your working, local environment here for comparison? (mamba list -r)

@esgomezm
Copy link
Contributor

Hi hi,
The zip file contains the same files that are individually uploaded. It's a silly duplication but it is a way for me to have a plan B in any course whenever the bioengine or the zoo stops working.

Here it goes:

Mamba env
(bioimageio-core) esti@estimacbookair ~ % mamba list -r

2024-01-11 14:29:43  (rev 0)
    +ca-certificates-2023.11.17 (conda-forge/osx-arm64)
    +libcxx-16.0.6 (conda-forge/osx-arm64)
    +libffi-3.3 (conda-forge/osx-arm64)
    +libsqlite-3.44.2 (conda-forge/osx-arm64)
    +libzlib-1.2.13 (conda-forge/osx-arm64)
    +ncurses-6.4 (conda-forge/osx-arm64)
    +openssl-1.1.1w (conda-forge/osx-arm64)
    +pip-23.3.2 (conda-forge/noarch)
    +python-3.9.0 (conda-forge/osx-arm64)
    +readline-8.2 (conda-forge/osx-arm64)
    +setuptools-69.0.3 (conda-forge/noarch)
    +sqlite-3.44.2 (conda-forge/osx-arm64)
    +tk-8.6.13 (conda-forge/osx-arm64)
    +tzdata-2023d (conda-forge/noarch)
    +wheel-0.42.0 (conda-forge/noarch)
    +xz-5.2.6 (conda-forge/osx-arm64)
    +zlib-1.2.13 (conda-forge/osx-arm64)

2024-01-11 14:30:41  (rev 1)
    +aom-3.7.1 (conda-forge/osx-arm64)
    +bioimageio.core-0.5.11 (conda-forge/noarch)
    +bioimageio.spec-0.4.9.post5 (conda-forge/noarch)
    +blosc-1.21.5 (conda-forge/osx-arm64)
    +brotli-1.1.0 (conda-forge/osx-arm64)
    +brotli-bin-1.1.0 (conda-forge/osx-arm64)
    +brotli-python-1.1.0 (conda-forge/osx-arm64)
    +brunsli-0.1 (conda-forge/osx-arm64)
    +bzip2-1.0.8 (conda-forge/osx-arm64)
    +c-blosc2-2.12.0 (conda-forge/osx-arm64)
    +certifi-2023.11.17 (conda-forge/noarch)
    +charls-2.4.2 (conda-forge/osx-arm64)
    +charset-normalizer-3.3.2 (conda-forge/noarch)
    +click-8.1.7 (conda-forge/noarch)
    +colorama-0.4.6 (conda-forge/noarch)
    +cpuonly-2.0 (pytorch/noarch)
    +dav1d-1.2.1 (conda-forge/osx-arm64)
    +filelock-3.13.1 (conda-forge/noarch)
    +freetype-2.12.1 (conda-forge/osx-arm64)
    +fsspec-2023.12.2 (conda-forge/noarch)
    +giflib-5.2.1 (conda-forge/osx-arm64)
    +gmp-6.3.0 (conda-forge/osx-arm64)
    +gmpy2-2.1.2 (conda-forge/osx-arm64)
    +idna-3.6 (conda-forge/noarch)
    +imagecodecs-2024.1.1 (conda-forge/osx-arm64)
    +imageio-2.33.1 (conda-forge/noarch)
    +jinja2-3.1.3 (conda-forge/noarch)
    +jxrlib-1.1 (conda-forge/osx-arm64)
    +lcms2-2.16 (conda-forge/osx-arm64)
    +lerc-4.0.0 (conda-forge/osx-arm64)
    +libabseil-20230802.1 (conda-forge/osx-arm64)
    +libaec-1.1.2 (conda-forge/osx-arm64)
    +libavif16-1.0.3 (conda-forge/osx-arm64)
    +libblas-3.9.0 (conda-forge/osx-arm64)
    +libbrotlicommon-1.1.0 (conda-forge/osx-arm64)
    +libbrotlidec-1.1.0 (conda-forge/osx-arm64)
    +libbrotlienc-1.1.0 (conda-forge/osx-arm64)
    +libcblas-3.9.0 (conda-forge/osx-arm64)
    +libdeflate-1.19 (conda-forge/osx-arm64)
    +libgfortran-5.0.0 (conda-forge/osx-arm64)
    +libgfortran5-13.2.0 (conda-forge/osx-arm64)
    +libjpeg-turbo-3.0.0 (conda-forge/osx-arm64)
    +liblapack-3.9.0 (conda-forge/osx-arm64)
    +libopenblas-0.3.25 (conda-forge/osx-arm64)
    +libpng-1.6.39 (conda-forge/osx-arm64)
    +libprotobuf-4.24.4 (conda-forge/osx-arm64)
    +libtiff-4.6.0 (conda-forge/osx-arm64)
    +libuv-1.46.0 (conda-forge/osx-arm64)
    +libwebp-base-1.3.2 (conda-forge/osx-arm64)
    +libxcb-1.15 (conda-forge/osx-arm64)
    +libzopfli-1.0.3 (conda-forge/osx-arm64)
    +llvm-openmp-17.0.6 (conda-forge/osx-arm64)
    +lz4-c-1.9.4 (conda-forge/osx-arm64)
    +markdown-it-py-3.0.0 (conda-forge/noarch)
    +markupsafe-2.1.3 (conda-forge/osx-arm64)
    +marshmallow-3.20.2 (conda-forge/noarch)
    +marshmallow-jsonschema-0.13.0 (conda-forge/noarch)
    +marshmallow-union-0.1.15.post1 (conda-forge/noarch)
    +mdurl-0.1.2 (conda-forge/noarch)
    +mpc-1.3.1 (conda-forge/osx-arm64)
    +mpfr-4.2.1 (conda-forge/osx-arm64)
    +mpmath-1.3.0 (conda-forge/noarch)
    +networkx-3.2.1 (conda-forge/noarch)
    +nomkl-1.0 (conda-forge/noarch)
    +numpy-1.26.3 (conda-forge/osx-arm64)
    +openjpeg-2.5.0 (conda-forge/osx-arm64)
    +packaging-23.2 (conda-forge/noarch)
    +pandas-2.1.4 (conda-forge/osx-arm64)
    +pillow-10.2.0 (conda-forge/osx-arm64)
    +pthread-stubs-0.4 (conda-forge/osx-arm64)
    +pygments-2.17.2 (conda-forge/noarch)
    +pysocks-1.7.1 (conda-forge/noarch)
    +python-dateutil-2.8.2 (conda-forge/noarch)
    +python-stdnum-1.19 (conda-forge/noarch)
    +python-tzdata-2023.4 (conda-forge/noarch)
    +python_abi-3.9 (conda-forge/osx-arm64)
    +pytorch-2.1.0 (conda-forge/osx-arm64)
    +pytorch-mutex-1.0 (pytorch/noarch)
    +pytz-2023.3.post1 (conda-forge/noarch)
    +pyyaml-6.0.1 (conda-forge/osx-arm64)
    +rav1e-0.6.6 (conda-forge/osx-arm64)
    +requests-2.31.0 (conda-forge/noarch)
    +rich-13.7.0 (conda-forge/noarch)
    +ruamel.yaml-0.18.5 (conda-forge/osx-arm64)
    +ruamel.yaml.clib-0.2.7 (conda-forge/osx-arm64)
    +shellingham-1.5.4 (conda-forge/noarch)
    +six-1.16.0 (conda-forge/noarch)
    +sleef-3.5.1 (conda-forge/osx-arm64)
    +snappy-1.1.10 (conda-forge/osx-arm64)
    +svt-av1-1.8.0 (conda-forge/osx-arm64)
    +sympy-1.12 (conda-forge/noarch)
    +tifffile-2023.12.9 (conda-forge/noarch)
    +torchvision-0.16.1 (conda-forge/osx-arm64)
    +tqdm-4.66.1 (conda-forge/noarch)
    +typer-0.9.0 (conda-forge/noarch)
    +typing-extensions-4.9.0 (conda-forge/noarch)
    +typing_extensions-4.9.0 (conda-forge/noarch)
    +urllib3-2.1.0 (conda-forge/noarch)
    +xarray-2023.12.0 (conda-forge/noarch)
    +xorg-libxau-1.0.11 (conda-forge/osx-arm64)
    +xorg-libxdmcp-1.1.3 (conda-forge/osx-arm64)
    +yaml-0.2.5 (conda-forge/osx-arm64)
    +zfp-1.0.1 (conda-forge/osx-arm64)
    +zlib-ng-2.0.7 (conda-forge/osx-arm64)
    +zstd-1.5.5 (conda-forge/osx-arm64)

@FynnBe
Copy link
Member

FynnBe commented Jan 23, 2024

ah, well. A trivial issue!

The collection's default pytorch version is 1.10 (which was resolved to 1.10.2), but you have 2.1.0 in your env...
The default has been used as pytorch_version is not specified in the rdf.yaml. (We don't have a default in spec, but currently do allow to not specify the pytorch version. So for practical purposes the collection repo had to choose a default).

The next minor bioimageio.spec release will make pytorch_version a mandatory field which will prevent this issue in the future.

@FynnBe
Copy link
Member

FynnBe commented Jan 23, 2024

thanks for sharing the env @esgomezm . I'll patch the rdf.yaml here in the collection repo...

@esgomezm
Copy link
Contributor

Hi @FynnBe!
Ok, then I'll update also the export to specify the torch version. I thought that the library was automatically testing what's the installed version. Now I see that there's a different error :D

@FynnBe
Copy link
Member

FynnBe commented Jan 23, 2024

Hi @FynnBe! Ok, then I'll update also the export to specify the torch version. I thought that the library was automatically testing what's the installed version. Now I see that there's a different error :D

my bad, I messed up the YAML syntax of the patch...

@esgomezm
Copy link
Contributor

I'm excited about seeing this one becoming green! I think this will never change xD
Screenshot 2024-01-23 at 10 45 59

@bioimageiobot
Copy link
Collaborator Author

preview
preview-collection-json

@esgomezm esgomezm merged commit 0892cca into main Jan 23, 2024
8 checks passed
@esgomezm esgomezm deleted the auto-update-10.5281/zenodo.10366411 branch January 23, 2024 10:51
@esgomezm
Copy link
Contributor

passed!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants