v1.9.0 tarball checksum changed

[mabraham]

The macports port of arpeggio v1.9.0 is currently broken because the checksums computed when the port was made (see https://github.com/macports/macports-ports/blob/master/python/py-arpeggio/Portfile) don't match those computed for the current download:

:notice:fetch --->  Attempting to fetch Arpeggio-1.9.0.tar.gz from https://github.com/igordejanovic/Arpeggio/tarball/v1.9.0
:debug:fetch Privilege de-escalation not attempted as not running as root.
:debug:checksum checksum phase started at Thu Oct 24 13:20:38 CEST 2019
:notice:checksum --->  Verifying checksums for py37-arpeggio
:debug:checksum Executing org.macports.checksum (py37-arpeggio)
:info:checksum --->  Checksumming Arpeggio-1.9.0.tar.gz
:debug:checksum Calculated (rmd160) is 8576eb4fb7419fa0835e71d356bd145fcf3f1adb
:error:checksum Checksum (rmd160) mismatch for Arpeggio-1.9.0.tar.gz
:info:checksum Portfile checksum: Arpeggio-1.9.0.tar.gz rmd160 2e9823273ad9a382aaa42dbc6553e3892f7f25d5
:info:checksum Distfile checksum: Arpeggio-1.9.0.tar.gz rmd160 8576eb4fb7419fa0835e71d356bd145fcf3f1adb
:debug:checksum Calculated (sha256) is 3bd25e9b21a514b9a6384c0929f1caa710a42a417819dd2136b2c04d6cc62507
:error:checksum Checksum (sha256) mismatch for Arpeggio-1.9.0.tar.gz
:info:checksum Portfile checksum: Arpeggio-1.9.0.tar.gz sha256 fc6e1bd37fd1c5620f86c338cfad3bfd73217e46d79af831a62c22c3fc0b184a
:info:checksum Distfile checksum: Arpeggio-1.9.0.tar.gz sha256 3bd25e9b21a514b9a6384c0929f1caa710a42a417819dd2136b2c04d6cc62507
:debug:checksum Calculated (size) is 757459
:error:checksum Checksum (size) mismatch for Arpeggio-1.9.0.tar.gz
:info:checksum Portfile checksum: Arpeggio-1.9.0.tar.gz size 758169
:info:checksum Distfile checksum: Arpeggio-1.9.0.tar.gz size 757459
:info:checksum The correct checksum line may be:
:info:checksum checksums           rmd160  8576eb4fb7419fa0835e71d356bd145fcf3f1adb \
:info:checksum                     sha256  3bd25e9b21a514b9a6384c0929f1caa710a42a417819dd2136b2c04d6cc62507 \
:info:checksum                     size    757459
:error:checksum Failed to checksum py37-arpeggio: Unable to verify file checksums
:debug:checksum Error code: NONE

I downloaded it manually on two different systems and verified that the sha256sum matches that found by macports. So I conclude that the tarball was rebuilt somehow.

Please do not make "updates" to tarballs once computed, because downstream automation relies on them not changing. (Learned this myself once the hard way! :-))

I will submit a fix to macports

[igordejanovic]

Hi @mabraham. Thanks for the report. It was a surprising to me as I don't recall that I did any rebuild.
I increase the version number each time there is a change no matter how small. Actually, tarballs from github IIRC are built on-the-fly from tagged version so the only way this could happened is that tag was moved.

On a quick search it seems to be a problem with GitHub.
easybuilders/easybuild-easyconfigs#5151
bazel-contrib/rules_go#820

I guess that the best bet is to download releases from PyPI. Those tarballs are uploaded from my machine and the PyPI prohibits multiple uploads of the same version tarball.

[mabraham]

Thanks for the info. Another possibility is revealed by macports buildbot - that it is getting the download from https://distfiles.macports.org/py-arpeggio (see https://paste.macports.org/0480d2201c5c), which might be a tarball originally from PyPI. The git history for the portfile notes the maintainers intent to use a tarball that has the tests, but perhaps that was not correctly implemented / documented. Only the buildbots of old MacOS versions are failing, however. Will explore more tomorrow.

[igordejanovic]

The tests have become part of the source tarball PyPI release since v1.9.1, so that might be the underlying reason. Please see here

[mabraham]

I went to PyPI to investigate, and found that from https://pypi.org/project/Arpeggio/1.9.2/#files the source code link gives a 40K file from https://files.pythonhosted.org/packages/0e/a0/1fe16e650729c121af617d2038608b60359454e93e652f152a5c69abadf8/Arpeggio-1.9.2.tar.gz, but the download link on the left https://github.com/textX/Arpeggio/archive/v1.9.2.tar.gz comes from textX on github and is 748K (presumably because it has tests). Is that a problem to fix?

MacPorts downloads the small one in its testing, so I'll try to move forward and fix that there.

The larger one has some additional regression tests, docs, examples, and perf tests. Any idea why that would be, @igordejanovic?

[igordejanovic]

@mabraham The large one is created by GitHub (probably on-the-fly) from the whole source tree. The small one is what I created when the release has been made. So the small one should be used.