heroku container:release fails on version 10.0.0

[ptwobrussell]

Mysterious heroku container:release errors suddenly started happening this afternoon with v10.0.0 with the following kind of cryptic error message:

Run heroku container:release --app xxx web worker worker_b worker_c etc1 etc2 etc3
Releasing images web,worker,worker_b,worker_c,etc1,etc2,etc3 to xxx...
... 
done
Running release command...
    TypeError: Cannot read properties of undefined (reading 'statusCode')
Error: Process completed with exit code 1.

Fortunately, I was quick to realize that v10.0.0 had just been released a couple of hours ago, and I verified that my GitHub Actions was picking up that latest version via the recommended install script of curl https://cli-assets.heroku.com/install.sh | sh that pulls the latest

I noticed in the release notes that v10 requires Node 20, so I bumped my Node version from 18 to 20, but that didn't solve it. The error was the same with v10.0.0 and Node 20 per heroku --version returning heroku/10.0.0 linux-x64 node-v20.17.0 as output.

I then reverted back to v9.5.1 (via pinning with npm install -g [email protected]), and heroku container:release started working again as expected.

---

Unfortunately, I don't have much for you other than that. Just wanted to report this because it's not at all obvious to me why that command would be failing on 10.0.0 vs v9.5.1 based on what I see in the release notes for v10.0.0.

This seems like a regression, so I wanted it to be on your radar in hopes that it helps others.

[naslundx]

Can confirm the same issue occurs for me on 10.0.0.

Is there no way of installing the older release via the deb package in the mean time? Looks like only 10.0.0 is available:

$ apt-show-versions -a heroku
heroku:amd64 10.0.0.b084308-1 stable cli-assets.heroku.com
heroku:amd64 not installed
heroku:arm64 10.0.0.b084308-1 stable cli-assets.heroku.com
heroku:arm64 not installed
heroku:armel 10.0.0.b084308-1 stable cli-assets.heroku.com
heroku:armel not installed
$ 

[AndreasHaaversen]

We're getting the same error doing a heroku pg:copy in postdeploy script, so the error isn't just contained to this specific command. We're using the buildpack version.

[smshuja]

Facing the same issue and breaking in CI/CD pipeline

[chtzvt]

I can confirm that we're also experiencing this issue (using cli v10.0.0.b084308-1 in GitHub Actions on Ubuntu 22.04.5 LTS).

[eablack]

Thanks for the report. We're looking into it.

[eablack]

Hey @ptwobrussell, could you tell us how you are installing the Heroku CLI. Are you using homebrew or are you installing directly via npm install -g?

[eablack]

Noticing now in your output it lists heroku/10.0.0 linux-x64. Did you install via curl https://cli-assets.heroku.com/install.sh | sh?

[ptwobrussell]

@eablack - yes, my GitHub Actions had been using curl https://cli-assets.heroku.com/install.sh | sh up until this point per the docs at https://devcenter.heroku.com/articles/heroku-cli#standalone-installation-with-a-tarball

[julien51]

Same thing on our end!

[paltman]

Seeing the same thing. I'm using the release command in my Github Actions and though it ends the CI with a failure the release did complete successfully. And if I re-run the failed job, it will complete with success (releasing again).

@ptwobrussell I'm quite familiar with that part of the Github Actions in your setup. I happen to be doing the same. :)

[paltman]

9.5.1 looks to be having the same problem for me (redacted app name):

Run heroku container:release -a XXXXXX web release worker
 ›   Warning: heroku update available from 9.5.1 to 10.0.0.
Releasing images web,release,worker to XXXXXX...
Releasing images web,release,worker to XXXXXX... ⣾
Releasing images web,release,worker to XXXXXX... ⣽
Releasing images web,release,worker to XXXXXX... done
Running release command...
    TypeError: Cannot read properties of undefined (reading 'statusCode')

[paltman]

I've tried back to v9.5.0 which I know both v9.5.0 and v9.5.1 because I have deploys on those before v10.0.0 landed and am getting the same thing. I'm suspecting something is wonky with Heroku's API at this point in a way that this CLI doesn't anticipate.

I've gone back to latest with shell script and cobbled together a quick little shell script to filter out and ignore the statusCode error while letting the command fail for all other errors. It's a lot uglier than the one liner but I'm using this in a shared workflow across all my repositories so not a huge deal.

- name: Release
  # run: heroku container:release -a ${{ inputs.app-name }} ${{ inputs.processes }}
  run: |
    echo "Attempting release..."
    if ! heroku container:release -a ${{ inputs.app-name }} ${{ inputs.processes }}; then
      echo "Release command exited with non-zero status. Checking latest release status..."
      latest_release=$(heroku releases -a ${{ inputs.app-name }} -n 1)
      echo "Latest release info:"
      echo "$latest_release"
      if [[ "$latest_release" =~ "Deployed" ]]; then
        echo "Latest release shows success, overriding error"
        exit 0
      else
        echo "Latest release does not show success, failing"
        exit 1
      fi
  fi

[geronimod]

I can confirm, we're getting the same issue with the following:

O.S:
  Ubuntu
    24.04.1
    LTS
  Image: ubuntu-24.04
  Version: 20241208.1.0

Heroku cli:
  Instalation via: curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
  version installed: heroku_10.0.0.b084308-1_amd64.deb

[jlipponen]

In case someone needs to go back to 9.5.1 in a pipeline and is looking for a bash script for it, try this:

curl https://cli-assets.heroku.com/install-standalone.sh | sed 's#/channels/stable/heroku-#/versions/9.5.1/1aaf605/heroku-v9.5.1-1aaf605-#' | sh

[rlt]

Hi @eablack, could we get an update on this issue please?

[eablack]

Hi, we're still investigating this issue. We've determined the source (a somewhat gnarly incompatibility with node 20) and are working on a fix.

[mr-niels-christensen]

Happy New Year! @eablack Any updates on this?

I don't suppose anyone has a clever workaround that doesn't involve downgrading the CLI?

[florianweh]

Happy New Year! @eablack We are also waiting for updates. Downgrading does not work. We will need a fix or another workaround very soon!

[rlt]

In our case the command was actually succeeding but the exit code was 1, so I just used || true to ignore that

[k80bowman]

We just released v10.0.1 of the CLI which should hopefully fix this issue for you. I'm going to leave this issue open until we get confirmation from at least a few of the folks in this thread. Thank you all for your patience.

[mr-niels-christensen]

Thank you! 🙏 In our case, we'll know once the update is available in Github Action's Docker images, but I'll make sure to post here when we have a confirmation.

[florianweh]

Updated to v10.0.1 worked for us combined with some additional code changes.

[Yardboy]

I am no longer experiencing the problem after latest update.

[rlt]

Working for us too, thanks!

[k80bowman]

Wonderful! Thank you all for responding. I'm going to go ahead and close this now.