Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Waits for terminal stack status when summarizing #284

Merged
merged 1 commit into from
Jun 21, 2024

Conversation

Kazzer
Copy link
Contributor

@Kazzer Kazzer commented Jun 21, 2024

For some reason, we are getting non-terminal statuses when summarizing. It's possible this is a timing issue, so we'll naïvely attempt to get the stack description again until we receive a response with a terminal status.

By the time we make this call, we already know that we'll be in a terminal status because the watcher will only stop once we've seen a terminal status. This is why the logging will always have something like:

09:46:01 GMT-07:00   Thu Jun 20 2024 16:45:57 UPDATE_COMPLETE                              AWS::IAM::Role                           
09:46:01 GMT-07:00                                                                            CloudwatchLogRole (17s)
09:46:01 GMT-07:00  - 39 seconds elapsed total. 21 since last event.
09:46:06 GMT-07:00   Thu Jun 20 2024 16:46:00 UPDATE_COMPLETE_CLEANUP_IN_PROGRESS          AWS::CloudFormation::Stack               
09:46:06 GMT-07:00                                                                            eks-etc-titan-production
09:46:06 GMT-07:00   Thu Jun 20 2024 16:46:02 UPDATE_COMPLETE                              AWS::CloudFormation::Stack               
09:46:06 GMT-07:00                                                                            eks-etc-titan-production (36s)

where the last line will be UPDATE_COMPLETE, but iidy will summarize with

09:46:08 GMT-07:00  Current Stack Status:     UPDATE_COMPLETE_CLEANUP_IN_PROGRESS 

which is clearly not the status from the last line.

For some reason, iidy is getting an in progress state when summarizing stack contents. It's possible this is just due to timing.
When we know we're going to be in a terminal state, we can just wait for a terminal status to be returned instead of accepting the in progress status.
It's not clear why this specifically happens with `UPDATE_COMPLETE_CLEANUP_IN_PROGRESS`, but it could be a special case for timing.
@Kazzer Kazzer force-pushed the x-cleanup-in-progress branch from 31e872b to e4e1fe4 Compare June 21, 2024 21:15
Copy link
Contributor

@gpritchi-unbounce gpritchi-unbounce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍🏾

@gpritchi-unbounce
Copy link
Contributor

@tavisrudd Although it did resolve the immediate issue, there was some concern about considering UPDATE_COMPLETE_CLEANUP_IN_PROGRESS a final state, so we're actually going with this approach.

@Kazzer Kazzer merged commit cceeb4d into master Jun 21, 2024
1 check passed
@Kazzer Kazzer deleted the x-cleanup-in-progress branch July 8, 2024 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants