Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove .package-version file and its usage #6840

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
+15 −41
Draft

Remove .package-version file and its usage #6840

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions .buildkite/scripts/integration-tests.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,6 @@ $PSVersionTable.PSVersion
go install gotest.tools/gotestsum
gotestsum --version

# Read package version from .package-version file
$PACKAGE_VERSION = Get-Content .package-version -ErrorAction SilentlyContinue
if ($PACKAGE_VERSION) {
$PACKAGE_VERSION = "${PACKAGE_VERSION}-SNAPSHOT"
}
$env:TEST_BINARY_NAME = "elastic-agent"
# Parsing version.go. Will be simplified here: https://github.com/elastic/ingest-dev/issues/4925
# $AGENT_VERSION = (Get-Content version/version.go | Select-String -Pattern 'const defaultBeatVersion =' | ForEach-Object { $_ -replace '.*?"(.*?)".*', '$1' })
Expand Down Expand Up @@ -44,7 +39,7 @@ $outputXML = "build/${fully_qualified_group_name}.integration.xml"
$outputJSON = "build/${fully_qualified_group_name}.integration.out.json"
$TestsExitCode = 0
try {
Get-Ess-Stack -StackVersion $PACKAGE_VERSION
Get-Ess-Stack -StackVersion $AGENT_VERSION
Write-Output "~~~ Running integration test group: $GROUP_NAME as user: $env:USERNAME"
& gotestsum --no-color -f standard-quiet --junitfile "${outputXML}" --jsonfile "${outputJSON}" -- -tags=integration -shuffle=on -timeout=2h0m0s "github.com/elastic/elastic-agent/testing/integration" -v -args "-integration.groups=$GROUP_NAME" "-integration.sudo=$TEST_SUDO"
$TestsExitCode = $LASTEXITCODE
Expand Down
7 changes: 4 additions & 3 deletions .buildkite/scripts/steps/ess_start.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,11 @@ source .buildkite/scripts/common2.sh

source .buildkite/scripts/steps/ess.sh

OVERRIDE_STACK_VERSION="$(cat .package-version)"
OVERRIDE_STACK_VERSION=${OVERRIDE_STACK_VERSION}"-SNAPSHOT"
# Parsing version.go. Will be simplified here: https://github.com/elastic/ingest-dev/issues/4925
STACK_VERSION=$(grep "const defaultBeatVersion =" version/version.go | cut -d\" -f2)
STACK_VERSION="${STACK_VERSION}-SNAPSHOT"

ess_up $OVERRIDE_STACK_VERSION
ess_up $STACK_VERSION

echo "ES_HOST: ${ELASTICSEARCH_HOST}"
buildkite-agent meta-data set "es.host" $ELASTICSEARCH_HOST
Expand Down
11 changes: 2 additions & 9 deletions .buildkite/scripts/steps/integration_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,8 @@ STACK_PROVISIONER="${1:-"stateful"}"
MAGE_TARGET="${2:-"integration:test"}"
MAGE_SUBTARGET="${3:-""}"


# Override the stack version from `.package-version` contents
# There is a time when the current snapshot is not available on cloud yet, so we cannot use the latest version automatically
# This file is managed by an automation (mage integration:UpdateAgentPackageVersion) that check if the snapshot is ready.

STACK_VERSION="$(cat .package-version)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK reading everything this does, we actually want to keep this as a way to have a lagging update of the stack version. Since we will bump the agent version from N to N+1 but need to test against stack version N until the first snapshot for N+1 is produced.

Let's keep that part of this.

We should not need to change the actual agent version anymore, the fact that we do this without consequence is a hint we can just relax the version constraints for convenience where necessary. We already did this for minors, but didn't handle majors properly yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rename the file to .stack-version (and the associated mage target) and remove anything that sets the agent package version (which is the version agent reports itself as to the stack).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the .stack-version should also be auto-updated, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand it correctly that this PR doesn't make much sense?

if [[ -n "$STACK_VERSION" ]]; then
STACK_VERSION=${STACK_VERSION}"-SNAPSHOT"
fi
STACK_VERSION=$(grep "const defaultBeatVersion =" version/version.go | cut -d\" -f2)
STACK_VERSION="${STACK_VERSION}-SNAPSHOT"

# Run integration tests
set +e
Expand Down
6 changes: 3 additions & 3 deletions .buildkite/scripts/steps/integration_tests_tf.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ fi
# Override the agent package version using a string with format <major>.<minor>.<patch>
# There is a time when the snapshot is not built yet, so we cannot use the latest version automatically
# This file is managed by an automation (mage integration:UpdateAgentPackageVersion) that check if the snapshot is ready.
OVERRIDE_STACK_VERSION="$(cat .package-version)"
OVERRIDE_STACK_VERSION=${OVERRIDE_STACK_VERSION}"-SNAPSHOT"
STACK_VERSION=$(grep "const defaultBeatVersion =" version/version.go | cut -d\" -f2)
STACK_VERSION="${STACK_VERSION}-SNAPSHOT"

echo "~~~ Building test binaries"
mage build:testBinaries
Expand All @@ -35,7 +35,7 @@ mage build:testBinaries
if [[ "${BUILDKITE_RETRY_COUNT}" -gt 0 ]]; then
echo "~~~ The steps is retried, starting the ESS stack again"
trap 'ess_down' EXIT
ess_up $OVERRIDE_STACK_VERSION || echo "Failed to start ESS stack" >&2
ess_up $STACK_VERSION || echo "Failed to start ESS stack" >&2
else
# For the first run, we start the stack in the start_ess.sh step and it sets the meta-data
echo "~~~ Receiving ESS stack metadata"
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/bump-agent-versions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#!/bin/bash
set -e

package_version=$(mage integration:updatePackageVersion)
package_version=$(mage integration:GetPackageVersion)
version_requirements=$(mage integration:updateVersions)
changes=$(git status -s -uno testing/integration/testdata/.upgrade-test-agent-versions.yml .package-version)
changes=$(git status -s -uno testing/integration/testdata/.upgrade-test-agent-versions.yml)
if [ -z "$changes" ]
then
echo "The version files didn't change, skipping..."
Expand All @@ -22,7 +22,7 @@ else
# the mage target above requires to be on a release branch
# so, the new branch should not be created before the target is run
git checkout -b "$pr_branch"
git add testing/integration/testdata/.upgrade-test-agent-versions.yml .package-version
git add testing/integration/testdata/.upgrade-test-agent-versions.yml

nl=$'\n' # otherwise the new line character is not recognized properly
commit_desc="These files are used for picking the starting (pre-upgrade) or ending (post-upgrade) agent versions in upgrade integration tests.${nl}${nl}The content is based on responses from https://www.elastic.co/api/product_versions and https://snapshots.elastic.co${nl}${nl}The current update is generated based on the following requirements:${nl}${nl}Package version: ${package_version}${nl}${nl}\`\`\`json${nl}${version_requirements}${nl}\`\`\`"
Expand Down
19 changes: 2 additions & 17 deletions magefile.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2085,10 +2085,7 @@ func (Integration) UpdateVersions(ctx context.Context) error {
return nil
}

// UpdatePackageVersion update the file that contains the latest available snapshot version
func (Integration) UpdatePackageVersion(ctx context.Context) error {
const packageVersionFilename = ".package-version"

func (Integration) GetPackageVersion(ctx context.Context) error {
currentReleaseBranch, err := git.GetCurrentReleaseBranch(ctx)
if err != nil {
return fmt.Errorf("failed to identify the current release branch: %w", err)
Expand All @@ -2102,19 +2099,7 @@ func (Integration) UpdatePackageVersion(ctx context.Context) error {
if len(versions) != 1 {
return fmt.Errorf("expected a single version, got %v", versions)
}
packageVersion := versions[0].CoreVersion()
file, err := os.OpenFile(packageVersionFilename, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o644)
if err != nil {
return fmt.Errorf("failed to open %s for write: %w", packageVersionFilename, err)
}
defer file.Close()
_, err = file.WriteString(packageVersion)
if err != nil {
return fmt.Errorf("failed to write the package version file %s: %w", packageVersionFilename, err)
}

fmt.Println(packageVersion)

fmt.Println(versions[0].CoreVersion())
return nil
}

Expand Down