Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build: Update CircleCI configuration to remove build-sandboxes job #30360

Merged
merged 35 commits into from
Jan 29, 2025

Conversation

valentinpalkovic
Copy link
Contributor

@valentinpalkovic valentinpalkovic commented Jan 23, 2025

Closes #

What I did

  • Merged build-sandboxes into create-sandboxes (saved 3-5 mins)
  • Rebalanced CI resources so that jobs run more efficiently. Non-blocking tasks can run longer if we can save some resources. Blocking tasks should run faster.
  • Made the workspace persistence more efficient: The build-sandboxes step persisted all sandboxes and their node_modules leading to a huge persisted workspace. Re-attaching it in a follow-up step took up to 2:30 mins. Now node_modules from sandboxes do not persist anymore, decreasing the “Attach workspace” step of the last jobs by another 1:30 mins.
  • Updated yarn from 4.3.0 to 4.6.0

Workflow daily before:
image (5)

Workflow daily after:
image (6)

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

  • Add or update documentation reflecting your changes
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=<PR_NUMBER>

name before after diff z %
createSize 0 B 0 B 0 B - -
generateSize 77.9 MB 77.9 MB 0 B 1.54 0%
initSize 131 MB 131 MB 614 B 1.12 0%
diffSize 53 MB 53 MB 614 B -0.26 0%
buildSize 7.17 MB 7.17 MB 0 B -1.36 0%
buildSbAddonsSize 1.85 MB 1.85 MB 0 B 1.36 0%
buildSbCommonSize 195 kB 195 kB 0 B - 0%
buildSbManagerSize 1.86 MB 1.86 MB 0 B -1.36 0%
buildSbPreviewSize 0 B 0 B 0 B - -
buildStaticSize 0 B 0 B 0 B - -
buildPrebuildSize 3.91 MB 3.91 MB 0 B -1.36 0%
buildPreviewSize 3.26 MB 3.26 MB 0 B -1.36 0%
testBuildSize 0 B 0 B 0 B - -
testBuildSbAddonsSize 0 B 0 B 0 B - -
testBuildSbCommonSize 0 B 0 B 0 B - -
testBuildSbManagerSize 0 B 0 B 0 B - -
testBuildSbPreviewSize 0 B 0 B 0 B - -
testBuildStaticSize 0 B 0 B 0 B - -
testBuildPrebuildSize 0 B 0 B 0 B - -
testBuildPreviewSize 0 B 0 B 0 B - -
name before after diff z %
createTime 24.7s 24.8s 107ms 1.34 0.4%
generateTime 20.2s 21.7s 1.5s 0.54 7%
initTime 13.7s 12.9s -788ms -0.89 -6.1%
buildTime 9.9s 8.9s -1s -25ms -0.55 -11.5%
testBuildTime 0ms 0ms 0ms - -
devPreviewResponsive 4.9s 4.5s -329ms -0.88 -7.2%
devManagerResponsive 3.5s 3.4s -142ms -0.7 -4.2%
devManagerHeaderVisible 621ms 640ms 19ms 0.46 3%
devManagerIndexVisible 651ms 649ms -2ms 0.05 -0.3%
devStoryVisibleUncached 2s 3.5s 1.5s 6.24 🔺42.6%
devStoryVisible 652ms 667ms 15ms 0.25 2.2%
devAutodocsVisible 446ms 626ms 180ms 0.88 28.8%
devMDXVisible 514ms 618ms 104ms 1.23 16.8%
buildManagerHeaderVisible 605ms 636ms 31ms 0.37 4.9%
buildManagerIndexVisible 709ms 717ms 8ms 0.26 1.1%
buildStoryVisible 594ms 581ms -13ms -0.05 -2.2%
buildAutodocsVisible 437ms 478ms 41ms -0.28 8.6%
buildMDXVisible 458ms 543ms 85ms 0.8 15.7%

Greptile Summary

Here's my summary of the key changes in this PR:

This PR optimizes CircleCI configuration by merging jobs and improving workspace persistence, while also updating core dependencies.

  • Merges build-sandboxes into create-sandboxes job, saving 3-5 minutes per workflow
  • Optimizes workspace persistence by excluding sandbox node_modules, reducing attach time by ~1:30 minutes
  • Updates Node.js from 22.6.0 to 22.13.1 and Yarn from 4.3.0 to 4.6.0 across all configuration files
  • Rebalances CircleCI resource classes (xlarge -> large, large -> medium+) for better efficiency
  • Adds new get-sandbox-dir.ts script to handle sandbox directory name generation

The changes show improved build times but introduce some concerning performance regressions in dev environment responsiveness that should be investigated.

@valentinpalkovic valentinpalkovic self-assigned this Jan 23, 2025
@valentinpalkovic valentinpalkovic added ci:normal build Internal-facing build tooling & test updates and removed ci:normal labels Jan 23, 2025
Copy link

nx-cloud bot commented Jan 23, 2025

View your CI Pipeline Execution ↗ for commit d5e903a.

Command Status Duration Result
nx affected -t check -c production --parallel=7 ✅ Succeeded <1s View ↗
nx run-many -t build -c production --parallel=3 ✅ Succeeded 3s View ↗

☁️ Nx Cloud last updated this comment at 2025-01-29 13:38:44 UTC

@valentinpalkovic valentinpalkovic force-pushed the valentin/remove-build-sandboxes-step branch from 6cab86a to 544de57 Compare January 23, 2025 14:58
@storybook-pr-benchmarking
Copy link

storybook-pr-benchmarking bot commented Jan 27, 2025

Package Benchmarks

Commit: d5e903a, ran on 29 January 2025 at 13:46:31 UTC

No significant changes detected, all good. 👏

@valentinpalkovic valentinpalkovic added ci:daily Run the CI jobs that normally run in the daily job. and removed ci:normal labels Jan 28, 2025
@valentinpalkovic valentinpalkovic force-pushed the valentin/remove-build-sandboxes-step branch from 43857dc to cd5fe5e Compare January 28, 2025 12:46
@valentinpalkovic valentinpalkovic added ci:normal and removed ci:daily Run the CI jobs that normally run in the daily job. labels Jan 28, 2025
@valentinpalkovic valentinpalkovic added ci:daily Run the CI jobs that normally run in the daily job. and removed ci:normal labels Jan 28, 2025
@valentinpalkovic valentinpalkovic added ci:merged Run the CI jobs that normally run when merged. ci:normal and removed ci:daily Run the CI jobs that normally run in the daily job. ci:merged Run the CI jobs that normally run when merged. labels Jan 29, 2025
@valentinpalkovic valentinpalkovic marked this pull request as ready for review January 29, 2025 13:36
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

17 file(s) reviewed, 5 comment(s)
Edit PR Review Bot Settings | Greptile

Comment on lines +412 to +414
TEMPLATE=$(yarn get-template --cadence << pipeline.parameters.workflow >> --task sandbox)
yarn task --task build --template $TEMPLATE --no-link --start-from=sandbox --junit
if [[ $TEMPLATE != bench/* ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: The build task is now run with --start-from=sandbox instead of the sandbox task with --start-from=never. Verify this change doesn't skip necessary build steps.

yarnPath: ../.yarn/releases/yarn-4.3.0.cjs
# Sometimes you get a "The remote archive doesn't match the expected checksum" error, uncommenting this line will fix it
# checksumBehavior: 'update'
yarnPath: .yarn/releases/yarn-4.6.0.cjs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: The yarnPath is now relative to the current directory (.yarn/releases) instead of the parent directory (../.yarn/releases). Verify this path is correct and accessible in all build contexts.

Comment on lines +11 to +12
async function run({ template }: RunOptions) {
console.log(template.replace('/', '-'));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: template is marked as optional in RunOptions but not checked for undefined here. Could throw error if template is undefined.

Suggested change
async function run({ template }: RunOptions) {
console.log(template.replace('/', '-'));
async function run({ template }: RunOptions) {
if (!template) throw new Error('Template name is required');
console.log(template.replace('/', '-'));

Comment on lines +5 to +7
type RunOptions = {
template?: string;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: template should be required since the script won't work without it

Suggested change
type RunOptions = {
template?: string;
};
type RunOptions = {
template: string;
};

if (esMain(import.meta.url)) {
program
.description('Retrieve the sandbox directory for template name')
.option('--template <template>', 'Template name');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: consider making --template required using .requiredOption() instead of .option()

Suggested change
.option('--template <template>', 'Template name');
.requiredOption('--template <template>', 'Template name');

@valentinpalkovic valentinpalkovic merged commit 0d7da86 into next Jan 29, 2025
121 of 126 checks passed
@valentinpalkovic valentinpalkovic deleted the valentin/remove-build-sandboxes-step branch January 29, 2025 14:22
@github-actions github-actions bot mentioned this pull request Jan 29, 2025
11 tasks
@valentinpalkovic valentinpalkovic added the patch:yes Bugfix & documentation PR that need to be picked to main branch label Feb 3, 2025
@github-actions github-actions bot mentioned this pull request Feb 4, 2025
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Internal-facing build tooling & test updates ci:normal patch:yes Bugfix & documentation PR that need to be picked to main branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants