Skip to content

Commit

Permalink
fix: Wait for R2 token to sync
Browse files Browse the repository at this point in the history
After creating an R2 token, there is a slight delay before if can be
used. Previously we would sleep for some amount of time, but this method
is really sensitive to latency.

Instead, use the S3 SDK and try using the token until we exhaust all
attempts, or we finally succeed in using it. Each failure results in a
constant backoff of 1 second.

This commit does add the dependency `@aws-sdk/client-s3`.
  • Loading branch information
cmackenzie1 committed Jan 2, 2025
1 parent 3d2ea2a commit c63dbb0
Show file tree
Hide file tree
Showing 3 changed files with 1,443 additions and 113 deletions.
1 change: 1 addition & 0 deletions packages/wrangler/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@
"type:tests": "tsc -p ./src/__tests__/tsconfig.json && tsc -p ./e2e/tsconfig.json"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.721.0",
"@cloudflare/kv-asset-handler": "workspace:*",
"@esbuild-plugins/node-globals-polyfill": "^0.2.3",
"@esbuild-plugins/node-modules-polyfill": "^0.2.2",
Expand Down
27 changes: 24 additions & 3 deletions packages/wrangler/src/pipelines/index.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
import { HeadBucketCommand, S3Client } from "@aws-sdk/client-s3";
import { readConfig } from "../config";
import { sleep } from "../deploy/deploy";
import { FatalError, UserError } from "../errors";
import { printWranglerBanner } from "../index";
import { logger } from "../logger";
import * as metrics from "../metrics";
import { APIError } from "../parse";
import { requireAuth } from "../user";
import { retryOnError } from "../utils/retry";
import {
createPipeline,
deletePipeline,
Expand Down Expand Up @@ -51,8 +52,28 @@ async function authorizeR2Bucket(
pipelineName
);

// wait for token to settle/propagate
!__testSkipDelaysFlag && (await sleep(3000));
const r2 = new S3Client({
region: "auto",
credentials: {
accessKeyId: serviceToken.accessKeyId,
secretAccessKey: serviceToken.secretAccessKey,
},
endpoint: getAccountR2Endpoint(accountId),
});

// Wait for token to settle/propagate, retry up to 10 times, with 1s waits in-between errors
!__testSkipDelaysFlag &&
(await retryOnError(
async () => {
await r2.send(
new HeadBucketCommand({
Bucket: bucketName,
})
);
},
1000,
10
));

return serviceToken;
}
Expand Down
Loading

0 comments on commit c63dbb0

Please sign in to comment.