feat: Use OAuth flow to generate R2 tokens for Pipelines #7534
Conversation
🦋 Changeset detectedLatest commit: 92842a6 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@penalosa here is a variant of what we discussed surrounding the OAuth flow. We instead are hosting the Worker doing the token exchange instead of making wrangler a trusted client. I'll definitely need some help/guidance around the testing aspect of this PR. |
|
A wrangler prerelease is available for testing. You can install this latest build in your project with: npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-wrangler-7534You can reference the automatically updated head of this PR with: npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/prs/7534/npm-package-wrangler-7534Or you can use npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-wrangler-7534 dev path/to/script.jsAdditional artifacts:wget https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-cloudflare-workers-bindings-extension-7534 -O ./cloudflare-workers-bindings-extension.0.0.0-vf2ed8e9ff.vsix && code --install-extension ./cloudflare-workers-bindings-extension.0.0.0-vf2ed8e9ff.vsixnpx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-create-cloudflare-7534 --no-auto-updatenpm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-cloudflare-kv-asset-handler-7534npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-miniflare-7534npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-cloudflare-pages-shared-7534npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-cloudflare-unenv-preset-7534npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-cloudflare-vitest-pool-workers-7534npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-cloudflare-workers-editor-shared-7534npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-cloudflare-workers-shared-7534npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/12638040461/npm-package-cloudflare-workflows-shared-7534Note that these links will no longer work once the GitHub Actions artifact expires.
Please ensure constraints are pinned, and |
cmackenzie1
force-pushed
the
cole/pipelines-auto-r2-tokens
branch
from
6f27584 to
c11f0f4
Compare
There was a problem hiding this comment.
@cmackenzie1 thanks very much for this! I've got just a few comments about code organisation and documentation.
cmackenzie1
force-pushed
the
cole/pipelines-auto-r2-tokens
branch
from
522332c to
08b2507
Compare
There was a problem hiding this comment.
is it not possible to test this with mocks like the existing ones for wrangler login, which also depend on user interactivity and callbacks. Or have you considered that already and decided it would be mocking so much that there's not too much point in the first place?
This commit changes the generateR2Tokens flow which will direct the user to the web browser to perform a OAuth flow to grant the Workers Pipelines client the ability to generate R2 tokens on behalf of the user. This will only run if the user does not provide the credentials as CLI parameters. Due to requiring user interactivity, and reliance on the callbacks, there is no easy way to support a "headless" mode for `wrangler pipelines create` (or `update`) unless the user provides the tokens as arguments. The same applies for testing this flow, which can only be done manually at this time.
Create odd-ducks-attack.md
After creating an R2 token, there is a slight delay before if can be used. Previously we would sleep for some amount of time, but this method is really sensitive to latency. Instead, use the S3 SDK and try using the token until we exhaust all attempts, or we finally succeed in using it. Each failure results in a constant backoff of 1 second. This commit does add the dependency `@aws-sdk/client-s3`.
cmackenzie1
force-pushed
the
cole/pipelines-auto-r2-tokens
branch
from
5ad62f8 to
c63dbb0
Compare
This uses the promise based version of `setTimeout` from NodeJS and registers the AbortController to handle cancellation signal. The http server `.close()` method is also registered to the abort controller for cleanup as `controller.abort()` is always called before returning the result.
Fixes https://jira.cfdata.org/browse/PIPE-159.
This commit changes the generateR2Tokens flow which will direct the user to the web browser to perform a OAuth flow to grant the Workers Pipelines client the ability to generate R2 tokens on behalf of the user. This will only run if the user does not provide the credentials as CLI parameters.
Due to requiring user interactivity, and reliance on the callbacks, there is no easy way to support a "headless" mode for
wrangler pipelines create(orupdate) unless the user provides the tokens as arguments. The same applies for testing this flow, which can only be done manually at this time.