[Feature] Add support for async OAuth token refreshes

[renaudhartert-db]

What changes are proposed in this pull request?

This PR aims at eliminating long-tail latency due to OAuth token refreshes in scenarios where a single client is responsible for a relatively high (e.g. > 1 QPS) continuous outbound traffic. The feature is disabled by default — which arguably makes this PR a functional no-op.

Precisely, the PR introduces a new token cache which attempts to always keep its token fresh by asynchronously refreshing the token before it expires. We differentiate three token states:

• fresh: The token is valid and is not close to its expiration.
• stale: The token is valid but will expire soon.
• expired: The token has expired and cannot be used.

Each time a request tries to access the token, we do the following:

• If the token is fresh, return the current token;
• If the token is stale, trigger an asynchronous refresh and return the current token;
• If the token is expired, make a blocking refresh call to update the token and return it.

In particular, asynchronous refreshes use a lock to guarantee that there can only be one pending refresh at a given time.

The performance of the algorithm depends on the length of the stale and fresh periods. On the first hand, the stale period must be long enough to prevent tokens from entering the expired state. On the other hand, a long stale period reduces the length of the fresh period, thus increasing the refresh frequency.

Right now, the stale period is configured to 3 minutes by default (i.e. 5% of the expected token lifespan of 1 hour). This value might be changed in the future to guarantee that the default behavior achieves the best performance for the majority of users.

For reviewers: this PR only uses the new cache in control-plane auth flows; I plan to send a follow-up PR to enable asynchronous refresh in data-plane flows once this one has been merged.

How is this tested?

Complete test coverage with a focus on various concurrency scenarios.

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-go

Inputs:

• PR number: 1135
• Commit SHA: dc40cbb3b1c7fb810cb6aa7473654a8ae3432a44

Checks will be approved automatically on success.