Set a maximum amount of pending/unpinned CIDs per user

[mbommerez]

As discussed with @alanshaw - sister issue for NFT

Overarching user needs / context:

• We've recently had a user pin hundreds of thousands of CIDs that they do not have access to and can never retrieve. This adds load to our cluster and dagcargo that is currently never resolved.
• The idea here is to add a quota for maximum "in flight" pins. This effectively puts a per user cap on the number of pins that can be submitted that never become pinned. The user then starts to receive 4xx errors for pin requests, which should prompt them to delete bad pins. It also serves as a crude rate limiting mechanism. The idea behind is to encourage users to clean up after themselves and remove inaccessible CIDs/pin requests.

Scope of this ticket:

• This ticket is only scoped to PSA and not to uploads

Acceptance criteria:

• We can set a certain 'Pending Pins' quota by user
• This amount is taken into account by the PSA API and blocks users once they take their "in flight" amount of pins over this limit.
• Different users can have different allowed quota.

Dependencies

• Discussed to keep this functionality in the new Elastic IPFS pinning pick-up service API, closer to the ingestion source, rather than letting the Pinning Service API handle such limits.
• To minimise friction in user experience we should consider implementing this first: Give up trying to pin a CID after a given time threshold #812

To be clarified

We can set a certain amount of Pending Pins quota in admin.storage that is stored as a user tag

• What's the use case for this? Can this be a constant throughout all users to start with? The per user setting could be progressive enhancement if required?
• Define a sensible amount of in-flight requests.

Notes