Feature: Files and Batches as a unified route

[narengogi]

Related Issues: (optional)

• [Feature] Support batches and files as unified API routes #819

Feature to add batches and files as a unified route
This add the following routes:

POST /v1/files
GET /v1/files
GET /v1/files/:fileId
DELETE /v1/files/:fileId

POST /v1/batches
GET /v1/batches
GET /v1/batches/:batchId
POST /v1/batches/:batchId/cancel

✨ Gateway specific additional route for convenience (Not in OpenI spec) ✨
GET /v1/batches/:batchId/output

Description

• For file uploads, the gateway needs to transform files to the provider format (like bedrock) on the fly, the files can be as large as 20GB, this implementation does not load the files into memory, rather it transforms the data streams on the fly with nodeJS streams and pushes to the other endpoint with a half duplex connection.

• Similar streaming transformers for getting batch output and getting files is implemented, where the file stream is transformed on the fly

• I've had to upgrade wrangler version for nodeJS compatibility to add Polyfill support for nodeJS crypto module which is used for signing the AWS requests, although the gateway has @smithy/signature-v4 package installed, I wasn't able to get it to work for the s3 file upload routes, I'll have to revisit this

Testing done

• To make sure that we're not adding any additional memory overhead, and that there are no memory leaks, I've monitored the memory consumption with the top command on my machine

during testing a 1gb file upload, memory went up by about 40MiB, but returned to normal shortly after

TO-DO for later

• Revisit and simplify the AWS signer code, we should be able to use the @smithy/signature-v4 package without having to write all the signing code ourselves, or alternatively we can remove the smithy signing package and implement the functions ourselves