Constructor
new ObjectManager(clientKey, clientSecret, options)
Creates a new instance of the constructor.
Name | Type | Description |
---|---|---|
clientKey | string | The access key ID for authentication. |
clientSecret | string | The secret access key for authentication. |
options | objectManagerOptions | Optional settings for the constructor. |
- Source
- Tutorials
import { ObjectManager } from "@filebase/sdk";
+ On this pageObjectManager
Interacts with an S3 client to perform various operations on objects in a bucket.
Constructor
new ObjectManager(clientKey, clientSecret, options)
Creates a new instance of the constructor.
Parameters:Name Type Description clientKey
string The access key ID for authentication.
clientSecret
string The secret access key for authentication.
options
objectManagerOptions Optional settings for the constructor.
- Source
- Tutorials
Exampleimport { ObjectManager } from "@filebase/sdk";
const objectManager = new ObjectManager("KEY_FROM_DASHBOARD", "SECRET_FROM_DASHBOARD", {
bucket: "my-default-bucket",
maxConcurrentUploads: 4,
@@ -8,7 +8,7 @@
endpoint: "https://my-default-gateway.mydomain.com
token: SUPER_SECRET_GATEWAY_TOKEN
}
-});
Methods
(async) upload(key, source, metadataopt, optionsopt) → {Promise.<objectHeadResult>}
Uploads a file or a CAR file to the specified bucket.
If the source parameter is an array of objects, it will pack multiple files into a CAR file for upload. The method returns a Promise that resolves to an object containing the CID (Content Identifier) of the uploaded file and an optional entries object when uploading a CAR file.
Parameters:Name Type Attributes Description key
string The key or path of the file in the bucket.
source
Buffer | ReadableStream | Array.<Object> The content of the object to be uploaded. If an array of files is provided, each file should have a 'path' property specifying the path of the file and a 'content' property specifying the content of the file. The SDK will then construct a CAR file locally and use that as the content of the object to be uploaded.
metadata
Object <optional>
Optional metadata for pin object
options
objectOptions <optional>
The options for uploading the object.
- Source
Returns:- Type:
- Promise.<objectHeadResult>
Example// Upload Object
+});
Methods
(async) upload(key, source, metadataopt, optionsopt) → {Promise.<objectHeadResult>}
Uploads a file or a CAR file to the specified bucket.
If the source parameter is an array of objects, it will pack multiple files into a CAR file for upload. The method returns a Promise that resolves to an object containing the CID (Content Identifier) of the uploaded file and an optional entries object when uploading a CAR file.
Parameters:Name Type Attributes Description key
string The key or path of the file in the bucket.
source
Buffer | ReadableStream | Array.<Object> The content of the object to be uploaded. If an array of files is provided, each file should have a 'path' property specifying the path of the file and a 'content' property specifying the content of the file. The SDK will then construct a CAR file locally and use that as the content of the object to be uploaded.
metadata
Object <optional>
Optional metadata for pin object
options
objectOptions <optional>
The options for uploading the object.
- Source
Returns:- Type:
- Promise.<objectHeadResult>
Example// Upload Object
await objectManager.upload("my-object", Buffer.from("Hello World!"));
// Upload Object with Metadata
await objectManager.upload("my-custom-object", Buffer.from("Hello Big World!"), {
@@ -28,12 +28,12 @@
path: "/topLevel.txt",
content: Buffer.from("upload top level test object", "utf-8"),
},
-]);
(async) get(key, optionsopt) → {Promise.<(objectHeadResult|false)>}
Gets an objects info and metadata using the S3 API.
Parameters:Name Type Attributes Description key
string The key of the object to be inspected.
options
objectOptions <optional>
The options for inspecting the object.
- Source
Returns:- Type:
- Promise.<(objectHeadResult|false)>
(async) download(key, optionsopt) → {Promise.<Object>}
Downloads an object from the specified bucket using the provided key.
Parameters:Name Type Attributes Description key
string The key of the object to be downloaded.
options
objectOptions <optional>
The options for downloading the object.
- Source
Returns:- A promise that resolves with the contents of the downloaded object as a Stream.
- Type:
- Promise.<Object>
Example// Download object with name of `download-object-example`
-await objectManager.download(`download-object-example`);
(async) list(options) → {Promise.<listObjectsResult>}
Retrieves a list of objects from a specified bucket.Retrieves a list of objects from a specified bucket.
Parameters:Name Type Description options
listObjectOptions The options for listing objects.
- Source
Returns:- A promise that resolves to an array of objects.
- Type:
- Promise.<listObjectsResult>
Example// List objects in bucket with a limit of 1000
+]);
(async) get(key, optionsopt) → {Promise.<(objectHeadResult|false)>}
Gets an objects info and metadata using the S3 API.
Parameters:Name Type Attributes Description key
string The key of the object to be inspected.
options
objectOptions <optional>
The options for inspecting the object.
- Source
Returns:- Type:
- Promise.<(objectHeadResult|false)>
(async) download(key, optionsopt) → {Promise.<Object>}
Downloads an object from the specified bucket using the provided key.
Parameters:Name Type Attributes Description key
string The key of the object to be downloaded.
options
objectOptions <optional>
The options for downloading the object.
- Source
Returns:- A promise that resolves with the contents of the downloaded object as a Stream.
- Type:
- Promise.<Object>
Example// Download object with name of `download-object-example`
+await objectManager.download(`download-object-example`);
(async) list(options) → {Promise.<listObjectsResult>}
Retrieves a list of objects from a specified bucket.Retrieves a list of objects from a specified bucket.
Parameters:Name Type Description options
listObjectOptions The options for listing objects.
- Source
Returns:- A promise that resolves to an array of objects.
- Type:
- Promise.<listObjectsResult>
Example// List objects in bucket with a limit of 1000
await objectManager.list({
MaxKeys: 1000
-});
(async) delete(key, optionsopt) → {Promise.<Boolean>}
Deletes an object from the specified bucket using the provided key.
Parameters:Name Type Attributes Description key
string The key of the object to be deleted.
options
objectOptions <optional>
The options for deleting the file.
- Source
Returns:- A Promise that resolves with the result of the delete operation.
- Type:
- Promise.<Boolean>
Example// Delete object with name of `delete-object-example`
-await objectManager.delete(`delete-object-example`);
(async) copy(sourceKey, destinationBucket, optionsopt) → {Promise.<Boolean>}
Copy the object from sourceKey in the sourceBucket to destinationKey in the destinationBucket.
If the destinationKey is not provided, the object will be copied with the same key as the sourceKey.
Parameters:Name Type Attributes Description sourceKey
string The key of the object to be copied from the sourceBucket.
destinationBucket
string The bucket where the object will be copied to.
options
copyObjectOptions <optional>
Additional options for the copy operation.
- Source
Returns:- A Promise that resolves with the result of the copy operation.
- Type:
- Promise.<Boolean>
Example// Copy object `copy-object-test` from `copy-object-test-pass-src` to `copy-object-test-pass-dest`
+});
(async) delete(key, optionsopt) → {Promise.<Boolean>}
Deletes an object from the specified bucket using the provided key.
Parameters:Name Type Attributes Description key
string The key of the object to be deleted.
options
objectOptions <optional>
The options for deleting the file.
- Source
Returns:- A Promise that resolves with the result of the delete operation.
- Type:
- Promise.<Boolean>
Example// Delete object with name of `delete-object-example`
+await objectManager.delete(`delete-object-example`);
(async) copy(sourceKey, destinationBucket, optionsopt) → {Promise.<Boolean>}
Copy the object from sourceKey in the sourceBucket to destinationKey in the destinationBucket.
If the destinationKey is not provided, the object will be copied with the same key as the sourceKey.
Parameters:Name Type Attributes Description sourceKey
string The key of the object to be copied from the sourceBucket.
destinationBucket
string The bucket where the object will be copied to.
options
copyObjectOptions <optional>
Additional options for the copy operation.
- Source
Returns:- A Promise that resolves with the result of the copy operation.
- Type:
- Promise.<Boolean>
Example// Copy object `copy-object-test` from `copy-object-test-pass-src` to `copy-object-test-pass-dest`
// TIP: Set bucket on constructor, it will be used as the default source for copying objects.
await objectManager.copy(`copy-object-test`, `copy-object-dest`, {
sourceBucket: `copy-object-src`
diff --git a/global.html b/global.html
index f29b6f2..2fb4faf 100644
--- a/global.html
+++ b/global.html
@@ -1,3 +1,3 @@
Global On this pageType Definitions
bucket
Type:- Object
PropertiesName Type Description Name
string The name of the bucket
Date
date the bucket was created
- Source
gateway
Type:- Object
PropertiesName Type Description name
string Name for the gateway
domain
string Custom Domain for the gateway
enabled
boolean Whether the gateway is enabled or not
private
string Whether the gateway is scoped to users content
created_at
date Date the gateway was created
updated_at
date Date the gateway was last updated
- Source
gatewayOptions
Type:- Object
PropertiesName Type Attributes Description domain
boolean <optional>
Optional Domain to allow for using a Custom Domain
enabled
string <optional>
Optional Toggle to use for enabling the gateway
private
boolean <optional>
Optional Boolean determining if gateway is Public or Private
- Source
name
Type:- Object
PropertiesName Type Description label
string Descriptive label for the Key
network_key
string IPNS Key CID
cid
string Value that name Publishes
sequence
number Version Number for the name
enabled
boolean Whether the name is being Published or not
published_at
date Date the name was last published to the DHT
created_at
date Date the name was created
updated_at
date Date the name was last updated
- Source
nameOptions
Type:- Object
PropertiesName Type Attributes Description enabled
boolean <optional>
Whether the name is enabled or not.
- Source
objectManagerOptions
Optional settings for the constructor.Optional settings for the constructor.
Type:- Object
PropertiesName Type Attributes Description bucket
string <optional>
Default bucket to use.
gateway
objectDownloadOptions <optional>
Default gateway to use.
maxConcurrentUploads
number <optional>
The maximum number of concurrent uploads.
- Source
objectDownloadOptions
Optional settings for downloading objects
.Optional settings for downloading objects
Type:- Object
PropertiesName Type Attributes Default Description endpoint
string Default gateway to use.
token
string <optional>
Token for the default gateway.
timeout
number <optional>
60000 Timeout for the default gateway
- Source
objectOptions
Type:- Object
PropertiesName Type Attributes Description bucket
string <optional>
The bucket to pin the IPFS CID into.
- Source
objectHeadResult
Type:- Object
PropertiesName Type Attributes Description cid
string The CID of the uploaded object
download
function Convenience function to download the object via S3 or the selected gateway
entries
array.<Object> <optional>
If a directory then returns an array of the containing objects
Properties
Name Type Description cid
string The CID of the uploaded object
path
string The path of the object
- Source
listObjectsResult
Type:- Object
PropertiesName Type Description IsTruncated
boolean Indicates if more results exist on the server
NextContinuationToken
string ContinuationToken used to paginate list requests
Contents
Array.<Object> List of Keys stored in the S3 Bucket
Properties
Name Type Description Key
string Key of the Object
LastModified
string Date Last Modified of the Object
CID
string CID of the Object
ETag
string ETag of the Object
Size
number Size in Bytes of the Object
StorageClass
string Class of Storage of the Object
download
function Convenience function to download the item using the S3 gateway
- Source
listObjectOptions
Type:- Object
PropertiesName Type Attributes Default Description Bucket
string <optional>
The name of the bucket. If not provided, the default bucket will be used.
ContinuationToken
string <optional>
null Continues listing from this objects name.
Delimiter
string <optional>
null Character used to group keys
MaxKeys
number <optional>
1000 The maximum number of objects to retrieve. Defaults to 1000.
- Source
copyObjectOptions
Type:- Object
PropertiesName Type Attributes Description sourceBucket
string <optional>
The source bucket from where the object is to be copied.
destinationKey
string <optional>
The key of the object in the destination bucket. By default, it is the same as the sourceKey.
- Source
pinManagerOptions
Optional settings for the constructor.Optional settings for the constructor.
Type:- Object
PropertiesName Type Attributes Description bucket
string <optional>
Default bucket to use.
gateway
pinDownloadOptions <optional>
Default gateway to use.
- Source
pinDownloadOptions
Optional settings for downloading pins
.Optional settings for downloading pins
Type:- Object
PropertiesName Type Attributes Default Description endpoint
string Default gateway to use.
token
string <optional>
Token for the default gateway.
timeout
number <optional>
60000 Timeout for the default gateway
- Source
pinStatus
Type:- Object
PropertiesName Type Attributes Description requestid
string Globally unique identifier of the pin request; can be used to check the status of ongoing pinning, or pin removal
status
string Status a pin object can have at a pinning service. ("queued","pinning","pinned","failed")
created
string Immutable timestamp indicating when a pin request entered a pinning service; can be used for filtering results and pagination
pin
Object Pin object
Properties
Name Type Description cid
string Content Identifier (CID) pinned recursively
name
string Name for pinned data; can be used for lookups later
origins
Array.<string> Optional list of multiaddrs known to provide the data
meta
Object Optional metadata for pin object
delegates
Array.<string> List of multiaddrs designated by pinning service that will receive the pin data
info
object <optional>
Optional info for PinStatus response
download
function Convenience function to download pin
- Source
pinOptions
Type:- Object
PropertiesName Type Attributes Description bucket
string <optional>
The bucket to pin the IPFS CID into.
- Source
listPinOptions
Type:- Object
PropertiesName Type Attributes Description cid
Array.<string> <optional>
Return pin objects responsible for pinning the specified CID(s); be aware that using longer hash functions introduces further constraints on the number of CIDs that will fit under the limit of 2000 characters per URL in browser contexts
name
string <optional>
Return pin objects with specified name (by default a case-sensitive, exact match)
match
string <optional>
Customize the text matching strategy applied when the name filter is present; exact (the default) is a case-sensitive exact match, partial matches anywhere in the name, iexact and ipartial are case-insensitive versions of the exact and partial strategies
status
Array.<string> <optional>
Return pin objects for pins with the specified status (when missing, service defaults to pinned only)
before
string <optional>
Return results created (queued) before provided timestamp
after
string <optional>
Return results created (queued) after provided timestamp
limit
number <optional>
Max records to return
meta
Object <optional>
Return pin objects that match specified metadata keys passed as a string representation of a JSON object; when implementing a client library, make sure the parameter is URL-encoded to ensure safe transport
- Source
listPinResults
Type:- Object
PropertiesName Type Description count
number Total number of pin objects that exist for passed query filters
Array
Array.<pinStatus> of PinStatus results
- Source
replacePinOptions
Type:- Object
PropertiesName Type Attributes Description metadata
Object <optional>
Optional metadata to set on pin during replacement
name
string <optional>
Optional name for pin to set during replacement
- Source
\ No newline at end of file
+ On this pageType Definitions
bucket
Type:- Object
PropertiesName Type Description Name
string The name of the bucket
Date
date the bucket was created
- Source
gateway
Type:- Object
PropertiesName Type Description name
string Name for the gateway
domain
string Custom Domain for the gateway
enabled
boolean Whether the gateway is enabled or not
private
string Whether the gateway is scoped to users content
created_at
date Date the gateway was created
updated_at
date Date the gateway was last updated
- Source
gatewayOptions
Type:- Object
PropertiesName Type Attributes Description domain
boolean <optional>
Optional Domain to allow for using a Custom Domain
enabled
string <optional>
Optional Toggle to use for enabling the gateway
private
boolean <optional>
Optional Boolean determining if gateway is Public or Private
- Source
name
Type:- Object
PropertiesName Type Description label
string Descriptive label for the Key
network_key
string IPNS Key CID
cid
string Value that name Publishes
sequence
number Version Number for the name
enabled
boolean Whether the name is being Published or not
published_at
date Date the name was last published to the DHT
created_at
date Date the name was created
updated_at
date Date the name was last updated
- Source
nameOptions
Type:- Object
PropertiesName Type Attributes Description enabled
boolean <optional>
Whether the name is enabled or not.
- Source
objectManagerOptions
Optional settings for the constructor.Optional settings for the constructor.
Type:- Object
PropertiesName Type Attributes Description bucket
string <optional>
Default bucket to use.
gateway
objectDownloadOptions <optional>
Default gateway to use.
maxConcurrentUploads
number <optional>
The maximum number of concurrent uploads.
- Source
objectDownloadOptions
Optional settings for downloading objects
.Optional settings for downloading objects
Type:- Object
PropertiesName Type Attributes Default Description endpoint
string Default gateway to use.
token
string <optional>
Token for the default gateway.
timeout
number <optional>
60000 Timeout for the default gateway
- Source
objectOptions
Type:- Object
PropertiesName Type Attributes Description bucket
string <optional>
The bucket to pin the IPFS CID into.
- Source
objectHeadResult
Type:- Object
PropertiesName Type Attributes Description cid
string The CID of the uploaded object
download
function Convenience function to download the object via S3 or the selected gateway
entries
array.<Object> <optional>
If a directory then returns an array of the containing objects
Properties
Name Type Description cid
string The CID of the uploaded object
path
string The path of the object
- Source
listObjectsResult
Type:- Object
PropertiesName Type Description IsTruncated
boolean Indicates if more results exist on the server
NextContinuationToken
string ContinuationToken used to paginate list requests
Contents
Array.<Object> List of Keys stored in the S3 Bucket
Properties
Name Type Description Key
string Key of the Object
LastModified
string Date Last Modified of the Object
CID
string CID of the Object
ETag
string ETag of the Object
Size
number Size in Bytes of the Object
StorageClass
string Class of Storage of the Object
download
function Convenience function to download the item using the S3 gateway
- Source
listObjectOptions
Type:- Object
PropertiesName Type Attributes Default Description Bucket
string <optional>
The name of the bucket. If not provided, the default bucket will be used.
ContinuationToken
string <optional>
null Continues listing from this objects name.
Delimiter
string <optional>
null Character used to group keys
MaxKeys
number <optional>
1000 The maximum number of objects to retrieve. Defaults to 1000.
- Source
copyObjectOptions
Type:- Object
PropertiesName Type Attributes Description sourceBucket
string <optional>
The source bucket from where the object is to be copied.
destinationKey
string <optional>
The key of the object in the destination bucket. By default, it is the same as the sourceKey.
- Source
pinManagerOptions
Optional settings for the constructor.Optional settings for the constructor.
Type:- Object
PropertiesName Type Attributes Description bucket
string <optional>
Default bucket to use.
gateway
pinDownloadOptions <optional>
Default gateway to use.
- Source
pinDownloadOptions
Optional settings for downloading pins
.Optional settings for downloading pins
Type:- Object
PropertiesName Type Attributes Default Description endpoint
string Default gateway to use.
token
string <optional>
Token for the default gateway.
timeout
number <optional>
60000 Timeout for the default gateway
- Source
pinStatus
Type:- Object
PropertiesName Type Attributes Description requestid
string Globally unique identifier of the pin request; can be used to check the status of ongoing pinning, or pin removal
status
string Status a pin object can have at a pinning service. ("queued","pinning","pinned","failed")
created
string Immutable timestamp indicating when a pin request entered a pinning service; can be used for filtering results and pagination
pin
Object Pin object
Properties
Name Type Description cid
string Content Identifier (CID) pinned recursively
name
string Name for pinned data; can be used for lookups later
origins
Array.<string> Optional list of multiaddrs known to provide the data
meta
Object Optional metadata for pin object
delegates
Array.<string> List of multiaddrs designated by pinning service that will receive the pin data
info
object <optional>
Optional info for PinStatus response
download
function Convenience function to download pin
- Source
pinOptions
Type:- Object
PropertiesName Type Attributes Description bucket
string <optional>
The bucket to pin the IPFS CID into.
- Source
listPinOptions
Type:- Object
PropertiesName Type Attributes Description cid
Array.<string> <optional>
Return pin objects responsible for pinning the specified CID(s); be aware that using longer hash functions introduces further constraints on the number of CIDs that will fit under the limit of 2000 characters per URL in browser contexts
name
string <optional>
Return pin objects with specified name (by default a case-sensitive, exact match)
match
string <optional>
Customize the text matching strategy applied when the name filter is present; exact (the default) is a case-sensitive exact match, partial matches anywhere in the name, iexact and ipartial are case-insensitive versions of the exact and partial strategies
status
Array.<string> <optional>
Return pin objects for pins with the specified status (when missing, service defaults to pinned only)
before
string <optional>
Return results created (queued) before provided timestamp
after
string <optional>
Return results created (queued) after provided timestamp
limit
number <optional>
Max records to return
meta
Object <optional>
Return pin objects that match specified metadata keys passed as a string representation of a JSON object; when implementing a client library, make sure the parameter is URL-encoded to ensure safe transport
- Source
listPinResults
Type:- Object
PropertiesName Type Description count
number Total number of pin objects that exist for passed query filters
Array
Array.<pinStatus> of PinStatus results
- Source
replacePinOptions
Type:- Object
PropertiesName Type Attributes Description metadata
Object <optional>
Optional metadata to set on pin during replacement
name
string <optional>
Optional name for pin to set during replacement
- Source
\ No newline at end of file
diff --git a/objectManager.js.html b/objectManager.js.html
index e0eaf54..c816cb8 100644
--- a/objectManager.js.html
+++ b/objectManager.js.html
@@ -17,15 +17,9 @@
import { car } from "@helia/car";
import { mfs } from "@helia/mfs";
import { unixfs } from "@helia/unixfs";
-import { FsBlockstore } from "blockstore-fs";
+import { MemoryBlockstore } from "blockstore-core";
import { MemoryDatastore } from "datastore-core";
// Utility Imports
-import { once } from "node:events";
-import { createReadStream, createWriteStream } from "node:fs";
-import { mkdir, rm, open } from "node:fs/promises";
-import os from "node:os";
-import path from "node:path";
-import { Readable } from "node:stream";
import { v4 as uuidv4 } from "uuid";
import { downloadFromGateway } from "./helpers.js";
import PQueue from "p-queue";
@@ -188,17 +182,23 @@
let temporaryCarFilePath, temporaryBlockstoreDir;
try {
// Setup Blockstore
- temporaryBlockstoreDir = path.resolve(
- os.tmpdir(),
- "filebase-sdk",
- "uploads",
- uploadUUID,
- );
- temporaryCarFilePath = `${temporaryBlockstoreDir}/main.car`;
- await mkdir(temporaryBlockstoreDir, { recursive: true });
- const temporaryBlockstore = new FsBlockstore(temporaryBlockstoreDir),
+ let temporaryBlockstore = new MemoryBlockstore(),
temporaryDatastore = new MemoryDatastore();
-
+ if (isNode()) {
+ const { mkdir } = await import("node:fs/promises");
+ const { FsBlockstore } = await import("blockstore-fs");
+ const os = await import("node:os");
+ const path = await import("node:path");
+ temporaryBlockstoreDir = path.resolve(
+ os.tmpdir(),
+ "filebase-sdk",
+ "uploads",
+ uploadUUID,
+ );
+ temporaryCarFilePath = `${temporaryBlockstoreDir}/main.car`;
+ await mkdir(temporaryBlockstoreDir, { recursive: true });
+ temporaryBlockstore = new FsBlockstore(temporaryBlockstoreDir);
+ }
let createdFiles = new Map();
const heliaFs = unixfs({
blockstore: temporaryBlockstore,
@@ -218,43 +218,61 @@
path: entry.path,
size: queue.size,
});
- let createdFile;
- if (
- (entry.type === "import" && entry.content !== null) ||
- entry.content instanceof Readable
- ) {
- let filehandle;
- try {
- if (entry.type === "import") {
- filehandle = await open(path.resolve(entry.content), "r");
- entry.content = filehandle.createReadStream();
+
+ if (isNode()) {
+ const { Readable } = await import("node:stream");
+ let createdFile;
+ if (
+ (entry.type === "import" && entry.content !== null) ||
+ entry.content instanceof Readable
+ ) {
+ let filehandle;
+ try {
+ if (entry.type === "import") {
+ const { open } = await import("node:fs/promises");
+ const path = await import("node:path");
+ filehandle = await open(path.resolve(entry.content), "r");
+ entry.content = filehandle.createReadStream();
+ }
+ createdFile = await heliaFs.addByteStream(entry.content);
+ } catch (err) {
+ if (typeof filehandle !== "undefined") {
+ await filehandle.close();
+ }
+ throw err;
}
- createdFile = await heliaFs.addByteStream(entry.content);
- } catch (err) {
if (typeof filehandle !== "undefined") {
await filehandle.close();
}
- throw err;
+ } else if (entry.content !== null) {
+ createdFile = await heliaFs.addBytes(entry.content);
+ } else {
+ return;
}
- if (typeof filehandle !== "undefined") {
- await filehandle.close();
- }
- } else if (entry.content !== null) {
- createdFile = await heliaFs.addBytes(entry.content);
+ createdFiles.set(entry.path, createdFile);
+ uploadLogger.verbose("SOURCE_IMPORT_COMPLETED", {
+ path: entry.path,
+ size: queue.size,
+ });
} else {
- return;
+ let createdFile;
+ if (entry.type === "import" && entry.content !== null) {
+ createdFile = await heliaFs.addByteStream(entry.content);
+ } else if (entry.content !== null) {
+ createdFile = await heliaFs.addBytes(entry.content);
+ } else {
+ return;
+ }
+ createdFiles.set(entry.path, createdFile);
+ uploadLogger.verbose("SOURCE_IMPORT_COMPLETED", {
+ path: entry.path,
+ size: queue.size,
+ });
}
- createdFiles.set(entry.path, createdFile);
- uploadLogger.verbose("SOURCE_IMPORT_COMPLETED", {
- path: entry.path,
- size: queue.size,
- });
});
})();
if (queue.size > 150) {
- while (queue.size > 100) {
- await once(queue, "next");
- }
+ await queue.onSizeLessThan(100);
}
createFilePromises.push(task);
uploadLogger.verbose("SOURCE_IMPORT_QUEUED", {
@@ -317,16 +335,22 @@
const carExporter = car({ blockstore: temporaryBlockstore }),
{ writer, out } = CarWriter.create([rootEntry.cid]);
- // Put carFile stream to disk
- const output = createWriteStream(temporaryCarFilePath);
- Readable.from(out).pipe(output);
- await carExporter.export(rootEntry.cid, writer);
- uploadLogger.verbose("CAR_EXPORTED", {
- root: rootEntry,
- });
+ if (isNode()) {
+ const { createReadStream, createWriteStream } = await import(
+ "node:fs"
+ );
+ const { Readable } = await import("node:stream");
+ // Put carFile stream to disk
+ const output = createWriteStream(temporaryCarFilePath);
+ Readable.from(out).pipe(output);
+ await carExporter.export(rootEntry.cid, writer);
+ uploadLogger.verbose("CAR_EXPORTED", {
+ root: rootEntry,
+ });
- // Set Uploader to Read from carFile on disk
- uploadOptions.params.Body = createReadStream(temporaryCarFilePath);
+ // Set Uploader to Read from carFile on disk
+ uploadOptions.params.Body = createReadStream(temporaryCarFilePath);
+ }
// Upload carFile via S3
uploadLogger.verbose("CAR_UPLOADING", {
@@ -347,7 +371,8 @@
console.error(err.message);
throw err;
} finally {
- if (typeof temporaryBlockstoreDir !== "undefined") {
+ if (typeof temporaryBlockstoreDir !== "undefined" && isNode()) {
+ const { rm } = await import("node:fs/promises");
// Delete Temporary Blockstore
await rm(temporaryBlockstoreDir, { recursive: true, force: true });
}
@@ -571,5 +596,10 @@
return (path.match(/\//g) || []).length;
}
+// Function to check if the code is running in Node.js or the browser
+function isNode() {
+ return typeof process !== "undefined" && process.release.name === "node";
+}
+
export default ObjectManager;