1.9.0

Important

This release includes some big changes and has had over 100 hours of work.
Please read carefully, as changes to internal APIs could impact your use case(s).

Logging

Modeling after various Linux logging interfaces, the new log API is very useful. It uses the same log levels as syslog, and is designed to be flexible and work well with any use case. There are a few built-in formats.

The majority of internal errors are logged. Errors that should be handled by the user (e.g. ENOENT when reading a file) are not logged. Additionally, deprecated functions and classes will log a deprecation warning.

The log is enabled by default. If you are working with resource constraints, you may want to disable logging since all entries are kept in memory. You could also periodically run log.entries.clear().

When using zenfs-test, you can set the level needed for logs to be displayed using the --log/-l options. The default is CRIT.

You can read more about logging in the documentation.

FileSystem metadata

FileSystemMetadata has been difficult to work with. It is computed every time it is fetched, damaging performance. This also has lead to numerous workarounds for its perceived immutability (e.g. _disableSync).

This release deprecates FileSystemMetadata, and introduces better designed and more intuitive ways to work with metadata:

• attributes is designed to keep track of simple values. This is analogous to options in /etc/fstab. At the moment, the built-in attributes are all booleans, which is indicated by a void type on FileSystemAttributes. This is because the presence of the key is used rather than the value. For non-boolean types, you can use attributes like a normal Map.

• usage returns UsageInfo, which is a subset of FileSystemMetadata. While the only file systems that return usable information for this are from @zenfs/archives, there are plans to introduce usage information for many others.

• name is for the name of the file system type. For example, tmpfs is the name of InMemory. This is modeled after file_system_type->name in Linux

• id is a unique 32-bit unsigned integer for the type of file system. It is currently unused internally but may in the future (one compelling use case is to resolve collisions of name). This is primarily designed for users, for example it could be used for partition tables.

• label is an optional field that is designed to be used with instances. This is the same as labels in the context of normal partitions and file systems.

Fetch

Fetch received some much needed improvements. You can now use HTTP range requests (#53) and update the remote (#181). Note if your server supports being updated, you will almost certainly want to perform lots of authentication and value checking. Additionally, Fetch is now tested with disableAsyncCache, to make sure it will still work. Along with these change to Fetch, IndexFS has been refactored.

StoreFS, Stores, and Transactions

As many of you know, StoreFS is used by many ZenFS backends (e.g. InMemory and IndexedDB). This release cleans up StoreFS and the underlying stores and transactions.

• StoreFS now builds an ino lookup table when initialized.
  • This significantly improves performance after initialization, especially for stores that are slow (e.g. IndexedDB).
  • The old behavior is no longer available, but could be in a future release if desirable
• Fixed write failing if the file didn't already exist
• read will no longer read data when the requested length is 0
  • It still checks for file existence and gets metadata
• StoreFS can now allocate new IDs for inodes and data in constant time instead of the previous O(n)
• Removed the type parameter from commitNew and commitNewSync

For stores and transactions:

• Renamed Simple* to Map* (e.g. SimpleTransaction to MapTransaction) for clarification
• Added caching to AsyncTransaction, which makes implementing an async backend even easier
  • Implementors: You will need to make sure you update the cache properly.
  • This also includes a new interface, AsyncStore
• Extracted the transaction rollback capabilities into WrappedTransaction, a new internal class used to abstract the "raw" transaction.
  • Removed the commit, abort, and [Symbol.dispose] properties from Transaction, along with their synchronous variants
• Removed keysSync from Transaction, since it isn't used internally.
• The Transaction methods get and getSync can now return undefined
  • This should happen when no data is available, rather than throwing an ENOENT/ENODATA
  • Error throwing for missing data is now done by StoreFS
• Deprecated the Store methods clear and clearSync since they aren't used

Miscellaneous

• Move raw test data from being inlined into files
• Cleaned up fs.promises.watch
• Fixed FSWatcher not respecting context when added to watchers
• Fixed emitChange from within context (#167)
• Correctly labeled emitChange as internal and hidden
• Moved the FileContents type to vfs/types.ts
• zenfs-test setup files can now be skipped.
  • You can prefix the filename with _
• Added a deadlock canary to context test
• Fixed PassthroughFS.readSync
• Consolidated Port test suites into a single file
• Added copySync and copyAsync functions for tests

Internal API organization

The various internal APIs have all been moved into the src/internal directory. Please make sure to update any imports you have.

1.8.8

• Added missing Sync methods: read and write
• Improved how Async handles async queueing
• Fixed an assertion in write tests (assert(...).equals(...) -> assert.equals)
• Improved a message in write tests

1.8.7

• Performance improvements to realpath (#174)
• Switched to strict assertion mode for tests
• Improved messages and file names for rename tests

1.8.6

• Index.directories is no longer memoized since caching lead to invalid entries
• Added Index.directoryEntries to get the directory entries for a single path in the index
• Enabled tsconfig verbatimModuleSyntax option
• Changed type-only imports to use type

1.8.5

• Changed default Inode.size from 4096 to 0
• Rename unit tests (Unit tests (...) → Unit tests: ...)
• Some CI steps (e.g. tests) now show up as GitHub checks
• Added CI CLI (zci)
• Added zenfs-test --ci/-C
• Added zenfs-test --inspect/-I

1.8.4

Fixed growBuffer failing when SharedArrayBuffer is missing (#171, thanks @mcandeia!)

1.8.3

• Changed LazyFile.prepareRead to return the end position
• Fix FileSystem.read being called with incorrect end
• Deprecated randomBigInt (internal function)

1.8.2

• Fixed growBuffer failing if ArrayBuffer.transfer isn't defined
• Cleaned up DeviceDriver.read and .write

1.8.1

Note

This release includes a bug fix that is a breaking change

• Moved over LazyFile
• OverlayFS no longer uses a mutex
• Improved FSRequest types in Port
• Fixed PortFS.read buffer transfer issues
• Implemented readSync and writeSync on Async
• Added growBuffer
• Changed from ArrayBuffer.slice to ArrayBuffer.subarray (performance)
• Fixed LazyFile.write and .writeSync using an incorrect position
• Added DeviceDriver.readD and DeviceDriver.writeD
• Deprecated DeviceDriver.read and .write
• Added documentation to new FileSystem methods
• Fixed FileSystem.read and .readSync not having a buffer parameter (breaking)
• Passthrough.sync and .syncSync now sync some stats
• The Inode constructor now accepts a partial InodeLike
• Readonly now throws EROFS on write and `writeSync

1.8.0

• Added LazyFile
  • This is not used at the moment by any backends, though it will in a future release
• Added FileSystem.read and FileSystem.write
• Added StatsCommon.toJSON
• Reduce default max byte length to 1 MiB (#168)
• Added FileReadResult type that uses built-in ArrayBufferView
• Removed PortFile, file RPC, and RPC scopes
• Fixed StoreFS.loadIndex (+Sync) not committing the transaction
• Fixed files trying to call FileSystem.sync on read-only file systems
• Reworked Port error passing
• Changed the default value of length in File.read and File.write (+Sync)
  • From this.stats.size to buffer.byteLength - offset
  • This now has the same behavior as the relevant Node.js APIs
• Fixed PreloadFile's handling of a buffer and stats size mismatch
• PortFS.sync now converts metadata to a plain object
• Fixed PortFile.read not copying to buffer
• Changed utimes to use numbers internally
• Tweaked Fetch.baseUrl trailing slash behavior
• Fixed backend options type checking
• Changed FetchFS to extend StoreFS
• Added StoreFS methods for working with Index:
  • loadIndex
  • loadIndexSync
  • toIndex
  • toIndexSync
• Deprecated IndexFS
  • You can use StoreFS and call loadIndex/loadIndexSync with your index
• Changed inode and data IDs from bigint to number
• Rewrote Index and IndexFS to be read/write
• Added while canaries
  • This should prevent any infinite locks due to while loops
• Fixed StoreFS not using creation options
• Improved internal _open options
• Added directory file type checks
• Added PureCreationOptions

Testing:

• Updated the npm test script
• Improved Fetch testing
• StoreFS using Index is now run through the full FS test suite
• Port (including Async) is now run through the full FS test suite
• Removed Async-specific common test suite
• Simplified temporary coverage directory removal
• Failing tests will now give an exit code of 1
• Reduced tests wait times
• zenfs-tests output is now much prettier. It has colors and fancy formatting and such.
• Added zenfs-test flags:
  • --clean will remove cached coverage data
  • --report will report any cached coverage data, then remove it
  • --preserve will run tests but will not remove the coverage data or report on it
• Deprecated APIs are no longer considered when calculating coverage
• CI/CD tests now display output about test suites