Skip to content

Commit

Permalink
docs: update manifesto (#84)
Browse files Browse the repository at this point in the history
Refs #35

- Removes permissions section as it's not relevant to general usage
- Updates out of date sections and adds links to relevant modules
- Adds emojis

---------

Co-authored-by: Steve Loeppky <[email protected]>
  • Loading branch information
achingbrain and BigLep authored Apr 13, 2023
1 parent 3446638 commit 33461fe
Showing 1 changed file with 45 additions and 30 deletions.
75 changes: 45 additions & 30 deletions MANIFESTO.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# Manifesto <!-- omit in toc -->
# 🗣️ Manifesto <!-- omit in toc -->

Helia aims to be a next generation IPFS implementation that takes the learnings of [js-ipfs] and applies them to a modern, modular, and efficient codebase.

## Table of Contents <!-- omit in toc -->

- [Modular](#modular)
- [BYO Filesystem](#byo-filesystem)
- [JavaScript first](#javascript-first)
- [(runtime-specific code where it makes sense)](#runtime-specific-code-where-it-makes-sense)
- [Permissions](#permissions)
- [ESM and TypeScript](#esm-and-typescript)
- [Non-goals](#non-goals)
- [Networking](#networking)
- [🧱 Modular](#-modular)
- [📁 BYO Filesystem](#-byo-filesystem)
- [⚗️ Isomorphic JavaScript](#️-isomorphic-javascript)
- [🚀 (runtime-specific code where it makes sense)](#-runtime-specific-code-where-it-makes-sense)
- [Observability](#observability)
- [📜 ESM and TypeScript](#-esm-and-typescript)
- [Non-goals](#-non-goals)
- [🌐 Networking](#-networking)

## Modular
## 🧱 Modular

The use cases for a distributed filesystem are incredibly far reaching and the "bundle-everything" approach of `js-ipfs`/`kubo` does not suit every application.

Expand All @@ -23,27 +23,32 @@ For example:
- Other applications may be deployed in extremely adversarial environments, and should limit the number of dependencies (throughout the dependency tree) to reduce the opportunities for supply chain attacks
- The user should not have to include the code for features their application does not use

The core of Helia will be very focused on use as a library: just [js-libp2p], a [blockstore], [js-bitswap] and a POSIX-like API which will be extendable to add additional features such as IPNS, an RPC-API, etc.
The core of Helia will be very focused on use as a library: just [js-libp2p], a [datastore], and a [blockstore] that transparently uses [js-bitswap] to load any requested blocks not already in the blockstore from the network and/or to transfer them to network peers.

A "get you started" bundle with some common components will be provided but users are very much encouraged to roll their own version of Helia to suit their use case.
Users are very much encouraged to bundle extra components with their version of Helia to suit their use case.

> **Note**
>
> An implementation of IPNS compatible with helia can be installed from npm [`@helia/ipns`](https://github.com/ipfs/helia-ipns)
Extra components currently available are:

## BYO Filesystem
- [`@helia/ipns`](https://github.com/ipfs/helia-ipns) - for publishing and resolving [IPNS] names
- [`@helia/unixfs`](https://github.com/ipfs/helia-unixfs) - for performing [UnixFS] operations

We hope there will be more soon!

## 📁 BYO Filesystem

The default filesystem for IPFS is [UnixFS](https://github.com/ipfs/specs/blob/main/UNIXFS.md), but UnixFS has several limitations. Support for some common Unix file attributes such as mode (permission bits) and `mtime` landed in UnixFSv1.5, but this has yet to make it to [kubo].

Several features are still missing from 1.5, such as arbitrary metadata (extended attributes) or versioning, but they have been implemented by other filesystems such as [WNFS](https://guide.fission.codes/developers/webnative/file-system-wnfs).

That these missing features are being implemented by other filesystems is incredibly exciting and will unlock new use cases that are not possible today, so Helia will not bless any one filesystem as the One True Implementation, instead it will present an abstraction of posix filesystem operations (`ls`, `cat`, etc) as an API but the underlying filesystem(s) will be configurable.
That these missing features are being implemented by other filesystems is incredibly exciting and will unlock new use cases that are not possible today, so Helia will not bless any one filesystem as the One True Implementation, instead it is intended to provide the features required to implement these file systems in a network-aware way that allows pulling data from and sending data to peers.

Currently available filesystems are:

- [`@helia/unixfs`](https://github.com/ipfs/helia-unixfs) - for performing [UnixFS] operations

> **Note**
>
> An implementation of unixFS compatible with helia can be installed from npm [`@helia/unixfs`](https://github.com/ipfs/helia-unixfs).
We hope there will be more soon!

## JavaScript first
## ⚗️ Isomorphic JavaScript

In the beginning there were Node.js [streams](https://nodejs.org/api/stream.html#readable-streams). Then there were [pull streams](https://www.npmjs.com/package/pull-stream). And finally, browsers have [native streams too](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream).

Expand All @@ -53,30 +58,40 @@ There are many other cases where primitives that existed only in Node.js or were

This also makes it easier to support other runtimes like [deno](https://deno.land/) or [bun](https://bun.sh/) since runtime-specific code should be behind configuration such as the `"browser"` or `"exports"` fields in the `package.json`.

### (runtime-specific code where it makes sense)
### 🚀 (runtime-specific code where it makes sense)

Some things are just faster in Node.js. The ability to run native code should not be underestimated so where the JS implementation of an algorithm is a proven performance bottleneck and a native version exists, it should be used.

## Permissions
## Observability

An IPFS implementation typically contains a lot of moving parts all of which must move in perfect synchronicity for an operation to be successful.

When things do not go smoothly it's not always clear in which part of the system the fault has occurred.

Helia aims to alleviate some of this pain by exposing comprehensive progress handlers that give insight into specific calls and the components that are interacted with.

`js-ipfs` has no concept of permissions so all capabilities are available to every user. This has caused lots of problems with the HTTP-RPC-API in particular and lead to hastily implemented attempts to lock access down like limiting HTTP verbs or detecting user agents.
For example catting a file might involve requesting blocks from the blockstore, adding them to your bitswap want list, making DHT queries to find providers of that block, connecting to the provider and finally pulling the block down - any of these can go wrong but the user should be able to pinpoint where the problem lays.

Helia will integrate permissions at its core, also allowing delegation of permissions via mechanisms such as [UCAN](https://ucan.xyz/)s.
In addition to this standard tools such as [Prometheus](https://prometheus.io/) should be usable with many metrics exposed to allow deep introspection of node behaviour and the debugging and isolatino of performance bottlenecks.

## ESM and TypeScript
## 📜 ESM and TypeScript

Trying to write applications that scale has always been a challenge for JavaScript. Tools like TypeScript ease some of this pain, so Helia will be written in TypeScript and published as ESM-only to take advantage of modern runtimes and tooling.

## Non-goals
## Non-goals

Helia is not attempting to reinvent the wheel when it comes to layers beneath the top-level API.
Helia is not attempting to reimplement layers beneath the top-level API so [js-libp2p] will remain as will [js-bitswap] and [js-unixfs].

### Networking
### 🌐 Networking

It will use [js-libp2p] and [js-bitswap] to ensure compatibility with existing IPFS clients.

[js-ipfs]: https://github.com/ipfs/js-ipfs
[js-libp2p]: https://github.com/libp2p/js-libp2p
[js-bitswap]: https://github.com/ipfs/js-ipfs-bitswap
[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore
[blockstore]: https://github.com/ipfs/js-stores/tree/master/packages/interface-blockstore
[datastore]: https://github.com/ipfs/js-stores/tree/master/packages/interface-datastore
[kubo]: https://github.com/ipfs/kubo
[IPNS]: https://docs.ipfs.tech/concepts/ipns/
[UnixFS]: https://github.com/ipfs/specs/blob/main/UNIXFS.md
[js-unixfs]: https://github.com/ipfs/js-ipfs-unixfs

0 comments on commit 33461fe

Please sign in to comment.