A complete layman explanation of the plebbit protocol

[estebanabaroa]

Note: for non-technical audience only, NOT factually correct

Hashing

Hashing is a program that converts a file of any size, big or small, into always the same small string. For example if I hash the book "Harry Potter" I might get a string like Z7jVs7JTFxsx7pEmGceQyDL1XMBiKdJC1WvHmf1H. If anyone else in the world hashes the same book, they would get the same string. This is used to uniquely identify and validate files.

Peer to peer discovery

In a P2P network, peers must discover each other's IP addresses. A P2P program like Bittorrent or Bitcoin has a hardcoded list of up to date peers and their IP addresses that you can connect to the first time you launch the program. The first thing you ask those peers is for them to send you more up to date peers and their IP addresses, so that the next time you launch the program, you don't need to rely on hardcoded peers. It only takes a few seconds to receive a new list of thousands of up to date peers, and as long as one of them is online the next time you launch the program, you will be able to rebuild a list of thousands of up to date peers.

Peer communication

Once you know the IP address of a peer, you can communicate with them like you communicate with a regular website. For example if you go to google.com/search?q=movies you will get a list of movies, if you go to google.com/search?q=books you will get a list of books. Peers do not have human readable names like google.com, instead they have IP addresses like 159.182.94.87. If you communicate with peer 159.182.94.87/?query=get_peers, you will get a list of peers. You can communicate with any online peer at any time. If a peer is malicious and doesn't send you what you requested, you can block them.

File discovery

If you want to download the file Harry Potter.pdf, you can ask a random peer you know 159.182.94.87/?get_file=Harry Potter.pdf, and he probably won't have this file, but he will send you a list of random peers he knows, and then you will ask each of them for the same file, and if they don't have it, they will also send you random peers they know, and you will repeat this process until you find a peer that has the file and sends you the file. This can take from a few seconds to several minutes, you might have to ask tens of thousands of peers.

File addressing (IPFS and Bittorrent)

There might be 2 files called Harry Potter.pdf, so how do you tell peers which one you want? You use the hash of the file as the file "address". For example if the file Harry Potter.pdf hashes to Z7jVs7JTFxsx7pEmGceQyDL1XMBiKdJC1WvHmf1H, you can ask a peer for this file by doing 159.182.94.87/?get_file=Z7jVs7JTFxsx7pEmGceQyDL1XMBiKdJC1WvHmf1H

Private key and public key

A private key is a long passphrase like vPopJBaE5f4H9RqpBYyZtWHmfAlWhGARUTutVbXhAO2GFksMU4AOSQecqs9b4mXXfY4ZrAdGk6kbRZdz, it can also be a list of words like disquiet ironlike dihedral broad dory cumbrous candy libeler alegar. A public key is a smaller random looking string like 4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8. A program is used to generate a private key and its corresponding public key. The private key is kept secret, the public key is visible to all.

Private key and public key signature

A program can use a private key to generate a signature for a message. Another program can combine this message, the signature, and the public key, and prove that it was correctly signed by the private key corresponding to the public key, without knowing the private key. This is used to prove the identity of the sender of messages over the internet.

Peer addressing

If you want to communicate with peer 159.182.94.87, there might be several people sharing this IP address, or maybe this peer's IP address changed, so instead of using only an IP address to identify a peer, you can use both an IP address and a public key. So you do:

159.182.94.87/?peer_public_key=4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8&get_file=Z7jVs7JTFxsx7pEmGceQyDL1XMBiKdJC1WvHmf1H

Which will ask for file Z7jVs7JTFxsx7pEmGceQyDL1XMBiKdJC1WvHmf1H but also for the message to be signed by peer 4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8. If it's not signed by that peer, you can reject the message and block the malicious peer.

Peer discovery by address

You might want to communicate with 159.182.94.87, but his IP isn't online, but you know his peer's public key is 4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8. You can ask another random peer you know if he knows another IP address for peer 4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8 by doing 159.182.94.87/?get_peer=4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8 which will go through the same process as the file discovery by address, but using a peer's address instead of a file address. Once you find this peer, you can confirm it's really him because all his messages are signed with his peer public key. A peer public key and a peer address are the same thing.

Signed file discovery

You might want to download files or messages signed by a peer, you don't need to know this peer's IP address or to communicate with him directly. You can use the file discovery process, but instead of using the file's address as the query, you use the peer's address to get all files signed by this peer. For example you ask a random peer you know 159.182.94.87/?get_files_signed_by=4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8 and he will give you all the files he has signed by peer 4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8, if he doesn't have any, he will give you random peers, and then you will ask them, and so on. This process can take several minutes and you might ask tens of thousands of peers, it is longer than the file discovery by file address process because you're looking for potentially multiple files signed by one peer instead of just one file by its address.

Latest signed file discovery (IPNS)

You might want to download the very last file or message signed by a peer, for example maybe the file you want is a peer's blog or website, and you only want to see the very last update to his website, you don't care about past updates. The peer can add a timestamp to each of his signed file or message, and after going through the entire signed file discovery process, which takes several minutes, you can ignore all signed files except the signed file with the most recent timestamp. File addressing discovery can only be used to download files that never change, like a book or movie. Latest signed file discovery can be used to download files that change, like a website, but only the peer who owns the private key used to sign can make the changes.

Real time signed file discovery (IPNS over pubsub)

You might want to download new files or messages signed by a peer in real time as the peer publishes them, for example a peer's chat messages, or a peer's livestream video. You don't need to know this peer's IP address or to communicate with him directly. You can use the file discovery process, but instead of looking for peers that have a specific file, you look for peers that are also listening for real time signed files from the specific peer you want, for example you can ask a random peer 159.182.94.87/?get_peers_listening_for_real_time_files_signed_by=4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8, and if this random peer doesn't know any, he will send you random peers that you can ask, and you can repeat this process up to thousands of times, which can take several minutes, until you have a list of all peers also listening for real time signed files by peer 4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8.

You can communicate with each one of these peers on the list, for example 239.34.254.84/?join_real_time_files_signed_by=4mXXfY4ZrAdGk6kbRZdzXEbv4yRlWWLccuvlqWX8 and after they receive this communication from you, they will start sending you every new signed file by this peer that they receive, in real time. And if you receive a new signed file before another peer, you will also do your part and send it to them automatically. All the new signed files will propagate to all peers looking for them in real time. If a malicious peer sends a file wrongly signed, all other peers will detect it and block him.

Real time unsigned file discovery (pubsub)

You might want to download new files or messages in real time that are not signed by a specific peer, but rather published to a frequency, like a radio. Anyone can publish, anyone can listen to that specific radio frequency. It also doesn't have to be a frequency, it can be be a topic, like cats. You can use the same process as real time signed file discovery, but instead of finding all peers listening for real time files signed by a specific peer, you would find all peers listening for real time files published to a specific topic, for example you would ask a random peer 159.182.94.87/?get_peers_listening_to_topic=cats, and if this random peer doesn't know anyone doing so, he would send you random peers, and then you would ask those random peers, and so on. It might take several minutes and thousands of requests, but eventually you would gather a list of all peers listening for real time files for the topic cats.

You can communicate with each one of these peers on the list, for example 239.34.254.84/?join_real_time_topic=cats and after they receive this communication from you, they will start sending you every new file that anyone publishes with the topic cats. Real time unsigned file discovery is not useful alone, because anyone can spam anything, but it is possible to add custom rules to this concept and make it useful. Plebbit uses this concept and adds extra rules which are explained at the end.

Note: this is the end of generic peer to peer concepts, anything below this is plebbit specific

Creating a subplebbit

When you create a subplebbit, you create a new public key pair, and the public key is the address of the subplebbit. Your subplebbit becomes a "peer" with its own public key pair, and every 2 minutes this "peer" publishes a new signed file. This signed file contains the up to date information of the subplebbit, like title, description, list of moderators, etc. It also contains the file addresses of the first pages of posts sorted by new, hot, top, etc.

Managing a subplebbit

Your subplebbit desktop or server node must be online 24/7 or it will stop publishing updates every 2 minutes, and new content will stop being published until you come back online. The node does not require a lot of bandwidth or disk space. Users who don't run a subplebbit don't need to be online 24/7 and don't need to run a node to post or read.

Searching for a subplebbit

There are no ways to search for a subplebbit as part of the plebbit protocol. You must discover it outside the protocol, kind of like how you must find a torrent address or Ethereum smart contract address outside the protocol before you can use it.

Viewing a subplebbit

To view a subplebbit you must know its address, which is the public key of its "peer". You can find the subplebbit peer's latest signed file using the latest signed file discovery process, or the real time signed file discovery process. The latest signed file contains the up to date information of the subplebbit, like its title, description, the file addresses of the first pages sorted by new, hot, top, etc.

Viewing a subplebbit page

A page is a regular file, you can download it by its file address using the file address discovery process. You can learn what the first page file address of a subplebbit is by first viewing the subplebbit, because the subplebbit's latest signed file contains the address of the first page for each sort type (new, hot, top, etc). After you download the first page, the first page contains the file address to the second page, and so on.

Viewing a comment

A comment is a regular file, you can download it by its file address using the file address discovery process. You can learn the file address of a comment by first downloading a page of a subplebbit, or by a friend sharing the address (e.g. https://plebbit.eth.limo/c/Z7jVs7JTFxsx7pEmGceQyDL1XMBiKdJC1WvHmf1H) of the comment with you. A comment file contains content, author information, author public key and author signature. A comment file also contains a public key for the comment itself, so each comment is a "peer" with its own public key pair. The subplebbit owner controls the comment's public key pair and every 2 minutes he publishes a new signed comment update file on behalf of the comment "peer". The comment update file contains the up to date upvotes, replies, edits and moderation status of the comment. You can download the latest "comment update" by using the latest signed file discovery process, or the real time signed file discovery process.

Viewing an author's profile and history

If the author runs a node and is currently online, you can access his profile and comment history by his peer address using the latest signed file discovery process. If the author does not run a node or is not online, you cannot access his profile via his peer address. But a single recent comment found on a subplebbit contains enough information alone to reconstruct most of an author's profile and history. It contains author address, display name, total author karma in the subplebbit, as well as the file address of the previous comment by the author, even from different subplebbits. Since each author comment contains the previous author comment file address, all of them can be accessed by visiting each one after the other.

Publishing a comment to a subplebbit

A peer can publish a comment using the real time unsigned file discovery process. The real time "topic" he would use would be the address of the subplebbit. The subplebbit "peer" is always listening for real time files being published under the topic of his own subplebbit address so he would receive the comment published.

Preventing comment spam on real time unsigned file discovery

Since any peer can publish anything with real time unsigned file discovery, extra rules must be added to prevent spam. Plebbit adds the rule that an author must first request a challenge, the subplebbit replies with the challenge, the author replies with the answer, and the subplebbit finally replies with the challenge verification.

This challenge exchange all happens publicly over the real time unsigned file discovery and all peers witness it. If a peer requests too many challenges without enough successes, this peer's IP address eventually gets fully blocked by all other peers, so infinite spam is prevented. The challenge can be anything the subplebbit owner decides, for example: a captcha, a password, a question, a proof of balance of a certain token, a whitelist, a certain karma requirement, etc. Some challenges occur automatically, like proof of balance, whitelist, karma requirement, some require human input, like captcha and question.

Publishing a vote to a sub

A peer can publish a vote using the same process as publishing a comment, a challenge is also required.

Publishing a comment moderation

A peer with moderator privileges can publish a comment moderation using the same process as publishing a comment. A challenge is not necessary. If the peer's signature matches one of the subplebbit's moderator addresses assigned by the owner of the subplebbit, it can be automatically accepted, or if not, it can be automatically rejected.