v0.19.0: Better config, DNS, improved fee & flash support, thinner SCP

Major improvements since v0.18.0:

Binary serialization now uses big endian.

This enables the serializer to be used for other network protocol than Agora's.
Since binary serialization is used for block serialization, it will require a node reset.

Network libraries dependencies have been updated

This fixes a few issues that would trigger in edge cases, such as the registry not properly
shutting down when SIGQUIT was sent to it.

TxBuilder has improved support for fees

TxBuilder is an utility used by Agora in unit tests, and other programs such as Faucet,
to efficiently and easily generate transactions.
The latest changes to it added better support for transactions fees,
as one can now specify the fee rate instead of an absolute value.
Additionally, small amounts that would otherwise create a dust-like output
(an output that is more expensive to spend than its own value) are now automatically
added to the fee.

Registry: Now a DNS server too

The name registry, used to locate validators and flash nodes on the network,
has been improved to respond to DNS queries.
The aim is to automatically provide a domain for every node on the network,
making accessing administrative interface and Votera APIs over HTTPS painless.
This feature is still experimental and will be improved on in future releases.
Currently, compression, recursion, EDNS are not implemented.
DNSSEC will also be considered once the previous items are available.

Flash: Implemented SIGHASH_SINGLE, SIGHASH_ANYONECANPAY, and a combination

This fixes an issue where uncooperative closes would require the counter party
to publish all updates instead of just the last one.

As a result, the API has slightly changed: signatures now come with metadata
(the combination is named SigPair), which affects requestCloseSig,
onRequestUpdateSig, requestUpdateSig.

Flash: Use NOINPUT | SINGLE | ANYONECANPAY for update transactions

This combination lets us create a floating TX with modifiable inputs and outputs
while also protecting the channel funds with a multi signature.

With this change, nodes can add fees to the update TXs before they publish them to the network.

The nodes have also been changed to properly include fees.

Recuring enrollment now happens in the last two blocks

Before, auto-enrollment was triggered very early (halfway through the cycle),
which could come at a surprise to users.
This is now done within 2 blocks of the enrollment expiring.

Event handlers configuration section has changed

Event handlers can be defined to send certain notifications (pre-image received,
block externalized, etc...) to a server for processing.
This is used by Stoa, Agora's API server, to generate a rich overview of the blockchain.
An example of the section as it was originally defined:

event_handlers:
  block_externalized:
    - http://127.0.0.1:3836/block_externalized

Other event types could be used besides block_externalized.
This should now be written as:

event_handlers:
  BlockExternalized:
    addresses:
      - http://127.0.0.1:3836/block_externalized

Adding an extra addresses under the event name, and using CamelCase instead of snake_case
for the event names. The list of event is still the same:

• BlockExternalized;
• BlockHeaderUpdated;
• PreimageReceived;
• TransactionReceived;

SCP protocol has been optimized to avoid transmitting network-known data

Since quorum sets are derived from collected pre-images,
the quorum sets for any given blocks are known to all validators.
Additionally, since enrollment is done in a BlockHeader,
the public key of any validator is also known in advance.
Those two informations were however copied in every SCP message,
the quorum as a hash (64 bytes) and the public key as a whole (64 bytes).

The SCP layer will now transmit only the NodeID,
which is the index of the validator for the block, as used in missing_preimages.

Agora will now error out on unrecognized configuration options

On startup, Agora parses a configuration file, config.yaml by default.
If a required option is missing, it will complain about it.
However, since most configuration options are actually optional (to simplify first-time uses),
it was fairly easy to misspell a configuration key and it would go unnoticed.

Starting from this release, Agora will now error out if the configuration contains keys
that are not recognized.

Additionally, the code has a whole has been overhauled for a better UX,
which should greatly reduce the risk for error.

Docker image now ships agora-config-dumper

As part of the configuration overhaul, a little utility was written, called agora-config-dumper.
It prints the content of the configuration that would be used by Agora for a given config file.
It ships with agora on the docker image and will be in the PATH.

Fixed a bug where transactions were not always sorted

Transactions would only be sorted if the number of transactions in the pool would fully fill the block.
They are now always sorted, which should lead to faster and more predictable block creation.

Fee distribution is now discrete

The fee distribution used to rely on a continuous formula, which could cause problems if an attacker
chose certain specific (prime) values, preventing anyone but the commons budget to receive those fees.
The distribution is now based on a discrete formula, hence validators will now need to add
a certain amount (400 BOA) to see their share of the fee increase.

TLS can now be set independently and explicitly

The first implementation of the TLS support saw it enabled automatically whenever a certificate
and a matching key was found. However, this could be problematic, as the user could have misplaced
or misnamed the files, resulting in TLS being silently disabled.

Hence, one need to use https for type when specifying the node's interfaces,
and HTTPS will either be enabled with the certificates that were found, or an error will be triggered
and the node will not start.

Additionally, the aforementioned automatic usage of TLS would affect both the node's public interface
and its admin interface. The admin interface is now using TLS by default and the node will error out
if no certificates were found. If one wish to disable TLS for the admin interface, tls = false
can be applied to the admin section in the configuration file.
However, this can cause grave security issues and should not be done lightly.

TransactionPool will now remove double-spend of externalized transactions

If a block was externalized that included a transaction that didn't end up in the node's pool,
but the node had a transaction in its pool that would spend the same(s) input,
the node would fail to remove those transactions.
While usually harmless, they would take up memory space for nothing. This is now correctly handled.

TxBuilder now requires the usage of unlockSigner

The TxBuilder uses a delegate to look up the private keys that control a certain output,
and create the signature (or the related script).
This delegate used to be provided when calling sign, but should now be set by calling the
unlockSigner setter method, preferably before taking any other action.

Fees are now part of CoinBase

A CoinBase is triggered every payout period. This is done to reduce the blockchain load
as the number of validator grows. However, fees were originally paid individually in each block.
All fees are now exclusively paid as part of the CoinBase transaction.

v0.18.0: Respect payout period, SSL fix, network and logging improvements

Major improvements since v0.17.0:

• Coinbase transactions are now only generated at each payout period instead of each block;
• The /validators endpoint will now return an empty result instead of an exception when called
  with a future block that doesn't have validators;
• The stats endpoint (/metrics) default timeout is not 70s, up from 10s, to cover Prometheus' default (60s);
• File logging will now flush every line after writing it, reducing writing delays;
• OpenSSL bindings have been updated to a more recent version;

v0.17.0: Pre-image bug fixes, SigHash.NoInput support

Major improvements since v0.16.0:

• The official Docker image is now hosted at bosagora/agora (moved from bpfk/agora);
• The execution engine now supports SigHash.NoInput;
• Flash now correctly binding over an already-externalized older state;
• The application stats now include a start_time field;
• Error handling has been improved to be more informative and less resource-consuming;
• Pre-images are now stored separately from enrollments,
  fixing various issues that could arises when a node would get
  a newer pre-image before seeing the re-enrollment;

v0.16.0: Improved logging, multiple minor crashes fixed, better libconsensus

Major improvements since v0.15.0:

• The /validators endpoint will now longer return a 500 when called with height=0;
• Ledger.getValidators: Return only validators that are not yet slashed;
• Remove an ambiguous usage of PublicKey (instead of UTXO) in signature aggregation;
• Key mapping have been removed from the EnrollmentManager, reducing memory usage;
• BitMask: Fixed serialization to not duplicate the length field, fixing a DoS vector;
• Admin endpoints now require the use of TLS (HTTPS);
• Admin endpoints now support basic auth, which can be configured from the config file;
• Some logging statements have been adjusted, resulting in less noise being produced at Info level;
• Validator nodes that cannot enroll, regardless of the reason, will no longer crash on shutdown;
• SCP logging will now produce human-readable strings for node ID;
• SCP logging is now fully configurable, including trace / debug level, like any other logging system;
• UTXODB will now always return ordered UTXO;
• Renamed endpoint stats for GET /block_height from block_heigth to block_height;

v0.15.0: Signature fixes, flash fees, better stats & config

Major improvements since v0.14.0:

• Blocks are not only signed after SCP's EXTERNALIZE, instead of COMMIT;
• block_interval_sec is now a parameter of consensus instead of node;
• Flash: Funding tx fee and closing tx fees are now supported;
• Block stats will now always be generated, even when no block has been generated yet;

v0.14.0: Guaranteed block time, integration fixes, many preimages, timer optimizations

Major improvements since v0.13.0:

• Agora will now produce blocks at the right interval, even if empty;
• The agora_preimages_gauge stats entry has been renamed to agora_preimages_counter;
• Fixed the VoterCard signature scheme to only hash once, not twice;
• Fixed QR admin methods to not escape the returned string;
• Nodes will now generate a large (5M) of pre-images on startup,
  allowing them to stay validators for longer timer before switching UTXOs;
• Various timers used for recurring tasks have been optimized to only fire when necessary,
  instead of a fixed regular interval;
• Agora will now longer automatically sign blocks when catching up, to avoid potential double signatgure issues;

v0.13.0: Signature adjustment, DoS prevent, Talos & SCP overhaul

Major improvements since v0.12.0:

• Fixed a bug that allowed validators to sign an extra block before slashing actually happened;
• SCP layer now requires C++17 support;
• Inconsistency between ConsensusParams' ValidatorCycle and the Genesis block's will now error out;
• Fixed a bug which would lead to the Logger throwing when being called from SCP in some rare cases;
• Fixed a bug which would crash the node (abort) if the Logger threw an Exception;
• Libconsensus: The validator set now include the validator's preimage;
• The SCP integration has been overhaul and greatly simplified, removing a lot of dead code;
• Nodes will only start gossipping transactions if they have a validator peer to avoid tx getting stuck;
• Nodes will now always keep connection to validators, regardless of min_peers;
• FullNode API now includes a handshake method which replaces getPublicKey as handshake method;
• Talos: Correctly show the error message if Agora returns an error;
• Talos: Update the config format to match current format;
• Stoa handlers have been combined in a single module;
• Outgoing requests now support proxy;
• Fixed a bug in the nomination protocol where a vote would change from yes to no when there were nodes to slash;
• Signatures can now only be added for the blocks that have been produced since the last payment;
• The bitfield in the BlockHeader used to record validators have been modified to unambiguously store the length;
• Fixed a DoS vector where a node could crash when asked invalid data;
• SCP now uses the UTXO as node ID instead of public key;
• Stoa: Block headers are now pushed when a new signature is received (block_header_updated handler);
• Fixed a DoS attack where a node would crash when receiving a specifically crafted signature;
• Fixed a DoS attack where a node would crash upon receiving alphanumeric but non-hex characters in large binary values;
• Added Swagger API specification to allow fuzzing the node;
• Added opt-out fuzzing component to the network integration tests;
• Talos: Improved dependency tree to reduce security risks;
• Talos: Fixed vulnerability with target=blank links;
• Talos: Improved first screen UX and fixed network address parsing;
• Talos now requires node 16;
• Flash: Add on-chain fees related endpoints to support fees in uncollaborating close attempts;
• Fixed a bug where signature catchup was dependent on the order of the returned items;
• Changed the admin interface methods (loginQR, encryptionKeyQR) to be GET instead of POST;
• Pre-images are now shared during SCP nomination to reduce the odds of slashing;
• Removed FullNode's getPreimage endpoint, obsoleted by getPreimages;

v0.12.0: Better stats, `TxBuilder` improvements and bugfix, `/validators` endpoint

Major improvements since v0.11.0:

• The minimum required version of LDC / DUB are now 1.26.0 and 1.25.0, respectively;
• The genesis timestamp can now be configured as consensus.genesis_timestamp;
• Configuring validator.slash_penalty_amount is no longer supported;
• The stats now include the genesis timestamp;
• TxBuilder has been simplified, with multiple parameters of sign now being functions;
• Flash: The remaining funds in a funding tx will now be properly credited back;
• Registry payload that fail signature validation will now be more verbose and prominent;
• A bug where the wrong key was used to verify the signature of a registry payload,
  leading to rejecting valid registry payload, has been fixed.
• All endpoints now provide endpoint stats (some were previously missing);
• Nodes now expose GET /validators[?height=$height] to get the list of validators at a given height;
• TxBuilder now properly sorts the inputs, fixing a bug where it would sign the wrong input;

Along with many internal changes to improve efficiency and reduce the risk of error.

v0.11.0: Better Flash notifications, runtime logger configuration, SCP cleanup

Major changes since v0.10.0:

• Flash: Discovery now includes Channel updates;
• Flash: Channels are now gossipped to new peers;
• Flash: Fixed a possible race condition between state change and notification to listener;
• Quorum balancing: The hash used as seed is now that of the externalized block, instead of the ledger state;
• TxBuilder now correctly orders inputs when creating a transaction;
• The SCP internals have been revamped, leading to lower memory footprint and easier integration;
• Validator.receiveEnvelope now longer blocks, it uses a queue for processing envelope asynchronously instead;
• Loggers can now be dynamically reconfigured through an endpoint in the admin interface;
• Serialization now always serialize to little endian regardless of the host;
• Multiple stability improvements / race conditions fixed;

v0.10.0: Extra syntax checks on lock script, improved Votera support, HTTPS

Major changes since v0.9.0:

• Admin interface now use the right pre-image to generate the encryption key for Votera;
• Transaction payload is now a simple array instead of a struct (breaking change for JSON serialization);
• Agora will now perform additional syntaxical validation on the lock script to ensure it is not malformed;
• Votera: Admin interface now include signature and public key with voter card;
• Disabled a new upstream behavior to prevent double destruction of GC-allocated object;
• Better diagnostics will be provided when Agora fails to start for abnormal reasons,
  such as corrupted disk state;
• Logger configuration will now propagate to child logger by default;
• Discovery / registry registration task is now more efficient;
• Support for HTTPS communication has been added;
• Consensus: Validators will now reject blocks with non-monotonic missing validators;