Add support for transparent-source-only (TEX) addresses

zcash_client_backend/CHANGELOG.md

@@ -6,18 +6,38 @@ and this library adheres to Rust's notion of
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+### Notable changes
+`zcash_client_backend` now supports TEX (transparent-source-only) addresses as specified
+in ZIP 320. Sending to one or more TEX addresses will automatically create a multi-step
+proposal that uses two transactions.
+
+In order to take advantage of this support, client wallets will need to be able to send
+multiple transactions created from `zcash_client_backend::data_api::wallet::create_proposed_transactions`.
+This API was added in `zcash_client_backend` 0.11.0 but previously could only return a
+single transaction.
+
+**Note:** This feature changes the use of transparent addresses in ways that are relevant
+to security and access to funds, and that may interact with other wallet behaviour. In
+particular it exposes new ephemeral transparent addresses belonging to the wallet, which
+need to be scanned in order to recover funds if the first transaction of the proposal is
+mined but the second is not, or if someone (e.g. the TEX-address recipient) sends back
+funds to those addresses. See [ZIP 320](https://zips.z.cash/zip-0320) for details.
 
 ### Added
 - `zcash_client_backend::data_api`:
   - `chain::BlockCache` trait, behind the `sync` feature flag.
   - `WalletRead::get_spendable_transparent_outputs`.
 - `zcash_client_backend::fees`:
-  - `ChangeValue::{transparent, shielded}`
+  - `ChangeValue::shielded, is_ephemeral`
+  - `ChangeValue::ephemeral_transparent` (when "transparent-inputs" is enabled)
   - `sapling::EmptyBundleView`
   - `orchard::EmptyBundleView`
+- `zcash_client_backend::proposal`:
+  - `impl Hash for {StepOutput, StepOutputIndex}`
 - `zcash_client_backend::scanning`:
   - `testing` module
-- `zcash_client_backend::sync` module, behind the `sync` feature flag
+- `zcash_client_backend::sync` module, behind the `sync` feature flag.
+- `zcash_client_backend::wallet::Recipient::map_ephemeral_transparent_outpoint`
 
 ### Changed
 - MSRV is now 1.70.0.
@@ -28,36 +48,47 @@ and this library adheres to Rust's notion of
   `change_memo` is given, and defends against losing money by using
   `DustAction::AddDustToFee` with a too-high dust threshold.
   See [#1430](https://github.com/zcash/librustzcash/pull/1430) for details.
-- `zcash_client_backend::zip321` has been extracted to, and is now a reexport 
+- `zcash_client_backend::zip321` has been extracted to, and is now a reexport
   of the root module of the `zip321` crate. Several of the APIs of this module
   have changed as a consequence of this extraction; please see the `zip321`
   CHANGELOG for details.
 - `zcash_client_backend::data_api`:
+  - `WalletRead` has new `get_reserved_ephemeral_addresses` and
+    `get_transparent_address_metadata` methods.
+  - `WalletWrite` has a new `reserve_next_n_ephemeral_addresses` method.
   - `error::Error` has a new `Address` variant.
   - `wallet::input_selection::InputSelectorError` has a new `Address` variant.
 - `zcash_client_backend::proto::proposal::Proposal::{from_standard_proposal, 
   try_into_standard_proposal}` each no longer require a `consensus::Parameters` 
   argument.
 - `zcash_client_backend::data_api::fees`
-  - The return type of `ChangeValue::output_pool`, and the type of the
-    `output_pool` argument to `ChangeValue::new`, have changed from
-    `ShieldedProtocol` to `zcash_protocol::PoolType`.
-  - The return type of `ChangeValue::new` is now optional; it returns `None`
-    if a memo is given for the transparent pool. Use `ChangeValue::shielded`
-    to avoid this error case when creating a `ChangeValue` known to be for a
-    shielded pool.
+  - When the "transparent-inputs" feature is enabled, `ChangeValue` can also
+    represent an ephemeral transparent output in a proposal. Accordingly, the
+    return type of `ChangeValue::output_pool` has (unconditionally) changed
+    from `ShieldedProtocol` to `zcash_protocol::PoolType`.
+  - `ChangeStrategy::compute_balance`: this trait method has an additional
+    `&EphemeralParameters` parameter when the "transparent-inputs" feature is
+    enabled. This can be used to specify whether the change memo should be
+    ignored, and the amounts of additional transparent P2PKH inputs and
+    outputs. Passing `&Default::default()` will retain the previous behaviour.
 - `zcash_client_backend::input_selection::GreedyInputSelectorError` has a
   new variant `UnsupportedTexAddress`.
+- `zcash_client_backend::proto::ProposalDecodingError` has a new variant
+  `InvalidEphemeralRecipient`.
 - `zcash_client_backend::wallet::Recipient` variants have changed. Instead of
-  wrapping protocol-address types, the `Recipient` type now wraps a
-  `zcash_address::ZcashAddress`. This simplifies the process of tracking the
-  original address to which value was sent.
+  wrapping protocol-address types, the `External` and `InternalAccount` variants
+  now wrap a `zcash_address::ZcashAddress`. This simplifies the process of
+  tracking the original address to which value was sent. There is also a new
+  `EphemeralTransparent` variant, and an additional generic parameter for the
+  type of metadata associated with an ephemeral transparent outpoint.
 
 ### Removed
 - `zcash_client_backend::data_api`:
   - `WalletRead::get_unspent_transparent_outputs` has been removed because its
     semantics were unclear and could not be clarified. Use
     `WalletRead::get_spendable_transparent_outputs` instead.
+- `zcash_client_backend::fees::ChangeValue::new`. Use `ChangeValue::shielded`
+  or `ChangeValue::ephemeral_transparent` instead.
 
 ## [0.12.1] - 2024-03-27
 

zcash_client_backend/proto/proposal.proto

@@ -73,14 +73,14 @@ message ReceivedOutput {
     uint64 value = 4;
 }
 
-// A reference a payment in a prior step of the proposal. This payment must
+// A reference to a payment in a prior step of the proposal. This payment must
 // belong to the wallet.
 message PriorStepOutput {
     uint32 stepIndex = 1;
     uint32 paymentIndex = 2;
 }
 
-// A reference a change output from a prior step of the proposal.
+// A reference to a change or ephemeral output from a prior step of the proposal.
 message PriorStepChange {
     uint32 stepIndex = 1;
     uint32 changeIndex = 2;
@@ -112,22 +112,29 @@ enum FeeRule {
 
 // The proposed change outputs and fee value.
 message TransactionBalance {
-    // A list of change output values.
+    // A list of change or ephemeral output values.
     repeated ChangeValue proposedChange = 1;
     // The fee to be paid by the proposed transaction, in zatoshis.
     uint64 feeRequired = 2;
 }
 
-// A proposed change output. If the transparent value pool is selected,
-// the `memo` field must be null.
+// A proposed change or ephemeral output. If the transparent value pool is
+// selected, the `memo` field must be null.
+//
+// When the `isEphemeral` field of a `ChangeValue` is set, it represents
+// an ephemeral output, which must be spent by a subsequent step. This is
+// only supported for transparent outputs. Each ephemeral output will be
+// given a unique t-address.
 message ChangeValue {
-    // The value of a change output to be created, in zatoshis.
+    // The value of a change or ephemeral output to be created, in zatoshis.
     uint64 value = 1;
-    // The value pool in which the change output should be created.
+    // The value pool in which the change or ephemeral output should be created.
     ValuePool valuePool = 2;
-    // The optional memo that should be associated with the newly created change output.
-    // Memos must not be present for transparent change outputs.
+    // The optional memo that should be associated with the newly created output.
+    // Memos must not be present for transparent outputs.
     MemoBytes memo = 3;
+    // Whether this is to be an ephemeral output.
+    bool isEphemeral = 4;
 }
 
 // An object wrapper for memo bytes, to facilitate representing the

zcash_client_backend/src/data_api.rs

@@ -88,16 +88,16 @@
     consensus::BlockHeight,
     memo::{Memo, MemoBytes},
     transaction::{
-        components::amount::{BalanceError, NonNegativeAmount},
+        components::{
+            amount::{BalanceError, NonNegativeAmount},
+            OutPoint,
+        },
         Transaction, TxId,
     },
 };
 
 #[cfg(feature = "transparent-inputs")]
-use {
-    crate::wallet::TransparentAddressMetadata,
-    zcash_primitives::{legacy::TransparentAddress, transaction::components::OutPoint},
-};
+use {crate::wallet::TransparentAddressMetadata, zcash_primitives::legacy::TransparentAddress};
 
 #[cfg(any(test, feature = "test-dependencies"))]
 use zcash_primitives::consensus::NetworkUpgrade;
@@ -890,10 +890,14 @@
         query: NullifierQuery,
     ) -> Result<Vec<(Self::AccountId, orchard::note::Nullifier)>, Self::Error>;
 
-    /// Returns the set of all transparent receivers associated with the given account.
+    /// Returns the set of non-ephemeral transparent receivers associated with the given
+    /// account controlled by this wallet.
+    ///
+    /// The set contains all non-ephemeral transparent receivers that are known to have
+    /// been derived under this account. Wallets should scan the chain for UTXOs sent to
+    /// these receivers.
     ///
-    /// The set contains all transparent receivers that are known to have been derived
-    /// under this account. Wallets should scan the chain for UTXOs sent to these
+    /// Use [`Self::get_reserved_ephemeral_addresses`] to obtain the ephemeral transparent
     /// receivers.
     #[cfg(feature = "transparent-inputs")]
     fn get_transparent_receivers(
@@ -903,8 +907,11 @@
         Ok(HashMap::new())
     }
 
-    /// Returns a mapping from transparent receiver to not-yet-shielded UTXO balance,
-    /// for each address associated with a nonzero balance.
+    /// Returns a mapping from each transparent receiver associated with the specified account
+    /// to its not-yet-shielded UTXO balance as of the end of the block at the provided
+    /// `max_height`, when that balance is non-zero.
+    ///
+    /// Balances of ephemeral transparent addresses will not be included.
     #[cfg(feature = "transparent-inputs")]
     fn get_transparent_balances(
         &self,
@@ -913,6 +920,78 @@
     ) -> Result<HashMap<TransparentAddress, NonNegativeAmount>, Self::Error> {
         Ok(HashMap::new())
     }
+
+    /// Returns the metadata associated with a given transparent receiver in an account
+    /// controlled by this wallet.
+    ///
+    /// This is equivalent to (but may be implemented more efficiently than):
+    /// ```compile_fail
+    /// if let Some(result) = self.get_transparent_receivers(account)?.get(address) {
+    ///     return Ok(result.clone());
+    /// }
+    /// if let Some(result) = self.get_reserved_ephemeral_addresses(account, false)?.get(address) {
+    ///     return Ok(result.clone());
+    /// }
+    /// Ok(None)
+    /// ```
+    ///
+    /// Returns `Ok(None)` if the address is not recognized, or we do not have metadata for it.
+    /// Returns `Ok(Some(metadata))` if we have the metadata.
+    #[cfg(feature = "transparent-inputs")]
+    fn get_transparent_address_metadata(
+        &self,
+        account: Self::AccountId,
+        address: &TransparentAddress,
+    ) -> Result<Option<TransparentAddressMetadata>, Self::Error> {
+        // This should be overridden.
+        if let Some(result) = self.get_transparent_receivers(account)?.get(address) {
+            return Ok(result.clone());
+        }
+        if let Some(result) = self
+            .get_reserved_ephemeral_addresses(account, false)?
+            .get(address)
+        {
+            return Ok(result.clone());
+        }
+        Ok(None)
+    }
+
+    /// Returns the set of reserved ephemeral transparent addresses associated with the
+    /// given account controlled by this wallet.
+    ///
+    /// The set contains all ephemeral transparent receivers that are known to have
+    /// been derived under this account. Wallets should scan the chain for UTXOs sent to
+    /// these receivers, but do not need to do so regularly. Under expected usage, outputs
+    /// would only be detected with these receivers in the following situations:
+    ///
+    /// - This wallet created a payment to a ZIP 320 (TEX) address, but the second
+    ///   transaction (that spent the output sent to the ephemeral address) did not get
+    ///   mined before it expired.
+    ///   - In this case the output will already be known to the wallet (because it
+    ///     stores the transactions that it creates).
+    ///
+    /// - Another wallet app using the same seed phrase created a payment to a ZIP 320
+    ///   address, and this wallet queried for the ephemeral UTXOs after the first
+    ///   transaction was mined but before the second transaction was mined.
+    ///   - In this case, the output should not be considered unspent until the expiry
+    ///     height of the transaction it was received in has passed. Wallets creating
+    ///     payments to TEX addresses generally set the same expiry height for the first
+    ///     and second transactions, meaning that this wallet does not need to observe
+    ///     the second transaction to determine when it would have expired.
+    ///
+    /// - A TEX address recipient decided to return funds that the wallet had sent to
+    ///   them.
+    ///
+    /// In all cases, the wallet should re-shield the unspent outputs, in a separate
+    /// transaction per ephemeral address, before re-spending the funds.
+    #[cfg(feature = "transparent-inputs")]
+    fn get_reserved_ephemeral_addresses(
+        &self,
+        _account: Self::AccountId,
+        _for_detection: bool,
+    ) -> Result<HashMap<TransparentAddress, Option<TransparentAddressMetadata>>, Self::Error> {
+        Ok(HashMap::new())
+    }
 }
 
 /// The relevance of a seed to a given wallet.
@@ -1243,7 +1322,7 @@
 /// This type is capable of representing both shielded and transparent outputs.
 pub struct SentTransactionOutput<AccountId> {
     output_index: usize,
-    recipient: Recipient<AccountId, Note>,
+    recipient: Recipient<AccountId, Note, OutPoint>,
     value: NonNegativeAmount,
     memo: Option<MemoBytes>,
 }
@@ -1260,7 +1339,7 @@
     /// * `memo` - the memo that was sent with this output
     pub fn from_parts(
         output_index: usize,
-        recipient: Recipient<AccountId, Note>,
+        recipient: Recipient<AccountId, Note, OutPoint>,
         value: NonNegativeAmount,
         memo: Option<MemoBytes>,
     ) -> Self {
@@ -1282,8 +1361,8 @@
         self.output_index
     }
     /// Returns the recipient address of the transaction, or the account id and
-    /// resulting note for wallet-internal outputs.
-    pub fn recipient(&self) -> &Recipient<AccountId, Note> {
+    /// resulting note/outpoint for wallet-internal outputs.
+    pub fn recipient(&self) -> &Recipient<AccountId, Note, OutPoint> {
         &self.recipient
     }
     /// Returns the value of the newly created output.
@@ -1542,6 +1621,28 @@
     ///
     /// There may be restrictions on heights to which it is possible to truncate.
     fn truncate_to_height(&mut self, block_height: BlockHeight) -> Result<(), Self::Error>;
+
+    /// Reserves the next `n` available ephemeral addresses for the given account.
+    /// This cannot be undone, so as far as possible, errors associated with transaction
+    /// construction should have been reported before calling this method.
+    ///
+    /// To ensure that sufficient information is stored on-chain to allow recovering
+    /// funds sent back to any of the used addresses, a "gap limit" of 20 addresses
+    /// should be observed as described in [BIP 44].
+    ///
+    /// Returns an error if there is insufficient space within the gap limit to allocate
+    /// the given number of addresses, or if the account identifier does not correspond
+    /// to a known account.
+    ///
+    /// Precondition: `n < 0x80000000`
+    ///
+    /// [BIP 44]: https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki#user-content-Address_gap_limit
+    #[cfg(feature = "transparent-inputs")]
+    fn reserve_next_n_ephemeral_addresses(
+        &mut self,
+        account_id: Self::AccountId,
+        n: u32,
+    ) -> Result<Vec<(TransparentAddress, TransparentAddressMetadata)>, Self::Error>;
 }
 
 /// This trait describes a capability for manipulating wallet note commitment trees.
@@ -1869,6 +1970,25 @@
         ) -> Result<HashMap<TransparentAddress, NonNegativeAmount>, Self::Error> {
             Ok(HashMap::new())
         }
+
+        #[cfg(feature = "transparent-inputs")]
+        fn get_transparent_address_metadata(
+            &self,
+            _account: Self::AccountId,
+            _address: &TransparentAddress,
+        ) -> Result<Option<TransparentAddressMetadata>, Self::Error> {
+            Ok(None)
+        }
+
+        #[cfg(feature = "transparent-inputs")]
+        fn get_reserved_ephemeral_addresses(
+            &self,
+            _account: Self::AccountId,
+            _for_detection: bool,
+        ) -> Result<HashMap<TransparentAddress, Option<TransparentAddressMetadata>>, Self::Error>
+        {
+            Ok(HashMap::new())
+        }
     }
 
     impl WalletWrite for MockWalletDb {
@@ -1931,6 +2051,16 @@
         ) -> Result<Self::UtxoRef, Self::Error> {
             Ok(0)
         }
+
+        #[cfg(feature = "transparent-inputs")]
+        fn reserve_next_n_ephemeral_addresses(
+            &mut self,
+            _account_id: Self::AccountId,
+            n: u32,
+        ) -> Result<Vec<(TransparentAddress, TransparentAddressMetadata)>, Self::Error> {
+            assert!(n < 0x80000000);
+            Err(())
+        }
     }
 
     impl WalletCommitmentTrees for MockWalletDb {