Add support for TurboSHAKE128 and TurboSHAKE256

Changelog.rst

@@ -1,6 +1,13 @@
 Changelog
 =========
 
+Under Development
+++++++++++++++++++++++++++
+
+New features
+---------------
+* Addde support for TurboSHAKE128 and TurboSHAKE256.
+
 3.19.1 (28 December 2023)
 ++++++++++++++++++++++++++
 

Doc/src/features.rst

@@ -49,7 +49,7 @@ A list of useful resources in that area can be found on `Matthew Green's blog`_.
   - SHA-2 hashes (224, 256, 384, 512, 512/224, 512/256)
   - SHA-3 hashes (224, 256, 384, 512) and XOFs (SHAKE128, SHAKE256)
   - Functions derived from SHA-3 (cSHAKE128, cSHAKE256, TupleHash128, TupleHash256)
-  - KangarooTwelve (XOF)
+  - KangarooTwelve, TurboSHAKE128, TurboSHAKE256 (XOF)
   - Keccak (original submission to SHA-3)
   - BLAKE2b and BLAKE2s
   - RIPE-MD160 (legacy)

Doc/src/hash/hash.rst

@@ -138,6 +138,7 @@ Instead, it has a ``read(N)`` method to extract the next ``N`` bytes of the outp
     - :doc:`cshake256`
 
 - :doc:`k12`
+- :doc:`turboshake`
 
 Message Authentication Code (MAC) algorithms
 --------------------------------------------

Doc/src/hash/turboshake.rst

@@ -0,0 +1,60 @@
+TurboSHAKE128 and TurboSHAKE256
+===============================
+
+TurboSHAKE is a family of *eXtendable-Output Functions* (XOFs) based on the Keccak permutation,
+which is also the basis for SHA-3.
+
+A XOF is a generalization of a cryptographic hash.
+The output digest of a XOF can take any length, as required by the caller,
+unlike SHA-256 (for instance) that always produces exactly 32 bytes.
+The output bits of a XOF do **not** depend on the output length,
+which means that the output length
+does not even need to be known (or declared) when the XOF is created.
+
+Therefore, a XOF object has a ``read(N: int)`` method (much like a file object)
+instead of a ``digest()`` method. ``read()`` can be called any number of times,
+and it will return different bytes each time.
+
+.. figure:: xof.png
+    :align: center
+    :figwidth: 50%
+
+    Generic state diagram for a XOF object
+
+The TurboSHAKE family is not standardized. However, an RFC_ is being written.
+It comprises of two members:
+
+.. csv-table::
+   :header: Name, (2nd) Pre-image strength, Collision strength
+
+   TurboSHAKE128, "128 bits (output >= 16 bytes)", "256 bits (output >= 32 bytes)"
+   TurboSHAKE256, "256 bits (output >= 32 bytes)", "512 bits (output >= 64 bytes)"
+
+In addition to hashing, TurboSHAKE allows for domain separation
+via a *domain separation byte* (that is, the ``domain`` parameter to :func:`Crypto.Hash.TurboSHAKE128.new`
+and to :func:`Crypto.Hash.TurboSHAKE256.new`).
+
+.. hint::
+
+  For instance, if you are using TurboSHAKE in two applications,
+  by picking different domain separation bytes you can ensure
+  that they will never end up using the same digest in practice.
+  The important factor is that the strings are different;
+  the actual value of the domain separation byte is irrelevant.
+
+In the following example, we extract 26 bytes (208 bits) from the TurboSHAKE128 XOF::
+
+    >>> from Crypto.Hash import TurboSHAKE128
+    >>>
+    >>> xof = TurboSHAKE128.new()
+    >>> xof.update(b'Some data')
+    >>> print(xof.read(26).hex())
+    d9dfade4ff8be344749908073916d3abd185ef88f5401024f029
+
+.. _RFC: https://datatracker.ietf.org/doc/draft-irtf-cfrg-kangarootwelve/
+
+.. automodule:: Crypto.Hash.TurboSHAKE128
+    :members:
+
+.. automodule:: Crypto.Hash.TurboSHAKE256
+    :members:

README.rst

@@ -58,7 +58,7 @@ with respect to the last official version of PyCrypto (2.6.1):
   - KMAC128 and KMAC256
   - TupleHash128 and TupleHash256
 
-* KangarooTwelve XOF (derived from Keccak)
+* KangarooTwelve, TurboSHAKE128, and TurboSHAKE256 XOFs
 * Truncated hash algorithms SHA-512/224 and SHA-512/256 (FIPS 180-4)
 * BLAKE2b and BLAKE2s hash algorithms
 * Salsa20 and ChaCha20/XChaCha20 stream ciphers

lib/Crypto/Hash/KangarooTwelve.py

@@ -28,16 +28,10 @@
 # POSSIBILITY OF SUCH DAMAGE.
 # ===================================================================
 
-from Crypto.Util._raw_api import (VoidPointer, SmartPointer,
-                                  create_string_buffer,
-                                  get_raw_buffer, c_size_t,
-                                  c_uint8_ptr, c_ubyte)
-
 from Crypto.Util.number import long_to_bytes
 from Crypto.Util.py3compat import bchr
 
-from .keccak import _raw_keccak_lib
-
+from . import TurboSHAKE128
 
 def _length_encode(x):
     if x == 0:
@@ -70,7 +64,8 @@ def __init__(self, data, custom):
         self._padding = None        # Final padding is only decided in read()
 
         # Internal hash that consumes FinalNode
-        self._hash1 = self._create_keccak()
+        # The real domain separation byte will be known before squeezing
+        self._hash1 = TurboSHAKE128.new(domain=1)
         self._length1 = 0
 
         # Internal hash that produces CV_i (reset each time)
@@ -83,42 +78,6 @@ def __init__(self, data, custom):
         if data:
             self.update(data)
 
-    def _create_keccak(self):
-        state = VoidPointer()
-        result = _raw_keccak_lib.keccak_init(state.address_of(),
-                                             c_size_t(32),  # 32 bytes of capacity (256 bits)
-                                             c_ubyte(12))   # Reduced number of rounds
-        if result:
-            raise ValueError("Error %d while instantiating KangarooTwelve"
-                             % result)
-        return SmartPointer(state.get(), _raw_keccak_lib.keccak_destroy)
-
-    def _update(self, data, hash_obj):
-        result = _raw_keccak_lib.keccak_absorb(hash_obj.get(),
-                                               c_uint8_ptr(data),
-                                               c_size_t(len(data)))
-        if result:
-            raise ValueError("Error %d while updating KangarooTwelve state"
-                             % result)
-
-    def _squeeze(self, hash_obj, length, padding):
-        bfr = create_string_buffer(length)
-        result = _raw_keccak_lib.keccak_squeeze(hash_obj.get(),
-                                                bfr,
-                                                c_size_t(length),
-                                                c_ubyte(padding))
-        if result:
-            raise ValueError("Error %d while extracting from KangarooTwelve"
-                             % result)
-
-        return get_raw_buffer(bfr)
-
-    def _reset(self, hash_obj):
-        result = _raw_keccak_lib.keccak_reset(hash_obj.get())
-        if result:
-            raise ValueError("Error %d while resetting KangarooTwelve state"
-                             % result)
-
     def update(self, data):
         """Hash the next piece of data.
 
@@ -127,7 +86,7 @@ def update(self, data):
 
         Args:
             data (byte string/byte array/memoryview): The next chunk of the
-            message to hash.
+              message to hash.
         """
 
         if self._state == SQUEEZING:
@@ -138,7 +97,7 @@ def update(self, data):
 
             if next_length + len(self._custom) <= 8192:
                 self._length1 = next_length
-                self._update(data, self._hash1)
+                self._hash1.update(data)
                 return self
 
             # Switch to tree hashing
@@ -148,7 +107,7 @@ def update(self, data):
             data_mem = memoryview(data)
             assert(self._length1 < 8192)
             dtc = min(len(data), 8192 - self._length1)
-            self._update(data_mem[:dtc], self._hash1)
+            self._hash1.update(data_mem[:dtc])
             self._length1 += dtc
 
             if self._length1 < 8192:
@@ -158,10 +117,10 @@ def update(self, data):
             assert(self._length1 == 8192)
 
             divider = b'\x03' + b'\x00' * 7
-            self._update(divider, self._hash1)
+            self._hash1.update(divider)
             self._length1 += 8
 
-            self._hash2 = self._create_keccak()
+            self._hash2 = TurboSHAKE128.new(domain=0x0B)
             self._length2 = 0
             self._ctr = 1
 
@@ -178,15 +137,15 @@ def update(self, data):
         while index < len_data:
 
             new_index = min(index + 8192 - self._length2, len_data)
-            self._update(data_mem[index:new_index], self._hash2)
+            self._hash2.update(data_mem[index:new_index])
             self._length2 += new_index - index
             index = new_index
 
             if self._length2 == 8192:
-                cv_i = self._squeeze(self._hash2, 32, 0x0B)
-                self._update(cv_i, self._hash1)
+                cv_i = self._hash2.read(32)
+                self._hash1.update(cv_i)
                 self._length1 += 32
-                self._reset(self._hash2)
+                self._hash2._reset()
                 self._length2 = 0
                 self._ctr += 1
 
@@ -210,7 +169,7 @@ def read(self, length):
         custom_was_consumed = False
 
         if self._state == SHORT_MSG:
-            self._update(self._custom, self._hash1)
+            self._hash1.update(self._custom)
             self._padding = 0x07
             self._state = SQUEEZING
 
@@ -225,20 +184,21 @@ def read(self, length):
 
             # Is there still some leftover data in hash2?
             if self._length2 > 0:
-                cv_i = self._squeeze(self._hash2, 32, 0x0B)
-                self._update(cv_i, self._hash1)
+                cv_i = self._hash2.read(32)
+                self._hash1.update(cv_i)
                 self._length1 += 32
-                self._reset(self._hash2)
+                self._hash2._reset()
                 self._length2 = 0
                 self._ctr += 1
 
             trailer = _length_encode(self._ctr - 1) + b'\xFF\xFF'
-            self._update(trailer, self._hash1)
+            self._hash1.update(trailer)
 
             self._padding = 0x06
             self._state = SQUEEZING
 
-        return self._squeeze(self._hash1, length, self._padding)
+        self._hash1._domain = self._padding
+        return self._hash1.read(length)
 
     def new(self, data=None, custom=b''):
         return type(self)(data, custom)

lib/Crypto/Hash/TupleHash128.py

@@ -133,7 +133,4 @@ def new(**kwargs):
 
     custom = kwargs.pop("custom", b'')
 
-    if kwargs:
-        raise TypeError("Unknown parameters: " + str(kwargs))
-
     return TupleHash(custom, cSHAKE128, digest_bytes)

lib/Crypto/Hash/TupleHash256.py

@@ -67,7 +67,4 @@ def new(**kwargs):
 
     custom = kwargs.pop("custom", b'')
 
-    if kwargs:
-        raise TypeError("Unknown parameters: " + str(kwargs))
-
     return TupleHash(custom, cSHAKE256, digest_bytes)

lib/Crypto/Hash/TurboSHAKE128.py

@@ -0,0 +1,112 @@
+from Crypto.Util._raw_api import (VoidPointer, SmartPointer,
+                                  create_string_buffer,
+                                  get_raw_buffer, c_size_t,
+                                  c_uint8_ptr, c_ubyte)
+
+from Crypto.Util.number import long_to_bytes
+from Crypto.Util.py3compat import bchr
+
+from .keccak import _raw_keccak_lib
+
+
+class TurboSHAKE(object):
+    """A TurboSHAKE hash object.
+    Do not instantiate directly.
+    Use the :func:`new` function.
+    """
+
+    def __init__(self, capacity, domain_separation, data):
+
+        state = VoidPointer()
+        result = _raw_keccak_lib.keccak_init(state.address_of(),
+                                             c_size_t(capacity),
+                                             c_ubyte(12))   # Reduced number of rounds
+        if result:
+            raise ValueError("Error %d while instantiating TurboSHAKE"
+                             % result)
+        self._state = SmartPointer(state.get(), _raw_keccak_lib.keccak_destroy)
+
+        self._is_squeezing = False
+        self._capacity = capacity
+        self._domain = domain_separation
+
+        if data:
+            self.update(data)
+
+
+    def update(self, data):
+        """Continue hashing of a message by consuming the next chunk of data.
+
+        Args:
+            data (byte string/byte array/memoryview): The next chunk of the message being hashed.
+        """
+
+        if self._is_squeezing:
+            raise TypeError("You cannot call 'update' after the first 'read'")
+
+        result = _raw_keccak_lib.keccak_absorb(self._state.get(),
+                                               c_uint8_ptr(data),
+                                               c_size_t(len(data)))
+        if result:
+            raise ValueError("Error %d while updating TurboSHAKE state"
+                             % result)
+        return self
+
+    def read(self, length):
+        """
+        Compute the next piece of XOF output.
+
+        .. note::
+            You cannot use :meth:`update` anymore after the first call to
+            :meth:`read`.
+
+        Args:
+            length (integer): the amount of bytes this method must return
+
+        :return: the next piece of XOF output (of the given length)
+        :rtype: byte string
+        """
+
+        self._is_squeezing = True
+        bfr = create_string_buffer(length)
+        result = _raw_keccak_lib.keccak_squeeze(self._state.get(),
+                                                bfr,
+                                                c_size_t(length),
+                                                c_ubyte(self._domain))
+        if result:
+            raise ValueError("Error %d while extracting from TurboSHAKE"
+                             % result)
+
+        return get_raw_buffer(bfr)
+
+    def new(self, data=None):
+        return type(self)(self._capacity, self._domain, data)
+
+    def _reset(self):
+        result = _raw_keccak_lib.keccak_reset(self._state.get())
+        if result:
+            raise ValueError("Error %d while resetting TurboSHAKE state"
+                             % result)
+        self._is_squeezing = False
+
+
+def new(**kwargs):
+    """Create a new TurboSHAKE128 object.
+
+    Args:
+       domain (integer):
+         Optional - A domain separation byte, between 0x01 and 0x7F.
+         The default value is 0x1F.
+       data (bytes/bytearray/memoryview):
+        Optional - The very first chunk of the message to hash.
+        It is equivalent to an early call to :meth:`update`.
+
+    :Return: A :class:`TurboSHAKE` object
+    """
+
+    domain_separation = kwargs.get('domain', 0x1F)
+    if not (0x01 <= domain_separation <= 0x7F):
+        raise ValueError("Incorrect domain separation value (%d)" %
+                         domain_separation)
+    data = kwargs.get('data')
+    return TurboSHAKE(32, domain_separation, data=data)