feature: hasher -> update -> finalize paradigm

[Dustin-Ray]

Similar to openSSL, the Message type should have traits that produce digests only when specifically asked. For instance, the Message should be able to absorb new data at any time, and produce a sha3 digest only when called.

Ex:

let msg = Message::new("some data")
msg.update("more data")
msg.update("even more data")
let digest = msg.finalize

or something similar

[Dustin-Ray]

the following equality should hold:

hasher = Message::new("some data")
hasher.update("more data")
hasher.finalize()

should be the same digest as:

Hash("some data" + "more data")

[Dustin-Ray]

Check out this exact version of what we need from kangaroo12:

// Hash an input incrementally.
let mut hasher = kangarootwelve_xkcp::Hasher::new();
hasher.update(b"foo");
hasher.update(b"bar");
hasher.update(b"baz");
assert_eq!(hasher.finalize(), kangarootwelve_xkcp::hash(b"foobarbaz"));

[Dustin-Ray]

the other change that needs to happen is that padtenone padding shouldnt be applied until finalize is called. thats an easy fix that just moves the padding into a different function. but basically the sponge should just keep absorbing stuff as long as it needs to and not worry about the padding until finalize is called

[Dustin-Ray]

Actually now that I think about it, the sponge is fine where it is. What we need to do instead is actually just update Message to support this new functionality, and actually this greatly simplifies things.

In lib.rs:

• define a new trait called UpdateFinalize:

pub trait UpdateFinalize {
    // it returns nothing and simply appends the write data into self.data, thats it
    fn update(self, write_data: &[u8]);
    // internally it calls compute_sha3_hash, passing in self.data and returns the result
    fn finalize(self) -> Vec<u8>;
}

in ops.rs

• implement the trait for Message:

impl UpdateFinalize for Message {
    fn update();
    fn finalize();
}

This is far simpler to carry out this way and aligns well with the rest of how the library is constructed. Some notes:
1, these details are simplified but this is the correct approach, it uses existing code so theres not a whole lot that needs to change
2. we will need to figure out how finalize should return output. For now, its ok to just return the entire output of cshake. but to solve issue #20 we will need to make finalize maintain some state and produce a single block instead. but for now, we dont need to worry about that

[Dustin-Ray]

What does this do and why is it useful?

this will eventually give us this syntax:

let m = Message::new("Initial data");
m.update("More data")
m.update("Even more data")

And then when we finalize, we essentially get the hash result of ("Initial dataMore dataEven more data")

This is useful for all sorts of applications in cryptography when we dont necessarily know the entire message that we need to process before hand. When we fix finalize to produce only a single block at a time, then we can use this to produce blocks one at a time to XOR into a message for decryption, instead of generating the entire key into memory at once.

this will effectively solve our issue of having to store the entire key which is equal to the length of the message into the heap

[Hyunggilwoo]

Actually now that I think about it, the sponge is fine where it is. What we need to do instead is actually just update Message to support this new functionality, and actually this greatly simplifies things.

In lib.rs:

• define a new trait called UpdateFinalize:

pub trait UpdateFinalize {
    // it returns nothing and simply appends the write data into self.data, thats it
    fn update(self, write_data: &[u8]);
    // internally it calls compute_sha3_hash, passing in self.data and returns the result
    fn finalize(self) -> Vec<u8>;
}

in ops.rs

• implement the trait for Message:

impl UpdateFinalize for Message {
    fn update();
    fn finalize();
}

This is far simpler to carry out this way and aligns well with the rest of how the library is constructed. Some notes: 1, these details are simplified but this is the correct approach, it uses existing code so theres not a whole lot that needs to change 2. we will need to figure out how finalize should return output. For now, its ok to just return the entire output of cshake. but to solve issue #20 we will need to make finalize maintain some state and produce a single block instead. but for now, we dont need to worry about that

I am just wondering if the signature of the fn update() should be fn update(&mut self, write_data: &[u8]) instead of fn update(self, write_data: &[u8]).

[Hyunggilwoo]

I notice that in the comments for fn finalize() should use fn compute_hash_sha3() implemented in impl Hashable for Message. I wonder if I can use the function from an impl A for Message into another function inside impl B for Message.

[Hyunggilwoo]

Check out this exact version of what we need from kangaroo12:

// Hash an input incrementally.
let mut hasher = kangarootwelve_xkcp::Hasher::new();
hasher.update(b"foo");
hasher.update(b"bar");
hasher.update(b"baz");
assert_eq!(hasher.finalize(), kangarootwelve_xkcp::hash(b"foobarbaz"));

When we are testing the impl UpdateFinalize for Message, then are we asserting the equality of an updated message versus a text that was hashed with fn cshake() in the ops.rs?

Are we trying to make this line of code work?

let mut m = Message::new();
m.update("foo");
m.update("bar");
m.update("baz");
assert_eq!(m.finalize(), vec!["foobarbaz"].compute_hash_sha3(&SecParam::D256));

[Hyunggilwoo]

Actually now that I think about it, the sponge is fine where it is. What we need to do instead is actually just update Message to support this new functionality, and actually this greatly simplifies things.
In lib.rs:

• define a new trait called UpdateFinalize:

pub trait UpdateFinalize {
    // it returns nothing and simply appends the write data into self.data, thats it
    fn update(self, write_data: &[u8]);
    // internally it calls compute_sha3_hash, passing in self.data and returns the result
    fn finalize(self) -> Vec<u8>;
}

in ops.rs

• implement the trait for Message:

impl UpdateFinalize for Message {
    fn update();
    fn finalize();
}

This is far simpler to carry out this way and aligns well with the rest of how the library is constructed. Some notes: 1, these details are simplified but this is the correct approach, it uses existing code so theres not a whole lot that needs to change 2. we will need to figure out how finalize should return output. For now, its ok to just return the entire output of cshake. but to solve issue #20 we will need to make finalize maintain some state and produce a single block instead. but for now, we dont need to worry about that

I am just wondering if the signature of the fn update() should be fn update(&mut self, write_data: &[u8]) instead of fn update(self, write_data: &[u8]).

KangarooTwelve uses a function signature of pub fn update(&mut self, input: &[u8]) and pub fn finalize(&mut self).

I will use a similar arguments for this fn update() and fn finalize().

[Hyunggilwoo]

we will need to figure out how finalize should return output. For now, its ok to just return the entire output of cshake. but to solve issue #20 we will need to make finalize maintain some state and produce a single block instead. but for now, we dont need to worry about that

Actually now that I think about it, the sponge is fine where it is. What we need to do instead is actually just update Message to support this new functionality, and actually this greatly simplifies things.

In lib.rs:

• define a new trait called UpdateFinalize:

pub trait UpdateFinalize {
    // it returns nothing and simply appends the write data into self.data, thats it
    fn update(self, write_data: &[u8]);
    // internally it calls compute_sha3_hash, passing in self.data and returns the result
    fn finalize(self) -> Vec<u8>;
}

in ops.rs

• implement the trait for Message:

impl UpdateFinalize for Message {
    fn update();
    fn finalize();
}

This is far simpler to carry out this way and aligns well with the rest of how the library is constructed. Some notes: 1, these details are simplified but this is the correct approach, it uses existing code so theres not a whole lot that needs to change 2. we will need to figure out how finalize should return output. For now, its ok to just return the entire output of cshake. but to solve issue #20 we will need to make finalize maintain some state and produce a single block instead. but for now, we dont need to worry about that

I see that fn finalize() should use cshake for now. I will reference the fn kmac_xof() that also returns cshake to see how cshake should be used in fn finalize().

[Hyunggilwoo]

Check out this exact version of what we need from kangaroo12:

// Hash an input incrementally.
let mut hasher = kangarootwelve_xkcp::Hasher::new();
hasher.update(b"foo");
hasher.update(b"bar");
hasher.update(b"baz");
assert_eq!(hasher.finalize(), kangarootwelve_xkcp::hash(b"foobarbaz"));

When we are testing the impl UpdateFinalize for Message, then are we asserting the equality of an updated message versus a text that was hashed with fn cshake() in the ops.rs?

Are we trying to make this line of code work?

let mut m = Message::new();
m.update("foo");
m.update("bar");
m.update("baz");
assert_eq!(m.finalize(), vec!["foobarbaz"].compute_hash_sha3(&SecParam::D256));

When a new Message is instantiated, most attributes are by default None (e.g. msg: some value, d = None, sym_nonce: None, asym_nonce: None, digest: Ok([]), op_result: Ok(()), sig: None). How can a newly created Message access the required attributes?

[Dustin-Ray]

How can a newly created Message access the required attributes?

Thats a great question. For this issue, when the message is first created, lets assume that the user will set the security parameter themselves beforehand. So add this extra line:

let mut m = Message::new();
m.d = &D256; // or similar
m.update("foo");
m.update("bar");
m.update("baz");
assert_eq!(m.finalize(), vec!["foobarbaz"].compute_hash_sha3(&SecParam::D256));

You can assume this for any other Message fields that might be needed

[Hyunggilwoo]

You can assume this for any other Message fields that might be needed

Thank you.

[Hyunggilwoo]

Hi there,

I added a fix feature for the above finalize paradigm under #fix/issue18.

[Dustin-Ray]

Hi there,

I added a fix feature for the above finalize paradigm under #fix/issue18.

Hey right on thank you! Can you please open a pull request for this? We have #55 open still, but it has been some time since we have worked on that, you might find it easier to start with a fresh PR. Let me know if you have any questions!