-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Message To Drop Semantics #47
Conversation
Signed-off-by: shubham <[email protected]>
Signed-off-by: shubham <[email protected]>
Signed-off-by: shubham <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you update the examples to use the new semantics?
What about adding a func |
src/map.rs
Outdated
pub fn with_keys(mut self,keys:Vec<String>)-> Self{ | ||
self.keys=keys; | ||
self | ||
} | ||
|
||
pub fn with_tags(mut self,tags:Vec<String>)-> Self{ | ||
self.tags=tags; | ||
self | ||
} | ||
|
||
pub fn keys(mut self) ->Vec<String>{ | ||
self.keys | ||
} | ||
pub fn value(mut self) ->Vec<u8>{ | ||
self.value | ||
} | ||
|
||
pub fn tags(mut self) ->Vec<String>{ | ||
self.tags | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was wondering, since these are public fields, do we really need getters and setters? I like the idea of the builder pattern because these fields are optional. Should we make the fields private instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@BulkBeing WDYT on builder pattern? It is getting popular in Rust.
src/map.rs
Outdated
pub struct Messages{ | ||
messages:Vec<Message> | ||
} | ||
|
||
impl Messages{ | ||
fn message_builder() -> Self { | ||
Messages { | ||
messages: Vec::new(), | ||
} | ||
} | ||
fn append(mut self, msg: Message) -> Self { | ||
self.messages.push(msg); | ||
self | ||
} | ||
|
||
fn items( &self) ->&Vec<Message>{ | ||
&self.messages | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can avoid this abstraction. Since users will be more comfortable using vectors directly.
you mean creating just a public method without having impl part for Message struct ? |
@shubhamdixit863 let's use builder pattern as recommended by @yhl25 |
Signed-off-by: shubham <[email protected]>
Signed-off-by: shubham <[email protected]>
Signed-off-by: shubham <[email protected]>
Please run rustfmt on the changes |
Signed-off-by: shubham <[email protected]>
@yhl25 @BulkBeing WDYT? |
i am not able to build an opinion on this, will let @yhl25 to weight in.
https://github.com/fede1024/rust-rdkafka/blob/master/src/producer/future_producer.rs#L42 @shubhamdixit863 Can we do something similar to this? |
Signed-off-by: shubham <[email protected]>
Signed-off-by: shubham <[email protected]>
src/map.rs
Outdated
pub fn new(value:Vec<u8>) -> Self { | ||
Self{ | ||
value, | ||
keys:None, | ||
tags:None | ||
} | ||
} | ||
pub fn message_to_drop(mut self) -> Self { | ||
if self.tags.is_none(){ | ||
self.tags=Some(Vec::new()); | ||
} | ||
self.tags.as_mut().unwrap().push(DROP.parse().unwrap()); | ||
self | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please add doc
Signed-off-by: shubham <[email protected]>
Signed-off-by: shubham <[email protected]>
src/map.rs
Outdated
/// value: input.value, | ||
/// tags: vec![], | ||
/// }] | ||
/// use numaflow::map::Message; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/// use numaflow::map::Message; | |
/// use numaflow::map::Message; |
src/map.rs
Outdated
pub fn message_to_drop(mut self) -> Self { | ||
if self.tags.is_none(){ | ||
self.tags=Some(Vec::new()); | ||
} | ||
self.tags.as_mut().unwrap().push(DROP.parse().unwrap()); | ||
self | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yhl25, @shubhamdixit863; isn't it better to have a standalone message_to_drop
? what is the point of new().message_to_drop()
, esp. since new
takes a mandatory arg?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
message_to_drop()
must be standalone!
Signed-off-by: shubham <[email protected]>
Signed-off-by: Yashash H L <[email protected]>
chore: minor changes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
This PR fixes the following issue: #12.
Map, reduce, and source transform have been modified to support drop message semantics.
How is it tested?
The scenario has been tested in a local Kubernetes cluster.