Fix kafka buffer metrics #3805
Fix kafka buffer metrics #3805
Conversation
Signed-off-by: Krishna Kondaka <[email protected]>
kkondaka
requested review from
chenqi0805,
engechas,
graytaylor0,
dinujoh,
asifsmohammed,
dlvenable and
oeyh
as code owners
| recordsWrittenCounter.increment(); | ||
| recordsInBuffer.incrementAndGet(); | ||
| if (!isByteBuffer()) { | ||
| recordsWrittenCounter.increment(); |
There was a problem hiding this comment.
With protobuf wrapper over byteBuffer can we store numRecords along with byteBuffer? and then increment/decrement these metrics? these are good metrics for customers IMO.
There was a problem hiding this comment.
Maybe. I am not sure if it's always possible because it means we have to parse the input, which we do not plan to do afaik.
There was a problem hiding this comment.
We should add some metrics for the byte counts.
Also, I think we should move away from this AbstractBuffer. We make our second buffer and it is already producing erroneous results. :)
But, that can come later.
| recordsWrittenCounter.increment(); | ||
| recordsInBuffer.incrementAndGet(); | ||
| if (!isByteBuffer()) { | ||
| recordsWrittenCounter.increment(); |
There was a problem hiding this comment.
We should add some metrics for the byte counts.
Also, I think we should move away from this AbstractBuffer. We make our second buffer and it is already producing erroneous results. :)
But, that can come later.
Description
Fix Kafka Buffer metrics
While reading from Kafka inner buffer
doReadAPI was used. This skips updating the metrics. modified to usereadAPI which callsdoReadand also updates metrics.Issues Resolved
Resolves #[Issue number to be closed when this PR is merged]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.