Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Client/Broker cargo features #71

Merged
merged 1 commit into from
Sep 3, 2024

Conversation

rukai
Copy link
Collaborator

@rukai rukai commented Aug 30, 2024

warm --release build times of kafka-protocol crate on my machine:

  • main branch
    • --no-default-features ~1.32s
    • --no-default-features --features messages_enum ~2.00s
  • this branch
    • --no-default-features 0.83s (effectively useless though)
    • --no-default-features --features broker ~1.07s
    • --no-default-features --features client ~1.07s
    • --no-default-features --features broker,client ~1.32s
    • --no-default-features --features broker,client,messages_enum ~2.00s
    • --no-default-features --features client,messages_enum ~1.40s

The measurements show:

  • a clear improvement to compile time for the use case of developing either just a client or just a broker.
  • no regressions for the use case of writing a proxy where both client and broker features are enabled.

There is still a small cost of extra complexity in maintenance and usage of kafka-protocol.
But I think its still definitely worth it.

I have enabled broker and client features by default since kafka-protocol is useless without at least one of these enabled. This PR is still a breaking change though, since a user could be depending on kafka-protocol with default-features = false

I also introduced cargo-hack to CI to ensure that all feature combinations continue to build and pass clippy lints.

@rukai rukai force-pushed the client_broker_features branch from 1430ba3 to dd8a0a8 Compare August 30, 2024 04:19
@rukai rukai marked this pull request as ready for review August 30, 2024 04:36
Copy link
Owner

@tychedelia tychedelia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! I agree with the approach of enabling both by default as this is something that the user can easily back out of later. One thing we might also consider in the future too is adding a kind of "blacklist" which would specifically filter out the broker-to-broker messages used in the Raft implementation. The chances someone ever wants to use these seems extraordinarily low although maybe still useful for the proxy use case?

@tychedelia tychedelia merged commit 6309144 into tychedelia:main Sep 3, 2024
3 checks passed
@rukai
Copy link
Collaborator Author

rukai commented Sep 3, 2024

Its not clear to me which methods are solely for broker-to-broker raft, having them behind a feature flag would be valuable to me though, since our proxy is only between the client and broker, not between brokers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants