Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add of offload threshold property in namespace configuration #109

Merged

Conversation

shodo
Copy link
Contributor

@shodo shodo commented Dec 29, 2023

Motivation

In the company I work with, we needed to enable "infinite" retention for certain topics. This paradigm requires offloading part of the topic to S3 when a certain size threshold is reached.
I know that Pulsar offers this configuration on a namespace level, but the Terraform plugin does not expose this configuration, so I'm creating this PR to add an offload_threshold_size_in_mb property to the namespace_config

Modifications

  • Add of offload_threshold_size_in_mb property in the namespace_config
  • Add a field in the related NamespaceConfig type
  • Added code to parse the property and call the related pulsar client function

Verifying this change

  • Make sure that the change passes the CI checks.

This change is already covered by existing tests, such as resource_pulsar_namespace_test.go, I've just added the new offload_threshold_size_in_mb in the testPulsarNamespace fixture.

Documentation

  • [ x ] doc

@shodo shodo requested a review from a team as a code owner December 29, 2023 09:31
Copy link

@shodo:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

@github-actions github-actions bot added doc-info-missing This pr needs to mark a document option in description and removed doc-info-missing This pr needs to mark a document option in description labels Dec 29, 2023
Copy link

@shodo:Thanks for providing doc info!

@github-actions github-actions bot added the doc This pr contains a document label Dec 29, 2023
@shodo shodo force-pushed the add-of-offload-threshold-in-namespace-cfg branch from 7e123b2 to 4b1db24 Compare January 31, 2024 15:10
Copy link
Member

@maxsxu maxsxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shodo Thanks for contribution!

Could you update the docs by run go generate, since this PR introduced new param?

@ca-reviewer
Copy link

@shodo Thanks for contribution!

Could you update the docs by run go generate, since this PR introduced new param?

done!

@giacomo-porro
Copy link

Hi guys! I see this PR have been opened 5 months ago and I need the same functionality as @shodo implemented, is there any blocker to include the change in the next release? How can I help?

@shodo shodo force-pushed the add-of-offload-threshold-in-namespace-cfg branch from bb6c183 to 4adc3f4 Compare April 29, 2024 10:12
@shodo
Copy link
Contributor Author

shodo commented Apr 29, 2024

I've rebased my forked repo on the last changes and re-generated the doc to be 100% sure everything is aligned. Let me know if there is something I can do to speed up the merge

@maxsxu maxsxu merged commit 9f30205 into streamnative:master May 1, 2024
2 checks passed
@maxsxu
Copy link
Member

maxsxu commented May 2, 2024

@shodo @giacomo-porro The terraform-provider-pulsar v0.3.1 has released and this PR included.

Thanks for your contributions again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc This pr contains a document
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants