Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Implement multi tenancy in Flow Framework (#980) #1024

Merged
merged 1 commit into from
Jan 25, 2025

Conversation

dbwiddis
Copy link
Member

Backports d8a5a4b from #980

* Import SdkClient and inject it

Signed-off-by: Daniel Widdis <[email protected]>

* Pass sdkClient to IndicesHandler and EncryptorUtils classes

Signed-off-by: Daniel Widdis <[email protected]>

* Extract tenant id from REST header into RestAction

Signed-off-by: Daniel Widdis <[email protected]>

* Pass tenant id to transport actions in template

Signed-off-by: Daniel Widdis <[email protected]>

* Validate tenant id existence in workflow transport actions

Signed-off-by: Daniel Widdis <[email protected]>

* Pass SdkClient and tenant id to util used for access control checks

Signed-off-by: Daniel Widdis <[email protected]>

* Perform tenant id validation checks for workflow APIs

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate Update workflow get action to SdkCleint

Signed-off-by: Daniel Widdis <[email protected]>

* Pass tenantId to IndicesHandler and use in EncryptorUtils

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate EncryptorUtils getting master key from index

Signed-off-by: Daniel Widdis <[email protected]>

* Refactor fetching master key to permit reuse

Signed-off-by: Daniel Widdis <[email protected]>

* Refactor initializeMasterKey to use common code

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate indexing new key to config

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate template indexing to sdkClient

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate template deletion to sdkClient

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate get template to sdkClient

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate provision template to sdkClient

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate max workflow search to sdkClient

Signed-off-by: Daniel Widdis <[email protected]>

* Add tenantId to GetWorkflowStateRequest

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate GetWorkflowStateRequest to multitenant client

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate getProvisioningProgress to avoid repetition

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate canDeleteWorkflowStateDoc to avoid repetition

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate initial state document creation to metadata client

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate state document deletion to metadata client

Signed-off-by: Daniel Widdis <[email protected]>

* Add Tenant aware Rest Tests for Workflows

Signed-off-by: Daniel Widdis <[email protected]>

* Fix javadocs

Signed-off-by: Daniel Widdis <[email protected]>

* Add publishToMavenLocal for more CI

Signed-off-by: Daniel Widdis <[email protected]>

* Fix some CI

Signed-off-by: Daniel Widdis <[email protected]>

* Enable tenant aware search

Signed-off-by: Daniel Widdis <[email protected]>

* Refactor state index update method using multitenant client

Signed-off-by: Daniel Widdis <[email protected]>

* Get metadata client artifacts from Maven Snapshot

Signed-off-by: Daniel Widdis <[email protected]>

* Update tests for new update async code

Signed-off-by: Daniel Widdis <[email protected]>

* Switch SdkClient to use default generic thread executor

Signed-off-by: Daniel Widdis <[email protected]>

* Migrate last updates to sdkClient

Signed-off-by: Daniel Widdis <[email protected]>

* Revert (most) changes to unit tests based on async client changes

Signed-off-by: Daniel Widdis <[email protected]>

* Pass tenant id when updating state during provisioning

Signed-off-by: Daniel Widdis <[email protected]>

* Integrate tenantId with synchronous provisioning

Signed-off-by: Daniel Widdis <[email protected]>

* Fix failing integ tests after rebase, code review updates

Signed-off-by: Daniel Widdis <[email protected]>

* Replace fakeTenantId placeholders with actual tenant id

Signed-off-by: Daniel Widdis <[email protected]>

* Use version catalog for commons-lang3 and httpcore dependencies

Signed-off-by: Daniel Widdis <[email protected]>

* Exclude transitive httpclient dependency from metadata and rest client

Signed-off-by: Daniel Widdis <[email protected]>

* Fix more test errors and tweak dependencies

Signed-off-by: Daniel Widdis <[email protected]>

* More code review comments and refactoring

Signed-off-by: Daniel Widdis <[email protected]>

---------

Signed-off-by: Daniel Widdis <[email protected]>
Copy link

codecov bot commented Jan 25, 2025

Codecov Report

Attention: Patch coverage is 62.74256% with 288 lines in your changes missing coverage. Please review.

Project coverage is 75.52%. Comparing base (c9442f4) to head (ff500da).
Report is 1 commits behind head on 2.x.

Files with missing lines Patch % Lines
...framework/indices/FlowFrameworkIndicesHandler.java 65.07% 82 Missing and 6 partials ⚠️
...ework/transport/CreateWorkflowTransportAction.java 66.05% 28 Missing and 9 partials ⚠️
.../opensearch/flowframework/util/EncryptorUtils.java 63.63% 26 Missing and 10 partials ⚠️
.../transport/ReprovisionWorkflowTransportAction.java 20.00% 14 Missing and 2 partials ⚠️
...ework/transport/DeleteWorkflowTransportAction.java 55.17% 10 Missing and 3 partials ⚠️
...flowframework/transport/handler/SearchHandler.java 45.83% 13 Missing ⚠️
.../org/opensearch/flowframework/util/ParseUtils.java 43.47% 9 Missing and 4 partials ⚠️
...h/flowframework/rest/RestCreateWorkflowAction.java 60.71% 6 Missing and 5 partials ⚠️
...rk/transport/ProvisionWorkflowTransportAction.java 47.05% 7 Missing and 2 partials ⚠️
.../opensearch/flowframework/FlowFrameworkPlugin.java 50.00% 7 Missing and 1 partial ⚠️
... and 13 more
Additional details and impacted files
@@             Coverage Diff              @@
##                2.x    #1024      +/-   ##
============================================
- Coverage     76.41%   75.52%   -0.89%     
- Complexity     1010     1050      +40     
============================================
  Files           100      101       +1     
  Lines          4884     5214     +330     
  Branches        455      498      +43     
============================================
+ Hits           3732     3938     +206     
- Misses          955     1038      +83     
- Partials        197      238      +41     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dbwiddis dbwiddis merged commit 82bfbc3 into opensearch-project:2.x Jan 25, 2025
33 of 34 checks passed
@dbwiddis dbwiddis deleted the backport-980-to-2.x branch January 25, 2025 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants