Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue #3685] save search get opportunity id #3704

Open
wants to merge 27 commits into
base: main
Choose a base branch
from

Conversation

babebe
Copy link
Collaborator

@babebe babebe commented Jan 30, 2025

Summary

Fixes #{3685}

Time to review: 10 mins

Changes proposed

Query the search index limiting the results to only opportunity ID
Store the list of opportunity IDs returned to the DB as a list/array (does not need to point to opportunity table as foreign keys)
Modify the users query to replace with static pagination

Additional information

Updated tests

Comment on lines 220 to 229
updated_search_query = {
**search_query,
"pagination": {
"order_by": "post_date",
"page_offset": 1,
"page_size": 1000,
"sort_direction": "descending",
},
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you move the definition of the static pagination values to a constant at the top of the file?

A very slightly briefer syntax you can do that avoids needing ** unpacking:

Suggested change
updated_search_query = {
**search_query,
"pagination": {
"order_by": "post_date",
"page_offset": 1,
"page_size": 1000,
"sort_direction": "descending",
},
}
updated_search_query = search_query | STATIC_PAGINATION

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 230 to 241
search_params = SearchOpportunityParams.model_validate(updated_search_query)

search_request = _get_search_request(search_params, False)

index_alias = get_search_config().opportunity_search_index_alias
logger.info(
"Querying search index alias %s", index_alias, extra={"search_index_alias": index_alias}
)

response = search_client.search(
index_alias, search_request, includes=["opportunity_id"], excludes=["attachments"]
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire chunk is duplicated across both this new implementation and the existing one (with just the aggregation flag different), could you move that into some shared bit of code they both use? That way if we need to make changes, it can affect both without missing it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GP. Created a helper function, done.

Comment on lines 263 to 267
# Retrieve opportunity IDs
opportunity_ids = search_opportunities_id(search_client, json_data["search_query"])

with db_session.begin():
saved_search = create_saved_search(db_session, user_id, json_data)
saved_search = create_saved_search(db_session, user_id, json_data, opportunity_ids)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, we'd have the following in this file:

with db_session.begin():
     saved_search = some_function_call(...)

I want to aim for as little logic as possible outside the service files - that makes the code more reusable (like how the search logic can be reused).

I'd just make create_saved_search have the first thing it does is fetch the opportunity IDs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 9 to 12
from tests.src.api.opportunities_v1.test_opportunity_route_search import (
NASA_INNOVATIONS,
NASA_SUPERSONIC,
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't import across unit tests like this - if the setup data has any shared use, move it to the conftest files that are already shared between the unit tests.

Effectively when we're talking test setup, it should only go in one of three places:

  • Top of the same test file - for when test setup only pertains to whatever we're testing in that file
  • conftest.py in the same directory - when multiple test files in the same directory want to reuse it
  • conftest.py in the root tests folder - a lot of tests across the codebase want to use it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not think creating common test data in the conftest was appropriate. I created reusable opportunities for the test module.

Comment on lines 104 to 108
# Swap the search index alias
alias = f"test-user_save_search-index-alias-{uuid.uuid4().int}"
monkeypatch.setenv("OPPORTUNITY_SEARCH_INDEX_ALIAS", alias)

search_client.swap_alias_index(opportunity_index, alias)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend putting anything like this into a utility/fixture - this is something we likely want in several unit tests, we shouldn't need to copy-paste chunks of code for every unit test.

We do already have a fixture for setting up the search index alias, but it's at the session level (so across all tests), might need to adjust that to also support other levels.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not want to change the existing session scoped fixture, as it is used in other session scoped fixtures. Creating another function scoped fixture looked like an overkill that's why i did this. You think we should create function scope fixture for the opportunity alias ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, a function scoped session would be fine - a module/class level one might also be worth setting up as we usually are fine with test setup for a whole file/class being reused across everything in that file/class (since that's easier to follow / faster running tests).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@chouinar
Copy link
Collaborator

@babebe - could you make sure your PR title follows our usual format? Should be: [Issue #XXXX] Some sort of description - these are used in the deploy metadata

@babebe babebe changed the title 3685/save search get opp [Issue 3685] save search get opp Jan 31, 2025
@babebe babebe changed the title [Issue 3685] save search get opp [Issue 3685] save search get opportunity id Jan 31, 2025
@babebe babebe changed the title [Issue 3685] save search get opportunity id [Issue #3685] save search get opportunity id Jan 31, 2025
@babebe babebe requested a review from chouinar January 31, 2025 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Modify the endpoint where we save user searches to also query the search index and get opportunity IDs
2 participants