Add set_log_upload_error for DOs #5448
Conversation
WalkthroughThe pull request introduces a new method Changes
Possibly related PRs
Suggested reviewers
Poem
✨ Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
fiftyone/operators/delegated.py (1)
255-265: Fix docstring formatting.There's a typo in the Args section of the docstring.
Apply this diff to fix the formatting:
Args: doc_id: the ID of the delegated operation - log status: the status of the logs to set + log_status: the status of the logs to set
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
fiftyone/factory/repos/delegated_operation.py(3 hunks)fiftyone/factory/repos/delegated_operation_doc.py(2 hunks)fiftyone/operators/delegated.py(1 hunks)
🧰 Additional context used
🪛 Ruff (0.8.2)
fiftyone/factory/repos/delegated_operation_doc.py
76-76: Use doc.get("log_status") instead of doc.get("log_status", None)
Replace doc.get("log_status", None) with doc.get("log_status")
(SIM910)
⏰ Context from checks skipped due to timeout of 90000ms (9)
- GitHub Check: lint / eslint
- GitHub Check: e2e / test-e2e
- GitHub Check: build / build
- GitHub Check: build / changes
- GitHub Check: test / test-python (ubuntu-latest-m, 3.11)
- GitHub Check: test / test-python (ubuntu-latest-m, 3.10)
- GitHub Check: test / test-python (ubuntu-latest-m, 3.9)
- GitHub Check: test / test-app
- GitHub Check: build
🔇 Additional comments (4)
fiftyone/factory/repos/delegated_operation_doc.py (1)
59-59: LGTM: Added log_status attribute.The new attribute is properly initialized as None.
fiftyone/factory/repos/delegated_operation.py (3)
120-124: LGTM: Well-defined abstract method.The abstract method follows the established pattern with proper type hints and documentation.
176-181: LGTM: Added index for dataset_id field.The index creation follows the established pattern and is properly guarded.
255-263: LGTM: Proper implementation of set_log_status.The implementation follows the established pattern of other similar methods and uses atomic operations.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
fiftyone/factory/repos/delegated_operation_doc.py (1)
76-76: Simplify the get() call.The explicit None default is redundant as
dict.get()returns None by default when the key is not found.- self.log_upload_error = doc.get("log_upload_error", None) + self.log_upload_error = doc.get("log_upload_error")🧰 Tools
🪛 Ruff (0.8.2)
76-76: Use
doc.get("log_upload_error")instead ofdoc.get("log_upload_error", None)Replace
doc.get("log_upload_error", None)withdoc.get("log_upload_error")(SIM910)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
fiftyone/factory/repos/delegated_operation.py(3 hunks)fiftyone/factory/repos/delegated_operation_doc.py(2 hunks)fiftyone/operators/delegated.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- fiftyone/operators/delegated.py
🧰 Additional context used
🪛 Ruff (0.8.2)
fiftyone/factory/repos/delegated_operation_doc.py
76-76: Use doc.get("log_upload_error") instead of doc.get("log_upload_error", None)
Replace doc.get("log_upload_error", None) with doc.get("log_upload_error")
(SIM910)
⏰ Context from checks skipped due to timeout of 90000ms (8)
- GitHub Check: test / test-python (ubuntu-latest-m, 3.11)
- GitHub Check: test / test-python (ubuntu-latest-m, 3.10)
- GitHub Check: test / test-python (ubuntu-latest-m, 3.9)
- GitHub Check: test / test-app
- GitHub Check: e2e / test-e2e
- GitHub Check: lint / eslint
- GitHub Check: build / build
- GitHub Check: build
🔇 Additional comments (5)
fiftyone/factory/repos/delegated_operation_doc.py (2)
59-59: LGTM! Clear and consistent attribute initialization.The
log_upload_errorattribute is properly initialized as None, following the established pattern in the class.
Line range hint
119-125: LGTM! Proper serialization of the new field.The
log_upload_errorfield is correctly handled by the existing__dict__serialization into_pymongo().fiftyone/factory/repos/delegated_operation.py (3)
120-124: LGTM! Well-defined abstract method.The
set_log_upload_errormethod is properly defined with clear type hints and follows the abstract method pattern.
176-181: LGTM! Proper index creation for dataset_id field.The index creation follows the established pattern and will improve query performance when filtering by dataset_id.
255-263: LGTM! Clean and consistent implementation.The
set_log_upload_errorimplementation follows the established pattern using atomic operations and proper return values.
| if "dataset_id_1" not in index_names: | ||
| indices_to_create.append( | ||
| IndexModel( | ||
| [("dataset_id", pymongo.ASCENDING)], name="dataset_id_1" | ||
| ) | ||
| ) | ||
|
|
There was a problem hiding this comment.
might be good to have a compound index on dataset id then other attributes we want to filter/sort by. but this (dataset_id index) is required at minimum.
What changes are proposed in this pull request?
Adding set_log_upload_error for DOs so that we can update that field when needed for use in delegated operations. This field will be populated if we fail to flush logs to the user provided log location after a DO finishes execution.
Also, adding an index on dataset_id so we can filter without needing to do a scan making this collection scale better as DOs get used more overtime.
How is this patch tested? If it is not, please explain why.
script:
Result:
Index created:
Release Notes
Is this a user-facing change that should be mentioned in the release notes?
notes for FiftyOne users.
(Details in 1-2 sentences. You can just refer to another PR with a description
if this PR is part of a larger change.)
What areas of FiftyOne does this PR affect?
fiftyonePython library changesSummary by CodeRabbit
New Features
log_upload_errorattribute to track log upload issues.Performance Improvements
Technical Enhancements