Support 3D object detection annotation on pcd files with CVAT #5458
Conversation
WalkthroughThis update enhances the CVAT integration by introducing support for annotating 3D data. The documentation now includes a section that explains the process of annotating 3D point cloud data, detailing the required file format, preprocessing steps, and an example procedure. Furthermore, the CVAT backend has been updated to recognize a new media type (3D) by modifying several method signatures and incorporating checks for the necessary parameters when processing 3D datasets. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CVATBackend
User->>CVATBackend: Calls upload_samples(samples, anno_key, backend, media_field)
alt 3D Media Provided
CVATBackend->>CVATBackend: Validate media_field is present
CVATBackend->>CVATBackend: Process samples with is_3d flag set
CVATBackend->>User: Successfully uploads 3D data
else Missing media_field
CVATBackend-->>User: Raise ValueError (prompting correct usage)
end
Suggested labels
Suggested reviewers
Poem
✨ Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
fiftyone/utils/cvat.py (4)
5428-5428: Redundant repetition ofis_3dcheckYou are defining
is_3dagain for the same purpose (line 4409). Consider refactoring to avoid duplication, for example by using a helper function or consolidating logic once at a higher level.
5957-5959: Consider merging conditional branchesRuff suggests merging the
if det.bounding_box:andelse:branches using a logical operator. The current structure is readable, but merging could slightly streamline the code:🧰 Tools
🪛 Ruff (0.8.2)
5954-5959: Combine
ifbranches using logicaloroperatorCombine
ifbranches(SIM114)
6199-6199:is_3ddeclared againYou redefine
is_3dhere for a second time. If this is intended, it’s fine; otherwise, you might unify these declarations to avoid confusion and duplication.
7178-7180: Remove unused local variablesThe variables
location,dimensions, androtationare assigned but never utilized. Consider removing them to declutter the code:- location = None - dimensions = None - rotation = None🧰 Tools
🪛 Ruff (0.8.2)
7178-7178: Local variable
locationis assigned to but never usedRemove assignment to unused variable
location(F841)
7179-7179: Local variable
dimensionsis assigned to but never usedRemove assignment to unused variable
dimensions(F841)
7180-7180: Local variable
rotationis assigned to but never usedRemove assignment to unused variable
rotation(F841)
docs/source/integrations/cvat.rst (1)
2239-2288: New Section for 3D Annotations: Clear and InformativeThe new "Annotating 3D data" section is well-written and clearly explains how to work with 3D point cloud data using CVAT. It details the key differences between 2D and 3D workflows, highlights the need for a manual preprocessing step, and instructs users to set the
media_fieldwhen annotating 3D datasets. The code example effectively demonstrates converting file paths (from.fo3dto.pcd), selecting a group slice, and performing annotation steps, which aligns well with the PR objectives.Minor suggestion: Consider adding a brief note for users on handling alternative file naming conventions or additional preprocessing scenarios, in case the simple replacement (
f.replace(".fo3d", ".pcd")) does not directly apply.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
docs/source/integrations/cvat.rst(2 hunks)fiftyone/utils/cvat.py(10 hunks)
🧰 Additional context used
🪛 Ruff (0.8.2)
fiftyone/utils/cvat.py
5954-5959: Combine if branches using logical or operator
Combine if branches
(SIM114)
7178-7178: Local variable location is assigned to but never used
Remove assignment to unused variable location
(F841)
7179-7179: Local variable dimensions is assigned to but never used
Remove assignment to unused variable dimensions
(F841)
7180-7180: Local variable rotation is assigned to but never used
Remove assignment to unused variable rotation
(F841)
⏰ Context from checks skipped due to timeout of 90000ms (6)
- GitHub Check: test / test-python (ubuntu-latest-m, 3.10)
- GitHub Check: test / test-app
- GitHub Check: lint / eslint
- GitHub Check: e2e / test-e2e
- GitHub Check: build / build
- GitHub Check: build
🔇 Additional comments (5)
fiftyone/utils/cvat.py (5)
3255-3255: Extend support to 3D media typesThe addition of
fom.THREE_Din the return list forsupported_media_typesis consistent with the broader 3D annotation support introduced in this file.
4409-4409: Detect 3D media at sample levelAssigning
is_3d = samples.media_type == fom.THREE_Dprovides a straightforward check for 3D data. Implementation looks correct.
4410-4417: Requiremedia_fieldfor 3DRaising an error for 3D samples when the
media_fieldis not set (i.e., still "filepath") clarifies usage and prevents accidental misconfiguration.
5444-5444: Enable single-job tasks for 3DUsing
if is_video or is_3d:ensures that 3D data is treated analogously to video in terms of task segmentation. This is coherent, and the change aligns with the new 3D annotation flow.
6215-6217: Setup for single-frame 3D annotationLeveraging
[sample]and an emptyframe_sizetuple here is a valid approach to process 3D data. Ensure that elsewhere in the pipeline, the absence of traditional width/height attributes is correctly handled.
What changes are proposed in this pull request?
This PR adds support for 3d pointcloud annotation, specifically of 3D detections/cuboids, through the integration with CVAT.
3D annotation in CVAT is limited to 3d detections on a single .pcd file (with the option of adding up to 12 reference images). There are multiple directory structures that it supports for this: https://docs.cvat.ai/docs/manual/basics/create_an_annotation_task/#data-formats-for-a-3d-task
Given that 3d datasets in FiftyOne require media to be stored in .fo3d files, annotating them requires an additional step which is on the user to structure the data as needed by cvat, and store the references to that data in an alternate media field on the dataset.
Once that setup is complete, you can send 3d detections with pointcloud data and load them just as you would for 2d labels. See the test for an example.
An example of this has been added to the CVAT integration docs as well.
How is this patch tested? If it is not, please explain why.
Release Notes
Is this a user-facing change that should be mentioned in the release notes?
notes for FiftyOne users.
Adds support for annotation of 3D cuboids in pointcloud data through the CVAT integration.
What areas of FiftyOne does this PR affect?
fiftyonePython library changesSummary by CodeRabbit
New Features
Documentation