Cut out objects with transparent background in the segmenter #47
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR modifies the output of the segmenter process by cropping objects with transparent background. The previous implementation cropped objects with an entire region of interest from the original image, which often included unnecessary background and parts of other objects. The file affected is
processing/segmenter/planktoscope/segmenter/__init__.py
to update and add methods for object detection, cropping and saving.Changes:
_save_object_image
to save the object cutout with RGB channels along with the alpha channel (RGBA) for transparency_open_and_apply_flat
_stream
to be compatible with RGBA mode and PNG file format of the object image_crop_and_apply_mask
that crops the raw image and mask to the bounding box of the detected object to ensure the cutout is as close as possible to the size of the object. This method adds an alpha channel to the cropped image, and sets the alpha channel to the segmentation mask. This will make areas outside the mask completely transparent and areas inside the mask opaque.__augment_slice
is no longer used within the method_slice_image
Testing:
These changes were tested in an external notebook environment with multiple images to ensure that each detected object is correctly isolated and saved in the expected format. The transparency was preserved in the output PNG files (see example).
Instructions for testing this PR on the PlanktoScope
To test new changes to this PR on a PlanktoScope with internet access:
feature/segmenter
branchsha-55e23d1
) corresponding to the latest commit (e.g. 55e23d1 ) which you had pushed to the branch. It should also have the labelpr-47
attached.forklift pallet switch github.com/PlanktoScope/pallet-standard@feature/segmentation-transparency
(note: the URL given here corresponds to Bump device-pkgs to make segmentation background transparent pallet-standard#15, which deploys the latestpr-47
version of the Docker container image for the segmenter).sudo systemctl start forklift-apply
. This command will disconnect Cockpit in your browser tab; if you wait ~10 seconds and refresh your browser tab, then you should be able to log in to Cockpit again.Afterwards, the updated segmenter should be running. You can check its live-updating logs with Dozzle (usually the logs will be at http://planktoscope.local/admin/dozzle/show?name=apps_ps_backend_proc-segmenter-server-1, or else you can access the live-updating logs from a link in the Node-RED dashboard's "Administration" page).