Herbarium Segmentation

This repository contains the code and links to the dataset and retrained models from the following publication: Evaluating Segmentation Approaches on Digitized Herbarium Specimens

Dataset

See: Dataset

Pretrained models

Binary Segmentation

Inference and Training Code: plant_binary_segmentation

Model	IoU	F1	Download
UNet++	0.951	0.975	UNet++
U-Net	0.950	0.974	U-Net
DeeplabV3+	0.915	0.954	DeeplabV3+

Instance Segmentation

YOLOv8 inference and training code: plant_instance_YOLOv8

Mask2Former inference code: mask2former_segmentation (training code TODO)

Left to right: Detectron2, Mask R-CNN, YOLOv8, and Mask2Former

Model	Box AP	Box AP50	Mask AP	Mask AP50	Plant AP	Object AP	Download
Detectron2	76.7	95.7	68.4	85.4	9.0	78.3	Detectron2
Mask R-CNN	78.2	94.6	76.7	92.7	31.9	84.1	Mask R-CNN
YOLOv8	87.0	98.5	78.5	96.1	48.1	83.5	YOLOv8l-seg
Mask2Former	80.7	93.2	78.9	91.0	77.0	79.2	Mask2Former

Panoptic Segmentation

Mask2Former inference code: mask2former_segmentation (training code TODO)

Model	Mask AP*	Mask AP50*	Plant IoU	Download
YOLOv8 + UNet++	83.7	98.3	0.951	UNet++ - YOLOv8l-seg (objects only)
Mask2Former	81.6	95.7	0.899	Mask2Former

*Mask APs for object classes only

Labeling new data

Semi-automatic labeling: generate_plant_masks

Interactive Manual validation with OpenCV via: label_plant_masks.py

Preparing custom dataset

1. Manually label objects with LabelMe or similar tool.
2. Convert to COCO format
3. Generate panoptic labels

Citation

If you use this dataset or code in your research, please use the following BibTeX entry:

@inproceedings{milleville2023evaluating,
  title={Evaluating Segmentation Approaches on Digitized Herbarium Specimens},
  author={Milleville, Kenzo and Chandrasekar, Krishna Kumar Thirukokaranam and Van de Weghe, Nico and Verstockt, Steven},
  booktitle="Advances in Visual Computing",
  year="2023",
  publisher="Springer Nature Switzerland",
  address="Cham",
  pages="65--78",
  isbn="978-3-031-47966-3"
}