Add object detection task #156

biancazadrozny · 2024-09-12T14:30:42Z

No description provided.

paolofraccaro · 2024-11-06T11:59:28Z

@daiki-kimura and @takaomoriyama will work on it. We will take inspiration from torchgeo OD task and try to have a factory that allows to use any backbone from our registries in one of the OD frameworks/architectures from torchvision.

daiki-kimura · 2024-11-21T06:56:02Z

Make a table for 91 datasets from torchgeo and EarthNet
Manually confirmed candidates (#sample>10k and the dataset is allowed to use as commercial-use)
Picked initial five candidates to have discussion

Table (IBMer-only): https://ibm-my.sharepoint.com/:x:/p/daiki_jp/EbdiOD6qleFLpQdXx4DpasUBMWDvUBHhrUT_jtGxhjgkZA?e=1htrxZ

takaomoriyama · 2024-12-02T06:35:53Z

The first prototype of ObjectDetectionTask and ObjectDetectionModelFactory is committed to obj_det_geobench though not completed.
How to run:

$ git clone -b obj_det_geobench [email protected]:IBM/terratorch.git
$ cd terratorch
$ pip install -r requirements/required.txt -r requirements/dev.txt
$ pip install -e .
$ terratorch fit --config examples/confs/object_detection_vhr10.yaml

Trying to reproduce example case using VHR10 data as in https://medium.com/@byeonghyeokyu/torchgeo-an-introduction-to-object-detection-example-b0fd43e89649

takaomoriyama · 2024-12-02T06:39:02Z

It may stop with following error.

Traceback (most recent call last):
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/bin/terratorch", line 8, in <module>                                                   
    sys.exit(main())
             ^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/__main__.py", line 9, in main                                     
    _ = build_lightning_cli()
        ^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/cli_tools.py", line 424, in build_lightning_cli                   
    return MyLightningCLI(
           ^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 391, in __init__          
    self.instantiate_classes()
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/cli_tools.py", line 372, in instantiate_classes                   
    super().instantiate_classes()
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 561, in instantiate_classes
    self._add_configure_optimizers_method_to_model(self.subcommand)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 689, in _add_configure_optimizers_method_to_model
    optimizer = instantiate_class(self.model.parameters(), optimizer_init)                                                                          
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                          
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 786, in instantiate_class 
    return args_class(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/optim/adamw.py", line 79, in __init__               
    super().__init__(params, defaults)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/optim/optimizer.py", line 368, in __init__          
    raise ValueError("optimizer got an empty parameter list")
ValueError: optimizer got an empty parameter list

This means that self.model.parameters() is returning empty. This case, self.model is an instance of ObjectDetectionModelWrapper.

takaomoriyama · 2024-12-02T08:19:09Z

The problem "ValueError: optimizer got an empty parameter list" was solved in commit 1e73181
Still encountering data fetch error.

Traceback (most recent call last):
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/bin/terratorch", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/__main__.py", line 9, in main
    _ = build_lightning_cli()
        ^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/cli_tools.py", line 424, in build_lightning_cli
    return MyLightningCLI(
           ^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 394, in __init__
    self._run_subcommand(self.subcommand)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 701, in _run_subcommand
    fn(**fn_kwargs)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 538, in fit
    call._call_and_handle_interrupt(
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/call.py", line 47, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 574, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 981, in _run
    results = self._run_stage()
              ^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 1023, in _run_stage
    self._run_sanity_check()
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 1052, in _run_sanity_check
    val_loop.run()
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/utilities.py", line 178, in _decorator
    return loop_run(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 128, in run
    batch, batch_idx, dataloader_idx = next(data_fetcher)
                                       ^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/fetchers.py", line 133, in __next__
    batch = super().__next__()
            ^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/fetchers.py", line 60, in __next__
    batch = next(self.iterator)
            ^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/utilities/combined_loader.py", line 341, in __next__
    out = next(self._iterator)
          ^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/utilities/combined_loader.py", line 142, in __next__
    out = next(self.iterators[0])
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 701, in __next__
    data = self._next_data()
           ^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1465, in _next_data
    return self._process_data(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 1491, in _process_data
    data.reraise()
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/_utils.py", line 715, in reraise
    raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/utils/data/_utils/worker.py", line 351, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
           ^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 50, in fetch
    data = self.dataset.__getitems__(possibly_batched_index)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/utils/data/dataset.py", line 420, in __getitems__
    return [self.dataset[self.indices[idx]] for idx in indices]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/utils/data/dataset.py", line 420, in <listcomp>
    return [self.dataset[self.indices[idx]] for idx in indices]
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torchgeo/datasets/vhr10.py", line 257, in __getitem__
    sample = self.transforms(sample)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/datamodules/torchgeo_data_module.py", line 42, in transforms_from_torch_tensor
    return callable_transform(numpy_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/datasets/transforms.py", line 17, in fn
    return albumentation(**data)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/albumentations/core/composition.py", line 195, in __call__
    self._check_args(**data)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/albumentations/core/composition.py", line 280, in _check_args
    raise TypeError("{} must be list of numpy arrays".format(data_name))
TypeError: masks must be list of numpy arrays

takaomoriyama · 2024-12-17T10:07:36Z

Updated training_step() of ObjectDetectionTask, and some clean up (https://github.com/IBM/terratorch/tree/obj_det_geobench)

Currently encountering following error after 1 epoch is executed. It looks that list is being passed to _multiclass_stat_scores_tensor_validation() instead of tensor.

XXX target[0]={'boxes': tensor([[ 28.,  53., 143., 164.],
        [ 89.,  79., 215., 200.],
        [158., 121., 289., 245.],
        [254., 158., 364., 290.],
        [307., 204., 413., 350.]], device='cuda:0'), 'labels': tensor([8, 8, 8,
8, 8], device='cuda:0')}
Epoch 0/1  ━━━━━━━━━━━━━━━━━━━━━ 25/25 0:00:22 • 0:00:00 1.26it/s v_num: 154.000

XXX torchgeo collate_fn_detection(): called
XXX Input data type
XXX   batch[0]['image']:  torch.Size([3, 564, 814]) of torch.float32
XXX   batch[0]['boxes']:  torch.Size([3, 4]) of torch.float32
XXX   batch[0]['labels']: torch.Size([3]) of torch.int64
XXX   batch[0]['masks']:  torch.Size([3, 564, 814]) of torch.uint8
XXX Output data type: All are list of T
XXX   batch['image']:  list of torch.Size([3, 564, 814]) of torch.float32
XXX   batch['boxes']:  list of torch.Size([3, 4]) of torch.float32
XXX   batch['labels']: list of torch.Size([3]) of torch.int64
XXX   batch['masks']:  list of torch.Size([3, 564, 814]) of torch.uint8
Traceback (most recent call last):
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/bin/terratorch", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/__main__.py", line 9, in main
    _ = build_lightning_cli()
        ^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/cli_tools.py", line 424, in build_lightning_cli
    return MyLightningCLI(
           ^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 394, in __init__
    self._run_subcommand(self.subcommand)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/cli.py", line 701, in _run_subcommand
    fn(**fn_kwargs)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 538, in fit
    call._call_and_handle_interrupt(
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/call.py", line 47, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 574, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 981, in _run
    results = self._run_stage()
              ^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/trainer.py", line 1031, in _run_stage
    self.fit_loop.run()
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/fit_loop.py", line 205, in run
    self.advance()
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/fit_loop.py", line 363, in advance
    self.epoch_loop.run(self._data_fetcher)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/training_epoch_loop.py", line 141, in run
    self.on_advance_end(data_fetcher)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/training_epoch_loop.py", line 295, in on_advance_end
    self.val_loop.run()
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/utilities.py", line 178, in _decorator
    return loop_run(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 135, in run
    self._evaluation_step(batch, batch_idx, dataloader_idx, dataloader_iter)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/loops/evaluation_loop.py", line 396, in _evaluation_step
    output = call._call_strategy_hook(trainer, hook_name, *step_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/trainer/call.py", line 319, in _call_strategy_hook
    output = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/lightning/pytorch/strategies/strategy.py", line 411, in validation_step
    return self.lightning_module.validation_step(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/dev/IBM/terratorch-obj_det_geobench/terratorch/tasks/object_detection_tasks.py", line 261, in validation_step
    metrics = self.val_metrics(y_hat, y)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torchmetrics/collections.py", line 198, in forward
    return self._compute_and_reduce("forward", *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torchmetrics/collections.py", line 335, in _compute_and_reduce
   res = m(*args, **m._filter_kwargs(**kwargs))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torchmetrics/metric.py", line 303, in forward
    self._forward_cache = self._forward_reduce_state_update(*args, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torchmetrics/metric.py", line 372, in _forward_reduce_state_update
    self.update(*args, **kwargs)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torchmetrics/metric.py", line 465, in wrapped_func
    update(*args, **kwargs)
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torchmetrics/classification/stat_scores.py", line 331, in update
    _multiclass_stat_scores_tensor_validation(
  File "/dccstor/usgs_dem/moriyama/.conda/envs/terratorch-od/lib/python3.11/site-packages/torchmetrics/functional/classification/stat_scores.py", line 283, in _multiclass_stat_scores_tensor_validation
    if preds.ndim == target.ndim + 1:
       ^^^^^^^^^^
AttributeError: 'list' object has no attribute 'ndim'

romeokienzler · 2025-01-07T15:44:22Z

@paolofraccaro currently working on it

biancazadrozny changed the title ~~Add detection task~~ Add object detection task Sep 12, 2024

biancazadrozny added the MoSCoW-should label Sep 12, 2024

biancazadrozny assigned jaionet Sep 12, 2024

$@paolofraccaro$ paolofraccaro assigned takaomoriyama and daiki-kimura Nov 6, 2024

romeokienzler assigned paolofraccaro Dec 12, 2024

romeokienzler unassigned daiki-kimura, takaomoriyama and jaionet Jan 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add object detection task #156

Add object detection task #156

biancazadrozny commented Sep 12, 2024

paolofraccaro commented Nov 6, 2024

daiki-kimura commented Nov 21, 2024

takaomoriyama commented Dec 2, 2024 •

edited

Loading

takaomoriyama commented Dec 2, 2024

takaomoriyama commented Dec 2, 2024

takaomoriyama commented Dec 17, 2024

romeokienzler commented Jan 7, 2025

Add object detection task #156

Add object detection task #156

Comments

biancazadrozny commented Sep 12, 2024

paolofraccaro commented Nov 6, 2024

daiki-kimura commented Nov 21, 2024

takaomoriyama commented Dec 2, 2024 • edited Loading

takaomoriyama commented Dec 2, 2024

takaomoriyama commented Dec 2, 2024

takaomoriyama commented Dec 17, 2024

romeokienzler commented Jan 7, 2025

takaomoriyama commented Dec 2, 2024 •

edited

Loading