feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor #9454

knzo25 · 2024-11-25T08:31:06Z

Description

This PR is part of a series of PRs that aim to accelerate the Sensing/Perception pipeline through an appropriate use of CUDA.

List of PRs:

feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor #9454 (pointcloud preprocessing)
feat(autoware_pointcloud_preprocessor): cuda-accelerated pointcloud concatenation #9455 (concatenation)
feat(autoware_lidar_centerpoint): added the cuda_blackboard to centerpoint #9453 (centerpoint)
transfusion (TODO)
feat(autoware_probabilistic_occupancy_grid_map): cuda accelerated implementation #9542 (OGM - the first implementation will be independent of the blackboard to ease the transition)
feat: acceleration and transport layer tier4/aip_launcher#348 (aip_launcher)
feat: acceleration and transport layer sample_sensor_kit_launch#111 (sample_sensor_kit_launch)

To use these branches, the following additions to the autoware.repos are necessary:

  vendor/cuda_blackboard:
    type: git
    url: [email protected]:knzo25/cuda_blackboard.git
    version: main
  vendor/negotiated:
    type: git
    url: https://github.com/osrf/negotiated.git
    version: master

Depending on your machine and how many nodes are in a container, the following branch may also be required:
https://github.com/knzo25/launch_ros/tree/fix/load_composable_node
There seems to be a but in ROS where if you send too many services at once some will be lost and ros_launch can not handle that.

How was this PR tested?

The sensing/perception pipeline was tested until centerpoint for TIER IV's taxi using the logging simulator.
The following tests were executed in a laptop equipped with a RTX 4060 (laptop) GPU and a Intel(R) Core(TM) Ultra 7 165H (22 cores)

Node / processing time [ms]	Current	PR
/sensing/lidar/top/crop_box_filter_self/debug/processing_time_ms	5.81	N/A
/sensing/lidar/top/crop_box_filter_mirror/debug/processing_time_ms	4.59	N/A
/sensing/lidar/top/distortion_corrector/debug/processing_time_ms	10.96	N/A
/sensing/lidar/top/ring_outlier_filter/debug/processing_time_ms	10.69	N/A
/sensing/lidar/top/cuda_pointcloud_preprocessor/debug/processing_time_ms	N/A	3.08 (2.03 are H->D copies)
/sensing/lidar/concatenate_data_synchronizer/debug/processing_time_ms	7.83	0.70
Total	38.8	3.78

10.26 speedup!

Notes for reviewers

The main branch that I used for development is feat/cuda_acceleration_and_transport_layer.
However, the changes were too big so I split the PRs. That being said, development, if any will still be on that branch (and then cherrypicked to the respective PRs), and the review changes will be cherrypicked into the development branch.

Interface changes

An additional topic is added to perform type negotiation:
Example: input/pointcloud -> input/pointcloud and input/pointcloud/cuda

Effects on system behavior

Enabling this preprocessing in the launchers should provide a much reduced latency and cpu usage (at the cost of a higher GPU usage)

…sonal repository Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

github-actions · 2024-11-25T08:31:26Z

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

You've checked our contribution guidelines.
Your PR follows our pull request guidelines.
All required CI checks pass before marking the PR ready for review.

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

…pointcloud changes after the first iteration Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

…ntcloud_preprocessing

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

mojomex

Thank you for the amazing PR, these performance improvements are desperately needed.

I haven't checked the PR for functionality yet, but I'll leave my first round of comments here.

The main points I'd like to address are

memory safety and idiomatic C++ (there is currently a lot of raw-pointer code which should be avoided whenever possible)
modulatiry: currently the pipeline is hard-coded and all in one place. This makes the module hard to adapt to different projects, and hard to maintain individual modules in the pipeline.

Thank you for your time!

sensing/autoware_cuda_pointcloud_preprocessor/README.md

sensing/autoware_cuda_pointcloud_preprocessor/config/cuda_pointcloud_preprocessor.param.yaml

sensing/autoware_cuda_pointcloud_preprocessor/README.md

sensing/autoware_cuda_pointcloud_preprocessor/docs/cuda-pointcloud-preprocessor.md

mojomex · 2024-12-24T05:41:22Z

...reprocessor/src/cuda_organized_pointcloud_adapter/cuda_organized_pointcloud_adapter_node.cpp

+
+  std::size_t max_ring = 0;
+
+  for (std::size_t i = 0; i < input_pointcloud_msg_ptr->width * input_pointcloud_msg_ptr->height;


Iteration without explicit bounds checking of the underlying array is not memory-safe. Thus, I would suggest using the abovementioned PointCloud2Iterators here.

Sorry, but I ended up removing the adapter since it was heavier than the preprocessing 😅
59144d8 and 5482f9c

mojomex · 2024-12-24T05:41:27Z

...reprocessor/src/cuda_organized_pointcloud_adapter/cuda_organized_pointcloud_adapter_node.cpp

+  num_rings_ = std::max(num_rings_, static_cast<std::size_t>(16));
+  std::vector<std::size_t> ring_points(num_rings_, 0);
+
+  for (std::size_t i = 0; i < input_pointcloud_msg_ptr->width * input_pointcloud_msg_ptr->height;


Iteration without explicit bounds checking of the underlying array is not memory-safe. Thus, I would suggest using the abovementioned PointCloud2Iterators here.

Sorry, but I ended up removing the adapter since it was heavier than the preprocessing 😅
59144d8 and 5482f9c

mojomex · 2024-12-24T05:42:34Z

...reprocessor/src/cuda_organized_pointcloud_adapter/cuda_organized_pointcloud_adapter_node.cpp

+    max_ring = std::max(max_ring, ring);
+  }
+
+  // Set max rings to the next power of two


Admittedly kind of a niche problem, but not all sensors (Pandar40P) have 2^n rings.

Although auto-detecting the number of rings is nice, it has no hard guarantee to be accurate (e.g. the sensor is under a cover when turned on and there are thus no points in the cloud).

Does cuda_pointcloud_preprocessor support changing dimenions of input pointclouds across iterations (e.g. starts with 0 rings in cloud 1, then 64 rings with 2000 points, then 64 rings with 5000 points each)?
If not, I'd suggest to make n_rings and max_points_per_ring parameters so that we can guarantee correct behavior at runtime.

In the new version
59144d8 and 5482f9c

Autodetection is performed automatically with very little overhead 👍

mojomex · 2024-12-24T05:47:44Z

...reprocessor/src/cuda_organized_pointcloud_adapter/cuda_organized_pointcloud_adapter_node.cpp

+bool CudaOrganizedPointcloudAdapterNode::orderPointcloud(
+  const sensor_msgs::msg::PointCloud2::ConstSharedPtr input_pointcloud_msg_ptr)
+{
+  const autoware::point_types::PointXYZIRCAEDT * input_buffer =


Same comment about bounds/type checking as above 🙇

Sorry, but I ended up removing the adapter since it was heavier than the preprocessing 😅
59144d8 and 5482f9c

mojomex · 2024-12-24T05:57:53Z

...uda_pointcloud_preprocessor/src/cuda_pointcloud_preprocessor/cuda_pointcloud_preprocessor.cu

+  if (idx < num_points && masks[idx] == 1) {
+    output_points[indices[idx] - 1] = input_points[idx];
+  }
+}


These two functions are identical except for their argument types. Consider making one templated function instead.

Unused kernels were deleted in
303b9ed

manato

@knzo25
Thank you very much for proposing a fantastic PR, and I'm sorry for taking a long time for the review. From a viewpoint of CUDA usage, I left some comments. I'd appreciate it if you could consider them.

...uda_pointcloud_preprocessor/src/cuda_pointcloud_preprocessor/cuda_pointcloud_preprocessor.cu

…loud-preprocessor.md Co-authored-by: Max Schmeller <[email protected]>

…oud_preprocessor/cuda_pointcloud_preprocessor.cu Co-authored-by: Manato Hirabayashi <[email protected]>

…loud-preprocessor.md Co-authored-by: Max Schmeller <[email protected]>

Co-authored-by: Max Schmeller <[email protected]>

…oud_preprocessor/cuda_pointcloud_preprocessor.cu Co-authored-by: Max Schmeller <[email protected]>

…oud_preprocessor/cuda_pointcloud_preprocessor.cu Co-authored-by: Manato Hirabayashi <[email protected]>

…l proposal Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

… approach is actually faster Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

…ized way Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

manato

@knzo25
Thank you for responding to my comments. I viewed the code and confirmed my major concern has been gone. Once the comments from @mojomex are solved, I believe this PR can be merged.

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

…y bottleneck is the H->D copy Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

knzo25 · 2025-02-05T07:13:31Z

@mojomex
Max, I am sorry, but in 59144d8 and 5482f9c I changed a lot of things by deleting the cuda adapter node and moving its implementation to cuda (now the only hotspot is the host -> device copy, as you can see in the updated table)

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

… later PR Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

…b.com:knzo25/autoware.universe into feat/cuda_blackboard_pointcloud_preprocessing

knzo25 · 2025-02-05T09:55:49Z

@manato
It seems to me that the host -> device transfer is too slow. It is paged (ROS), but it still seems off to me. Do you have a way to check these concerns? The theoretical bandwidth that I got with the cuda samples was super different than the bandwidth reported by nsight 😢 and there was nothing else happening on the GPU at the time

…ring it with the baseline Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

amadeuszsz

Added few minor comments. Regarding runtime, there was an issue with point cloud rings (see attached image), but I believe it's already fixed in f100224

About package architecture, I wonder if keeping cuda kernels with base c++ functions in same .cu files is the best way for doing this. On opposite side, kernels consist of actual cuda code and helper launch functions. I will let it up to you as you better see amount of remaining code here and potentially future development path. It may not be worth doubling the number of files.

For cuda kernels code, I will address my comments in next review, I need more time 🙏🏻

amadeuszsz · 2025-02-05T09:36:14Z

sensing/autoware_cuda_pointcloud_preprocessor/launch/cuda_pointcloud_preprocessor.launch.xml

+  <arg name="input/twist" default="/sensing/vehicle_velocity_converter/twist_with_covariance"/>
+  <arg name="output/pointcloud" default="/sensing/lidar/top/test"/>
+
+  <arg name="cuda_pointcloud_preprocessor_param_file" default="$(find-pkg-share cuda_pointcloud_preprocessor)/config/cuda_pointcloud_preprocessor.param.yaml"/>


Package name is autoware_cuda_pointcloud_preprocessor

amadeuszsz · 2025-02-05T09:36:29Z

sensing/autoware_cuda_pointcloud_preprocessor/launch/cuda_pointcloud_preprocessor.launch.xml

+
+  <arg name="cuda_pointcloud_preprocessor_param_file" default="$(find-pkg-share cuda_pointcloud_preprocessor)/config/cuda_pointcloud_preprocessor.param.yaml"/>
+
+  <node pkg="cuda_pointcloud_preprocessor" exec="cuda_pointcloud_preprocessor_node" name="cuda_pointcloud_preprocessor" output="screen">


Package name is autoware_cuda_pointcloud_preprocessor

amadeuszsz · 2025-02-05T10:43:35Z

.../autoware_cuda_pointcloud_preprocessor/src/cuda_pointcloud_preprocessor/undistort_kernels.cu

+    twist.cum_y = cum_y;
+    twist.cum_theta = cum_theta;
+
+    std::uint64_t twist_global_stamp_nsec = twist_stamp;


There is a case where twist_stamp is uninitialized.

amadeuszsz · 2025-02-05T11:08:51Z

sensing/autoware_cuda_pointcloud_preprocessor/CMakeLists.txt

+if(NOT ${CUDA_FOUND})
+  message(WARNING "cuda was not found, so the autoware_cuda_pointcloud_preprocessor package will not be built.")
+  return()
+endif()


nit:
Consider to add

elseif(CMAKE_BUILD_TYPE STREQUAL "Debug") list(APPEND CMAKE_CUDA_FLAGS "-g" "-G") list(APPEND CUDA_NVCC_FLAGS "-g" "-G") endif()

to make cuda kernels debugging possible.

amadeuszsz · 2025-02-05T11:09:39Z

sensing/autoware_cuda_pointcloud_preprocessor/CMakeLists.txt

+    add_compile_definitions(ROS_DISTRO_GALACTIC)
+elseif(${ROS_DISTRO} STREQUAL "humble")
+    add_compile_definitions(ROS_DISTRO_HUMBLE)
+endif()


nit:
Can we already address iron and jazzy distro?

amadeuszsz · 2025-02-05T11:14:06Z

..._preprocessor/include/autoware/cuda_pointcloud_preprocessor/cuda_pointcloud_preprocessor.hpp

+
+  void setCropBoxParameters(const std::vector<CropBoxParameters> & crop_box_parameters);
+  void setRingOutlierFilterParameters(const RingOutlierFilterParameters & ring_outlier_parameters);
+  void set3DUndistortion(bool value);


Function declarations and definitions usually consist of same parameter names (value vs. use_3d_undistortion)

amadeuszsz · 2025-02-05T11:15:19Z

...rocessor/include/autoware/cuda_pointcloud_preprocessor/cuda_pointcloud_preprocessor_node.hpp

+
+#include <deque>
+#include <memory>
+#include <vector>


Unused header

amadeuszsz · 2025-02-05T11:16:04Z

...da_pointcloud_preprocessor/include/autoware/cuda_pointcloud_preprocessor/outlier_kernels.hpp

+#define AUTOWARE__CUDA_POINTCLOUD_PREPROCESSOR__OUTLIER_KERNELS_HPP_
+
+#include "autoware/cuda_pointcloud_preprocessor/point_types.hpp"
+#include "autoware/cuda_pointcloud_preprocessor/types.hpp"


Unused header

amadeuszsz · 2025-02-05T11:16:13Z

...a_pointcloud_preprocessor/include/autoware/cuda_pointcloud_preprocessor/organize_kernels.hpp

+#define AUTOWARE__CUDA_POINTCLOUD_PREPROCESSOR__ORGANIZE_KERNELS_HPP_
+
+#include "autoware/cuda_pointcloud_preprocessor/point_types.hpp"
+#include "autoware/cuda_pointcloud_preprocessor/types.hpp"


Unused header

feat: moved the cuda pointcloud preprocessor and organized from a per…

af2e884

…sonal repository Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

github-actions bot added type:documentation Creating or refining documentation. (auto-assigned) component:sensing Data acquisition from sensors, drivers, preprocessing. (auto-assigned) tag:require-cuda-build-and-test labels Nov 25, 2024

knzo25 self-assigned this Nov 25, 2024

knzo25 added 4 commits November 25, 2024 18:06

chore: fixed incorrect links

774e099

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

chore: fixed dead links pt2

be04f76

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

chore: fixed spelling errors

db02ec7

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

chore: json schema fixes

5218a4a

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

This was referenced Nov 25, 2024

feat(autoware_lidar_centerpoint): added the cuda_blackboard to centerpoint #9453

Open

feat(autoware_pointcloud_preprocessor): cuda-accelerated pointcloud concatenation #9455

Draft

feat: acceleration and transport layer tier4/aip_launcher#348

Open

knzo25 marked this pull request as ready for review November 25, 2024 09:47

yukkysaito requested review from YoshiRi and drwnz November 26, 2024 01:22

knzo25 added 2 commits November 26, 2024 13:40

chore: removed comments and filled the fields

4a9daaf

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

fix: fixed the adapter for the case when the number of points in the …

84a4b9f

…pointcloud changes after the first iteration Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

knzo25 mentioned this pull request Nov 26, 2024

feat: acceleration and transport layer autowarefoundation/sample_sensor_kit_launch#111

Draft

knzo25 requested review from manato, mojomex and amadeuszsz November 26, 2024 05:38

knzo25 added 2 commits December 23, 2024 14:12

Merge remote-tracking branch 'awf/main' into feat/cuda_blackboard_poi…

1e27534

…ntcloud_preprocessing

feat: used the cuda host allocators for aster host to device copies

b3c1d72

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

knzo25 mentioned this pull request Dec 23, 2024

Leverage cuda acceleration in the sensing perception pipeline #9722

Open

3 tasks

technolojin self-requested a review December 24, 2024 01:22

mojomex requested changes Dec 24, 2024

View reviewed changes

manato reviewed Jan 7, 2025

View reviewed changes

knzo25 and others added 2 commits January 10, 2025 18:29

Update sensing/autoware_cuda_pointcloud_preprocessor/docs/cuda-pointc…

887f162

…loud-preprocessor.md Co-authored-by: Max Schmeller <[email protected]>

Update sensing/autoware_cuda_pointcloud_preprocessor/src/cuda_pointcl…

0b46fb6

…oud_preprocessor/cuda_pointcloud_preprocessor.cu Co-authored-by: Manato Hirabayashi <[email protected]>

knzo25 and others added 14 commits January 10, 2025 18:32

Update sensing/autoware_cuda_pointcloud_preprocessor/src/cuda_pointcl…

970055e

…oud_preprocessor/cuda_pointcloud_preprocessor.cu Co-authored-by: Manato Hirabayashi <[email protected]>

style(pre-commit): autofix

71b8a6f

Update sensing/autoware_cuda_pointcloud_preprocessor/docs/cuda-pointc…

d6ee47f

…loud-preprocessor.md Co-authored-by: Max Schmeller <[email protected]>

Update sensing/autoware_cuda_pointcloud_preprocessor/README.md

3f6953f

Co-authored-by: Max Schmeller <[email protected]>

Update sensing/autoware_cuda_pointcloud_preprocessor/README.md

db99ba6

Co-authored-by: Max Schmeller <[email protected]>

Update sensing/autoware_cuda_pointcloud_preprocessor/src/cuda_pointcl…

9ce2e06

…oud_preprocessor/cuda_pointcloud_preprocessor.cu Co-authored-by: Max Schmeller <[email protected]>

style(pre-commit): autofix

f474900

Update sensing/autoware_cuda_pointcloud_preprocessor/src/cuda_pointcl…

5941435

…oud_preprocessor/cuda_pointcloud_preprocessor.cu Co-authored-by: Manato Hirabayashi <[email protected]>

style(pre-commit): autofix

90ad0b2

Update sensing/autoware_cuda_pointcloud_preprocessor/src/cuda_pointcl…

13aff8c

…oud_preprocessor/cuda_pointcloud_preprocessor.cu Co-authored-by: Manato Hirabayashi <[email protected]>

style(pre-commit): autofix

f0e4c20

chore: fixed code compilation to reflect Hirabayashi-san's memory poo…

d05b988

…l proposal Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

feat: generalized the number of crop boxes. For two at least, the new…

5e0cc47

… approach is actually faster Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

chore: updated config, schema, and handled the null case in a special…

6baaea2

…ized way Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

manato approved these changes Jan 28, 2025

View reviewed changes

knzo25 added 2 commits February 5, 2025 16:11

feat: moving the pointcloud organization into gpu

59144d8

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

feat: reimplemented the organized pointcloud adapter in cuda. the onl…

5482f9c

…y bottleneck is the H->D copy Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

amadeuszsz and others added 6 commits February 5, 2025 17:52

Merge branch 'main' into feat/cuda_blackboard_pointcloud_preprocessing

6660210

chore: removed redundant ternay operator

c554290

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

chore: added a temporary memory check. the check will be unified in a…

d6ad9db

… later PR Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

chore: refactored the structure to avoid large files

303b9ed

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

chore: updated the copyright year

17799e5

Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

Merge branch 'feat/cuda_blackboard_pointcloud_preprocessing' of githu…

5f9c4a6

…b.com:knzo25/autoware.universe into feat/cuda_blackboard_pointcloud_preprocessing

knzo25 requested a review from mojomex February 5, 2025 09:34

fix: fixed a bug in the undistortion kernel setup. validated it compa…

f100224

…ring it with the baseline Signed-off-by: Kenzo Lobos-Tsunekawa <[email protected]>

amadeuszsz requested changes Feb 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor #9454

feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor #9454

knzo25 commented Nov 25, 2024 •

edited

Loading

github-actions bot commented Nov 25, 2024 •

edited

Loading

mojomex left a comment

mojomex Dec 24, 2024

knzo25 Feb 5, 2025

mojomex Dec 24, 2024

knzo25 Feb 5, 2025

mojomex Dec 24, 2024

knzo25 Feb 5, 2025

mojomex Dec 24, 2024

knzo25 Feb 5, 2025

mojomex Dec 24, 2024

knzo25 Feb 5, 2025

manato left a comment

manato left a comment

knzo25 commented Feb 5, 2025

knzo25 commented Feb 5, 2025

amadeuszsz left a comment

amadeuszsz Feb 5, 2025

amadeuszsz Feb 5, 2025

amadeuszsz Feb 5, 2025

amadeuszsz Feb 5, 2025

amadeuszsz Feb 5, 2025

amadeuszsz Feb 5, 2025

amadeuszsz Feb 5, 2025

amadeuszsz Feb 5, 2025

amadeuszsz Feb 5, 2025


		std::size_t max_ring = 0;

		for (std::size_t i = 0; i < input_pointcloud_msg_ptr->width * input_pointcloud_msg_ptr->height;


		<arg name="cuda_pointcloud_preprocessor_param_file" default="$(find-pkg-share cuda_pointcloud_preprocessor)/config/cuda_pointcloud_preprocessor.param.yaml"/>

		<node pkg="cuda_pointcloud_preprocessor" exec="cuda_pointcloud_preprocessor_node" name="cuda_pointcloud_preprocessor" output="screen">

feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor #9454

Are you sure you want to change the base?

feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor #9454

Conversation

knzo25 commented Nov 25, 2024 • edited Loading

Description

Related links

How was this PR tested?

Notes for reviewers

Interface changes

Effects on system behavior

github-actions bot commented Nov 25, 2024 • edited Loading

mojomex left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

manato left a comment

Choose a reason for hiding this comment

manato left a comment

Choose a reason for hiding this comment

knzo25 commented Feb 5, 2025

knzo25 commented Feb 5, 2025

amadeuszsz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

knzo25 commented Nov 25, 2024 •

edited

Loading

github-actions bot commented Nov 25, 2024 •

edited

Loading