Writing a wrapper for Orbbec Astra

[joaocandre]

After stumbling into this issue/request, I developed an interest to experiment running OpenPTrack on an Orbbec Astra sensor. How would one go about writing a wrapper for this device?

[bsubbaraman]

Hi @joaocandre, thanks for your interest! I'll actually be posting some notes to the wiki in the next few days about how to integrate new imagers into OpenPTrack. Here's an abridged version:

Each imager needs a ROS wrapper to subscribe to the various topics (i.e. data) needed to run tracking. In fact, it seems that the Astra already has a ROS wrapper that Orbecc has made - hooray! That makes things a lot easier. After installing the wrapper, you will have to make launch files that subscribe to the correct topics. You can proceed by analogy to the Kinectv2/Zed/Realsense by looking at these files.

For example, in the case of the RealSense, we first launch the imager using the launch file provided from the wrapper. Then for each tracking module (person/object/pose tracking), we launch a generic tracking node (which is the same for every imager) and an imager specific launch file. In this imager-specific file, you'll need to change the topic names to match those published by the Orbecc wrapper.

There may be further issues/peculiarities you run into while doing this, but hopefully this helps you get started. Again, I'll be adding notes to the wiki about this, and will update this thread when that has been completed!

[joaocandre]

@bsubbaraman Thank you very much for the info. Will try to get it working as soon as I can.

[bsubbaraman]

FYI- I've starting collecting notes here

[colinbouvry]

Thank you very much for the info.
Can anyone share astra configuration ?

[JoaoCarlosLL]

Hello, I'm trying to use the astra camera with a multi camera, but I'm having problems with the calibration. When I run the calibration_master file and then the sensor calibration runs as follows:

roslaunch opt_calibration opt_calibration_master.launch 

SUMMARY
========

PARAMETERS
 * /opt_calibration/cell_height: 0.115
 * /opt_calibration/cell_width: 0.115
 * /opt_calibration/cols: 5
 * /opt_calibration/num_sensors: 1
 * /opt_calibration/rows: 6
 * /opt_calibration/sensor_0/name: /realsense01
 * /opt_calibration/sensor_0/type: pinhole_rgb
 * /opt_calibration/world_computation: last_checkerboard
 * /rosdistro: melodic
 * /rosversion: 1.14.3

NODES
  /
    opt_calibration (opt_calibration/opt_calibration)
    opt_calibration_status_plot (opt_calibration/status_plot.py)
    rviz (rviz/rviz)

auto-starting new master
process[master]: started with pid [2215]
ROS_MASTER_URI=http://192.168.1.6:11311/

setting /run_id to 2a221d54-c250-11ec-8c8d-b46bfc1475ad
process[rosout-1]: started with pid [2226]
started core service [/rosout]
process[rviz-2]: started with pid [2229]
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
process[opt_calibration_status_plot-3]: started with pid [2234]
process[opt_calibration-4]: started with pid [2243]
wassup
[ INFO] [1650641013.148697509]: /realsense01 added.
[ WARN] [1650641013.156659960]: Not all messages received. Waiting...
[ WARN] [1650641018.156818304]: Not all messages received. Waiting...
[ WARN] [1650641023.156800529]: Not all messages received. Waiting...
[ WARN] [1650641028.156827363]: Not all messages received. Waiting...
[ WARN] [1650641033.157010826]: Not all messages received. Waiting...
[ INFO] [1650641038.156774758]: All sensors connected.
not initialized

roslaunch opt_calibration sensor_realsense01.launch

SUMMARY
========

PARAMETERS
 * /realsense01/depth_rectify_depth/interpolation: 0
 * /realsense01/depth_registered_rectify_depth/interpolation: 0
 * /realsense01/driver/auto_exposure: True
 * /realsense01/driver/auto_white_balance: True
 * /realsense01/driver/bootorder: 0
 * /realsense01/driver/color_depth_synchronization: False
 * /realsense01/driver/depth_camera_info_url: 
 * /realsense01/driver/depth_frame_id: realsense01_depth...
 * /realsense01/driver/depth_registration: True
 * /realsense01/driver/device_id: #1
 * /realsense01/driver/devnums: 1
 * /realsense01/driver/rgb_camera_info_url: 
 * /realsense01/driver/rgb_frame_id: realsense01_rgb_o...
 * /realsense01/realsense01_nodelet_manager/num_worker_threads: 4
 * /realsense01/realsense01_rgb/camera_info_url: 
 * /realsense01/realsense01_rgb/frame_rate: 30
 * /realsense01/realsense01_rgb/height: 480
 * /realsense01/realsense01_rgb/index: 0
 * /realsense01/realsense01_rgb/product: 0x0501
 * /realsense01/realsense01_rgb/serial: 0
 * /realsense01/realsense01_rgb/timestamp_method: start
 * /realsense01/realsense01_rgb/vendor: 0x2bc5
 * /realsense01/realsense01_rgb/video_mode: mjpeg
 * /realsense01/realsense01_rgb/width: 640
 * /rosdistro: melodic
 * /rosversion: 1.14.3

NODES
  /realsense01/
    depth_metric (nodelet/nodelet)
    depth_metric_rect (nodelet/nodelet)
    depth_points (nodelet/nodelet)
    depth_rectify_depth (nodelet/nodelet)
    depth_registered_hw_metric_rect (nodelet/nodelet)
    depth_registered_metric (nodelet/nodelet)
    depth_registered_rectify_depth (nodelet/nodelet)
    driver (nodelet/nodelet)
    points_xyzrgb_hw_registered (nodelet/nodelet)
    realsense01_nodelet_manager (nodelet/nodelet)
    realsense01_rgb (astra_camera/camera_node)
    rgb_rectify_color (nodelet/nodelet)
  /
    realsense01_base_link (tf2_ros/static_transform_publisher)
    realsense01_base_link1 (tf2_ros/static_transform_publisher)
    realsense01_base_link2 (tf2_ros/static_transform_publisher)
    realsense01_base_link3 (tf2_ros/static_transform_publisher)

ROS_MASTER_URI=http://192.168.1.6:11311/

process[realsense01/realsense01_rgb-1]: started with pid [2397]
unsupported descriptor subtype VS_COLORFORMAT
unsupported descriptor subtype VS_COLORFORMAT
attempt to claim already-claimed interface 1
process[realsense01/realsense01_nodelet_manager-2]: started with pid [2398]
[ INFO] [1650641034.873645920]: Initializing nodelet with 4 worker threads.
process[realsense01/driver-3]: started with pid [2413]
[ INFO] [1650641035.447963563]: Device "2bc5/0403@1/7" found.
[ WARN] [1650641035.450073611]: Unable to set scanning_mode to 0
[ WARN] [1650641035.451223946]: Unable to set auto_exposure to 8
[ WARN] [1650641035.452527229]: Unable to set auto_exposure_priority to 0
[ WARN] [1650641035.453556926]: Unable to set exposure_absolute to 1
[ WARN] [1650641035.454570245]: Unable to set auto_focus to 1
[ WARN] [1650641035.455580053]: Unable to set focus_absolute to 0
[ WARN] [1650641035.456774423]: Unable to set gain to 0
[ WARN] [1650641035.458143104]: Unable to set iris_absolute to 0
[ WARN] [1650641035.480354064]: Unable to set pantilt to 0, 0
[ INFO] [1650641035.503120192]: device name: Orbbec Astra Pro
process[realsense01/rgb_rectify_color-4]: started with pid [2425]
[ WARN] [1650641035.701672836]: Camera calibration file /root/.ros/camera_info/camera.yaml not found.
process[realsense01/depth_rectify_depth-5]: started with pid [2446]
process[realsense01/depth_metric_rect-6]: started with pid [2457]
process[realsense01/depth_metric-7]: started with pid [2478]
process[realsense01/depth_points-8]: started with pid [2485]
process[realsense01/depth_registered_rectify_depth-9]: started with pid [2492]
process[realsense01/points_xyzrgb_hw_registered-10]: started with pid [2499]
process[realsense01/depth_registered_hw_metric_rect-11]: started with pid [2506]
process[realsense01/depth_registered_metric-12]: started with pid [2541]
process[realsense01_base_link-13]: started with pid [2549]
process[realsense01_base_link1-14]: started with pid [2556]
process[realsense01_base_link2-15]: started with pid [2562]
process[realsense01_base_link3-16]: started with pid [2568]

I'm using realsense files as an example.

The sensor launches normally, but I can't get the calibration to work. What could I do to resolve this?

For now I'm running in single camera and the system works fine, but now I'm moving to a multi camera system.

(Sorry I'm new to this system and still learning)