Version
中文 | English

6.6. Diagnostic and Latency Analysis Tools

This section describes frame continuity, timestamp, and runtime status diagnostics.

6.6.1. Diagnostic Tool: Frame Drop Log and Timestamp CSV Recording

After enable_frame_drop_log is enabled, the camera node prints frame drop statistics for color, depth, left color, right color, left IR, and right IR in the log. This helps locate frame drops at the SDK receive stage and ROS publish stage. When frame_timestamp_csv_file is set, the camera node records timestamp data for color, depth, left_color, right_color, left_ir, right_ir, and enabled IMU streams to CSV files for analyzing frame continuity, publish latency, and timestamp anomalies. When frame synchronization (enable_frame_sync:=true) is enabled, color and depth timestamps are combined into one CSV file; otherwise, they are written to separate <stem>_color.csv and <stem>_depth.csv files. Left color, right color, left IR, and right IR are always written to separate files with the corresponding stream-name suffix. Synchronized accelerometer and gyroscope data are written to <stem>_imu.csv, while standalone accelerometer and gyroscope data are written to <stem>_accel.csv and <stem>_gyro.csv. All CSV files follow the same rotation rule.

roslaunch orbbec_camera gemini_330_series.launch \
enable_frame_drop_log:=true \
frame_timestamp_csv_file:=/tmp/frame_timestamp.csv

The CSV contains SDK frame index, hardware frame number, sensor timestamp, device/global/system timestamp, steady arrival/publish delta, ROS publish duration, and SDK delay fields.

6.6.1.1. CSV Rotation

Each CSV file contains at most 1,024,575 frame rows plus the header. When the limit is reached, the logger automatically starts the next indexed file, for example frame_timestamp_1.csv and frame_timestamp_2.csv.

6.6.1.2. Timestamp CSV Field Description

The CSV contains groups of identically structured fields for each image stream. In synchronized records, the color and depth groups use the color_ and depth_ prefixes; standalone records also use prefixes such as left_color_, right_color_, left_ir_, and right_ir_. The groups have identical definitions and differ only in data source.

Field suffix Description Unit / Notes
_sdk_frame_index SDK frame index frame->index()
_hardware_frame_number Hardware frame number frame->getMetadataValue(OB_FRAME_METADATA_TYPE_FRAME_NUMBER)
_sensor_ts_sec Sensor timestamp Seconds, usually exposure midpoint
_sensor_ts_delta_us Delta between adjacent sensor timestamps us
_device_ts_sec Device clock timestamp Seconds
_device_ts_delta_us Delta between adjacent device timestamps us
_global_ts_sec Global timestamp Seconds
_global_ts_delta_us Delta between adjacent global timestamps us
_system_ts_sec SDK system timestamp Seconds
_system_ts_delta_us Delta between adjacent SDK system timestamps us
_arrival_steady_delta_us Delta between adjacent host steady arrival times us
_publish_steady_delta_us Delta between adjacent host steady times before publishing us
_arrival_to_publish_steady_us Time from ROS receiving the frame to publishing it (steady) publish_steady - arrival_steady
_sdk_delay_from_global_us SDK publish delay (global reference) arrival_system - global_ts
_sdk_delay_from_system_us SDK publish delay (system reference) arrival_system - sdk_system_ts

6.6.1.3. Timestamp CSV Analysis Methods

6.6.1.3.1. Detecting Hardware Frame Drops from CSV

  • Check whether _hardware_frame_number is continuous.

  • Plot _sensor_ts_delta_us as a line chart or scatter plot and check for obvious jumps.

  • For example, at 30 fps, the adjacent frame interval should usually be close to 33333 us.

6.6.1.3.2. Detecting SDK / ROS Frame Drops from CSV

  • Check whether _sdk_frame_index is continuous.

  • Plot _device_ts_delta_us, _global_ts_delta_us, and _system_ts_delta_us as line charts or scatter plots and check for jumps.

  • After enable_frame_drop_log is enabled, stage=SDK_RECEIVE in the log indicates frame drops detected at the SDK receive stage, and stage=ROS_PUBLISH indicates frame drops detected at the ROS publish stage.

6.6.1.3.3. Latency Analysis from CSV

  • SDK latency: check _sdk_delay_from_global_us and _sdk_delay_from_system_us as line charts or scatter plots to observe delay changes from the low-level timestamp to arrival at the ROS node.

  • ROS latency: check _arrival_to_publish_steady_us as a line chart or scatter plot to measure the time from the SDK callback receiving a frame to publishing the image on the ROS side.

  • If you need a value closer to real processing time, prefer fields related to the steady clock.

6.6.1.3.4. Timestamp CSV Synchronization Note

This CSV is mainly used to analyze continuity and latency for each image stream. It cannot directly measure synchronization between color and depth.

6.6.2. monitor_fd.sh

monitor_fd.sh prints the file descriptor count of camera-related processes once per second. It is useful for checking file descriptor leaks during long-running tests.

cd src/OrbbecSDK_ROS1/scripts
./monitor_fd.sh