zed-playground/py_workspace/README.md

ZED SDK - Depth Sensing

This sample shows how to retreive the current point cloud.

Getting Started

• Get the latest ZED SDK and pyZED Package
• Check the Documentation

Setting up (Optional)

For improved data retrieval and handling on GPU:

• Install CuPy using pip

  pip install cupy-cuda11x  # For CUDA 11.x
  pip install cupy-cuda12x  # For CUDA 12.x
  

• Install cuda bindings using pip

  pip install cuda-python
  

Run the program

To run the program, use the following command in your terminal :

python depth_sensing.py

If you wish to run the program from an input_svo_file, or an IP adress, or specify a resolution run :

python depth_sensing.py --input_svo_file <input_svo_file> --ip_address <ip_address> --resolution <resolution> 

Arguments:

• --input_svo_file A path to an existing .svo file, that will be playbacked. If this parameter and ip_adress are not specified, the soft will use the camera wired as default.
• --ip_address IP Address, in format a.b.c.d:port or a.b.c.d. If specified, the soft will try to connect to the IP.
• --resolution Resolution, can be either HD2K, HD1200, HD1080, HD720, SVGA or VGA.
• --disable_gpu Disable GPU acceleration even if CuPy is available.

Features

• Camera live point cloud is retreived
• An OpenGL windows displays it in 3D

Camera Extrinsics

uv run calibrate_extrinsics.py \ 
   -s output/ -m aruco/markers/standard_box_markers_600mm.parquet \ 
   --aruco-dictionary DICT_APRILTAG_36h11 \ 
   --verify-depth -refine-depth -no-preview \ 
   -—max-samples 20 \ 
   --report-csv output/e2e_refine_depth_smoke.csv \ 
   --auto-align \ 
   --output output/e2e_refine_depth_smoke.json

Visualize Extrinsics

Visualize camera poses and frustums from a JSON extrinsics file using Plotly.

Basic 3D Visualization (Interactive HTML):

uv run visualize_extrinsics.py -i output/extrinsics.json --show

Static Image Export (PNG): Requires kaleido (installed by default).

uv run visualize_extrinsics.py -i output/extrinsics.json -o output/viz.png

Bird-Eye View: Use --birdseye for a top-down X-Z view (looking down Y axis).

uv run visualize_extrinsics.py -i output/extrinsics.json --birdseye --show

Ground Plane & Origin Overlay: Render a semi-transparent X-Z ground plane and/or a world origin triad.

• --show-ground/--no-show-ground: Toggle ground plane (default: show).
• --ground-y FLOAT: Set the Y height of the plane (default: 0.0).
• --ground-size FLOAT: Set the side length of the plane in meters (default: 8.0).
• --show-origin-axes/--no-show-origin-axes: Toggle world origin triad (X:red, Y:green, Z:blue) (default: show).

Example: Ground plane at Y=-1.5 with 10m size

uv run visualize_extrinsics.py -i output/extrinsics.json --ground-y -1.5 --ground-size 10 --show

Using ZED Configs for Accurate Frustums: Load intrinsics from ZED calibration files to render accurate frustum shapes.

• Directory: Matches SN<serial>.conf files to camera serials in the JSON.
• Single File: Applies one config to ALL cameras (useful for homogeneous setups).
• Defaults: Uses --resolution FHD1200 and --eye left unless overridden.

Example: Load from config directory

uv run visualize_extrinsics.py -i output/extrinsics.json \
    --zed-configs /workspaces/zed-playground/zed_settings \
    --resolution HD1080 --show

Troubleshooting:

• Cameras bunched up? Check --pose-convention. world_from_cam is the standard convention for calibrate_extrinsics.py outputs. cam_from_world is deprecated.
• Axes flipped? Use --world-basis opengl to match C++ viewer conventions (X right, Y up, Z backward). Default is cv (X right, Y down, Z forward).
• Config not matching? Ensure JSON keys match the serial numbers in SN<serial>.conf filenames.

For full options:

uv run visualize_extrinsics.py --help

Diagnostics Mode: Run numerical sanity checks on the poses (orthonormality, coplanarity, consistency).

uv run visualize_extrinsics.py -i output/extrinsics.json --diagnose