# ZED SDK - Depth Sensing

This sample shows how to retreive the current point cloud.

## Getting Started

 - Get the latest [ZED SDK](https://www.stereolabs.com/developers/release/) and [pyZED Package](https://www.stereolabs.com/docs/app-development/python/install/)
 - Check the [Documentation](https://www.stereolabs.com/docs/)

## Setting up (Optional)

For improved data retrieval and handling on GPU:
 - Install CuPy using pip
    ```sh
    pip install cupy-cuda11x  # For CUDA 11.x
    pip install cupy-cuda12x  # For CUDA 12.x
    ```
 - Install cuda bindings using pip
    ```sh
    pip install cuda-python
    ```

## Run the program

To run the program, use the following command in your terminal : 
```bash
python depth_sensing.py
```
If you wish to run the program from an input_svo_file, or an IP adress, or specify a resolution run : 

```bash
python depth_sensing.py --input_svo_file <input_svo_file> --ip_address <ip_address> --resolution <resolution> 
```
Arguments: 
  - `--input_svo_file` A path to an existing .svo file, that will be playbacked. If this parameter and ip_adress are not specified, the soft will use the camera wired as default.
  - `--ip_address` IP Address, in format a.b.c.d:port or a.b.c.d. If specified, the soft will try to connect to the IP.
  - `--resolution` Resolution, can be either HD2K, HD1200, HD1080, HD720, SVGA or VGA.
  - `--disable_gpu` Disable GPU acceleration even if CuPy is available.

### Features
 - Camera live point cloud is retreived
 - An OpenGL windows displays it in 3D

### Camera Extrinsics

```bash
uv run calibrate_extrinsics.py \ 
   -s output/ -m aruco/markers/standard_box_markers_600mm.parquet \ 
   --aruco-dictionary DICT_APRILTAG_36h11 \ 
   --verify-depth -refine-depth -no-preview \ 
   -—max-samples 20 \ 
   --report-csv output/e2e_refine_depth_smoke.csv \ 
   --auto-align \ 
   --output output/e2e_refine_depth_smoke.json
```

### Visualize Extrinsics

Visualize camera poses and frustums from a JSON extrinsics file using Plotly.

**Basic 3D Visualization (Interactive HTML):**
```bash
uv run visualize_extrinsics.py -i output/extrinsics.json --show
```

**Static Image Export (PNG):**
Requires `kaleido` (installed by default).
```bash
uv run visualize_extrinsics.py -i output/extrinsics.json -o output/viz.png
```

**Bird-Eye View:**
Use `--birdseye` for a top-down X-Z view (looking down Y axis).
```bash
uv run visualize_extrinsics.py -i output/extrinsics.json --birdseye --show
```

**Ground Plane & Origin Overlay:**
Render a semi-transparent X-Z ground plane and/or a world origin triad.
- `--show-ground/--no-show-ground`: Toggle ground plane (default: show).
- `--ground-y FLOAT`: Set the Y height of the plane (default: 0.0).
- `--ground-size FLOAT`: Set the side length of the plane in meters (default: 8.0).
- `--show-origin-axes/--no-show-origin-axes`: Toggle world origin triad (X:red, Y:green, Z:blue) (default: show).

*Example: Ground plane at Y=-1.5 with 10m size*
```bash
uv run visualize_extrinsics.py -i output/extrinsics.json --ground-y -1.5 --ground-size 10 --show
```

**Using ZED Configs for Accurate Frustums:**
Load intrinsics from ZED calibration files to render accurate frustum shapes.
- **Directory**: Matches `SN<serial>.conf` files to camera serials in the JSON.
- **Single File**: Applies one config to ALL cameras (useful for homogeneous setups).
- **Defaults**: Uses `--resolution FHD1200` and `--eye left` unless overridden.

*Example: Load from config directory*
```bash
uv run visualize_extrinsics.py -i output/extrinsics.json \
    --zed-configs /workspaces/zed-playground/zed_settings \
    --resolution HD1080 --show
```

**Troubleshooting:**
- **Cameras bunched up?** Check `--pose-convention`. `world_from_cam` is the standard convention for `calibrate_extrinsics.py` outputs. `cam_from_world` is deprecated.
- **Axes flipped?** Use `--world-basis opengl` to match C++ viewer conventions (X right, Y up, Z backward). Default is `cv` (X right, Y down, Z forward).
- **Config not matching?** Ensure JSON keys match the serial numbers in `SN<serial>.conf` filenames.

For full options:
```bash
uv run visualize_extrinsics.py --help
```

**Diagnostics Mode:**
Run numerical sanity checks on the poses (orthonormality, coplanarity, consistency).
```bash
uv run visualize_extrinsics.py -i output/extrinsics.json --diagnose
```