# Agent Context & Reminders

## ZED SDK Architecture

### Streaming API vs Fusion API

The ZED SDK provides two distinct network APIs that are often confused:

| Feature | Streaming API | Fusion API |
|---------|---------------|------------|
| **Data Transmitted** | Compressed video (H264/H265) | Metadata only (bodies, objects, poses) |
| **Bandwidth** | 10-40 Mbps | <100 Kbps |
| **Edge Compute** | Video encoding only | Full depth NN + tracking + detection |
| **Host Compute** | Full depth + tracking + detection | Lightweight fusion only |
| **API Methods** | `enableStreaming()` / `setFromStream()` | `startPublishing()` / `subscribe()` |

### Key Insight

**There is NO built-in mode for streaming computed depth maps or point clouds.** The architecture forces a choice:

1. **Streaming API**: Edge sends video → Host computes everything (depth, tracking, detection)
2. **Fusion API**: Edge computes everything → Sends only metadata (bodies/poses)

### Code Patterns

#### Streaming Sender (Edge)
```cpp
sl::StreamingParameters stream_params;
stream_params.codec = sl::STREAMING_CODEC::H265;
stream_params.port = 30000;
stream_params.bitrate = 12000;
zed.enableStreaming(stream_params);
```

#### Streaming Receiver (Host)
```cpp
sl::InitParameters init_params;
init_params.input.setFromStream("192.168.1.100", 30000);
zed.open(init_params);
// Full ZED SDK available - depth, tracking, etc.
```

#### Fusion Publisher (Edge or Host)
```cpp
sl::CommunicationParameters comm_params;
comm_params.setForLocalNetwork(30000);
// or comm_params.setForIntraProcess(); for same-machine
zed.startPublishing(comm_params);
```

#### Fusion Subscriber (Host)
```cpp
sl::Fusion fusion;
fusion.init(init_params);
sl::CameraIdentifier cam(serial_number);
fusion.subscribe(cam, comm_params, pose);
```

## Project: Multi-Camera Body Tracking

### Location
`/workspaces/zed-playground/playground/body tracking/multi-camera/cpp/`

### Architecture
- **ClientPublisher**: Receives camera streams, runs body tracking, publishes to Fusion
- **Fusion**: Subscribes to multiple ClientPublishers, fuses body data from all cameras
- **GLViewer**: 3D visualization of fused bodies

### Camera Configuration (Hard-coded)
From `inside_network.json`:

| Serial | IP | Streaming Port |
|--------|-----|----------------|
| 44289123 | 192.168.128.2 | 30000 |
| 44435674 | 192.168.128.2 | 30002 |
| 41831756 | 192.168.128.2 | 30004 |
| 46195029 | 192.168.128.2 | 30006 |

### Data Flow
```
Edge Camera (enableStreaming) → Network Stream
    ↓
ClientPublisher (setFromStream) → Body Tracking (host)
    ↓
startPublishing() → Fusion (INTRA_PROCESS)
    ↓
Fused Bodies → GLViewer
```

### Build
```bash
cd "/workspaces/zed-playground/playground/body tracking/multi-camera/cpp/build"
cmake ..
make -j4
```

### Run
```bash
./ZED_BodyFusion <config_file.json>
```

## Related Samples

### Camera Streaming Receiver
`/workspaces/zed-playground/playground/camera streaming/receiver/cpp/`
- Simple streaming receiver sample
- Shows basic `setFromStream()` usage with OpenCV display

## ZED SDK Headers
Located at: `/usr/local/zed/include/sl/`
- `Camera.hpp` - Main camera API
- `Fusion.hpp` - Fusion module API
- `CameraOne.hpp` - Single camera utilities