python workspace

2026-02-04 15:42:08 +08:00
parent e7c5176229
commit a0ad369211
37 changed files with 47631 additions and 0 deletions
@@ -0,0 +1,113 @@
+# Agent Context & Reminders
+
+## ZED SDK Architecture
+
+### Streaming API vs Fusion API
+
+The ZED SDK provides two distinct network APIs that are often confused:
+
+| Feature | Streaming API | Fusion API |
+|---------|---------------|------------|
+| **Data Transmitted** | Compressed video (H264/H265) | Metadata only (bodies, objects, poses) |
+| **Bandwidth** | 10-40 Mbps | <100 Kbps |
+| **Edge Compute** | Video encoding only | Full depth NN + tracking + detection |
+| **Host Compute** | Full depth + tracking + detection | Lightweight fusion only |
+| **API Methods** | `enableStreaming()` / `setFromStream()` | `startPublishing()` / `subscribe()` |
+
+### Key Insight
+
+**There is NO built-in mode for streaming computed depth maps or point clouds.** The architecture forces a choice:
+
+1. **Streaming API**: Edge sends video → Host computes everything (depth, tracking, detection)
+2. **Fusion API**: Edge computes everything → Sends only metadata (bodies/poses)
+
+### Code Patterns
+
+#### Streaming Sender (Edge)
+```cpp
+sl::StreamingParameters stream_params;
+stream_params.codec = sl::STREAMING_CODEC::H265;
+stream_params.port = 30000;
+stream_params.bitrate = 12000;
+zed.enableStreaming(stream_params);
+```
+
+#### Streaming Receiver (Host)
+```cpp
+sl::InitParameters init_params;
+init_params.input.setFromStream("192.168.1.100", 30000);
+zed.open(init_params);
+// Full ZED SDK available - depth, tracking, etc.
+```
+
+#### Fusion Publisher (Edge or Host)
+```cpp
+sl::CommunicationParameters comm_params;
+comm_params.setForLocalNetwork(30000);
+// or comm_params.setForIntraProcess(); for same-machine
+zed.startPublishing(comm_params);
+```
+
+#### Fusion Subscriber (Host)
+```cpp
+sl::Fusion fusion;
+fusion.init(init_params);
+sl::CameraIdentifier cam(serial_number);
+fusion.subscribe(cam, comm_params, pose);
+```
+
+## Project: Multi-Camera Body Tracking
+
+### Location
+`/workspaces/zed-playground/playground/body tracking/multi-camera/cpp/`
+
+### Architecture
+- **ClientPublisher**: Receives camera streams, runs body tracking, publishes to Fusion
+- **Fusion**: Subscribes to multiple ClientPublishers, fuses body data from all cameras
+- **GLViewer**: 3D visualization of fused bodies
+
+### Camera Configuration (Hard-coded)
+From `inside_network.json`:
+
+| Serial | IP | Streaming Port |
+|--------|-----|----------------|
+| 44289123 | 192.168.128.2 | 30000 |
+| 44435674 | 192.168.128.2 | 30002 |
+| 41831756 | 192.168.128.2 | 30004 |
+| 46195029 | 192.168.128.2 | 30006 |
+
+### Data Flow
+```
+Edge Camera (enableStreaming) → Network Stream
+    ↓
+ClientPublisher (setFromStream) → Body Tracking (host)
+    ↓
+startPublishing() → Fusion (INTRA_PROCESS)
+    ↓
+Fused Bodies → GLViewer
+```
+
+### Build
+```bash
+cd "/workspaces/zed-playground/playground/body tracking/multi-camera/cpp/build"
+cmake ..
+make -j4
+```
+
+### Run
+```bash
+./ZED_BodyFusion <config_file.json>
+```
+
+## Related Samples
+
+### Camera Streaming Receiver
+`/workspaces/zed-playground/playground/camera streaming/receiver/cpp/`
+- Simple streaming receiver sample
+- Shows basic `setFromStream()` usage with OpenCV display
+
+## ZED SDK Headers
+Located at: `/usr/local/zed/include/sl/`
+- `Camera.hpp` - Main camera API
+- `Fusion.hpp` - Fusion module API
+- `CameraOne.hpp` - Single camera utilities