cvmmap-streamer/README.md

# cv-mmap Streamer

A standalone C++ downstream project that reads frames from cv-mmap IPC, encodes with NVIDIA NVENC (with software fallback), and publishes RTMP + RTP streams with low-latency tuning on localhost.

## Overview

This project consumes video frames from the cv-mmap shared memory interface and publishes them as encoded streams. It operates as a downstream consumer only, never writing to the cv-mmap shared memory.

**Key Features:**
- Reads cv-mmap IPC frames via POSIX shared memory + ZeroMQ frame sync
- Consumes cv-mmap control/status/body over NATS
- NVENC H.264/H.265 encoding with deterministic software fallback
- RTP UDP-unicast publisher with automatic SDP generation
- RTMP publisher with dual H.265 modes (Enhanced-RTMP + domestic extension)
- Embedded standalone testers for server-independent validation
- Low-latency bounded queues with latest-frame semantics

## Quickstart

### Prerequisites

- C++23 compatible compiler (GCC 13+, Clang 16+)
- CMake 3.20+
- GStreamer 1.20+ with development headers
- ZeroMQ (cppzmq) with development headers
- NATS server reachable at runtime
- spdlog
- NVIDIA GPU with NVENC support (optional, falls back to software encoding)

**Arch Linux:**
```bash
sudo pacman -S cmake gstreamer gst-plugins-base gst-plugins-good \
    gst-plugins-bad gst-plugins-ugly gst-libav cppzmq spdlog
```

### Build

`cvmmap-streamer` uses `CVMMAP_CNATS_PROVIDER` to decide how `cnats` is resolved:

- `system` (default): use an installed `cnats` package, typically from a top-level `cv-mmap` install under a standard prefix like `/usr/local`
- `workspace`: use the local `cv-mmap` build-tree exports

```bash
cmake -B build -S .
cmake --build build
```

```bash
# Use a local cv-mmap build tree
cmake -B build -S . \
  -DCVMMAP_CNATS_PROVIDER=workspace \
  -DCVMMAP_LOCAL_ROOT=/path/to/cv-mmap
cmake --build build
```

**Verify binaries exist:**
```bash
ls -la build/{cvmmap_streamer,rtp_receiver_tester,rtmp_stub_tester}
```

### ZED SVO/SVO2 To MP4

The repo also includes an offline conversion tool for the left ZED color stream:

```bash
CUDA_VISIBLE_DEVICES=GPU-9cc7b26e-90d4-0c49-4d4c-060e528ffba6 \
./build/bin/zed_svo_to_mp4 \
    --input /workspaces/data/kindergarten/bar/2026-03-18T11-59-41/2026-03-18T11-59-41_zed1.svo2 \
    --encoder-device auto \
    --preset balanced \
    --quality 20 \
    --start-frame 0 \
    --end-frame 89
```

By default the tool writes `foo.mp4` next to `foo.svo` or `foo.svo2`, defaults to `h265`, and shows a tqdm-like progress bar when stderr is attached to a TTY. `--encoder-device auto` tries NVENC first and falls back to software (`libx264` or `libx265`) if the hardware encoder is unavailable or cannot be opened.

### Batch ZED SVO2 To MP4

Python dependencies for the batch wrapper are managed with `uv`:

```bash
uv sync
```

Use the wrapper to recurse through a folder, run `zed_svo_to_mp4` on every matched `.svo2`, and show one aggregate tqdm progress bar:

```bash
uv run python scripts/zed_batch_svo_to_mp4.py \
    /workspaces/data/kindergarten/bar \
    --pattern '*.svo2' \
    --recursive \
    --jobs 2 \
    --encoder-device auto \
    --start-frame 0 \
    --end-frame 29 \
    --cuda-visible-devices GPU-9cc7b26e-90d4-0c49-4d4c-060e528ffba6
```

The batch tool mirrors the common encoder options from `zed_svo_to_mp4`, skips existing sibling `.mp4` outputs by default, and continues after failures while returning a nonzero exit code if any conversion fails.

### ZED SVO Grid To MP4

Use the grid converter to merge four synced ZED recordings into a 2x2 CCTV-style MP4 with a Unix timestamp overlay in the top-left corner:

```bash
./build/bin/zed_svo_grid_to_mp4 \
    --segment-dir /workspaces/data/kindergarten/bar/2026-03-18T11-59-41 \
    --encoder-device auto \
    --codec h265 \
    --duration-seconds 2
```

The tool syncs the four inputs using the same common-start timestamp rule as the ZED multi-camera playback sample, defaults to a 2x2 layout ordered as `zed1 zed2 / zed3 zed4`, and writes `<segment>/<segment>_grid.mp4` unless `--output` is provided. By default each tile is scaled to `0.5x`, so a four-camera 1920x1200 segment produces a 1920x1200 composite. Use repeated `--input` flags instead of `--segment-dir` when you want explicit row-major ordering.

Use the batch wrapper to run `zed_svo_grid_to_mp4` over many segment directories with one aggregate progress bar:

```bash
uv run python scripts/zed_batch_svo_grid_to_mp4.py \
    /workspaces/data/kindergarten \
    --recursive \
    --jobs 2 \
    --encoder-device auto \
    --duration-seconds 2
```

You can also provide the exact segments to convert:

```bash
uv run python scripts/zed_batch_svo_grid_to_mp4.py \
    --segment-dir /workspaces/data/kindergarten/jump/external/recording/2026-03-18T11-23-22 \
    --segment-dir /workspaces/data/kindergarten/jump/experiment/1/2026-03-18T11-26-23 \
    --jobs 2
```

Or preserve a precomputed CSV ordering:

```bash
uv run python scripts/zed_batch_svo_grid_to_mp4.py \
    --segments-csv /workspaces/data/kindergarten/svo2_segments_sorted.csv \
    --jobs 2 \
    --duration-seconds 2
```

The batch grid wrapper mirrors the grid encoder options, skips existing `<segment>/<segment>_grid.mp4` outputs by default, and returns a nonzero exit code if any segment fails.

#### Expected CSV Input Format

The `--segments-csv` input expects a header row with at least a `segment_dir` column. Extra columns are allowed and ignored by the batch wrapper. `segment_dir` values may be absolute paths or paths relative to the CSV file's parent directory. Use `--csv-root` to override that base directory.

Repeated rows for the same `segment_dir` are allowed; the wrapper converts each unique segment once, preserving the first-seen CSV order. This makes `/workspaces/data/kindergarten/svo2_segments_sorted.csv` a valid input even though it stores one row per camera file:

```csv
timestamp,activity,group_path,segment_dir,camera,relative_path
2026-03-18T11-23-22,jump,jump/external/recording,jump/external/recording/2026-03-18T11-23-22,zed1,jump/external/recording/2026-03-18T11-23-22/2026-03-18T11-23-22_zed1.svo2
2026-03-18T11-23-22,jump,jump/external/recording,jump/external/recording/2026-03-18T11-23-22,zed2,jump/external/recording/2026-03-18T11-23-22/2026-03-18T11-23-22_zed2.svo2
```

### Mandatory Acceptance (Standalone)

Run the full mandatory acceptance suite. This executes the complete protocol/codec matrix without requiring external servers.

```bash
./scripts/acceptance_standalone.sh
```

**Expected result:** Exit code 0 with summary showing `total=5 pass=5 fail=0 skip=0`

**Individual matrix rows verified:**
1. RTP + H.264
2. RTP + H.265
3. RTMP + H.264 (enhanced mode)
4. RTMP + H.265 enhanced mode
5. RTMP + H.265 domestic mode

### Fault Suite Baseline

Run the fault injection and latency validation suite.

```bash
./scripts/fault_suite.sh
```

**Expected result:** Exit code 0 with all scenarios passing.

**Scenarios tested:**
- Torn read handling (coherent snapshot validation)
- Sink stall resilience (backpressure containment)
- Reset storm recovery (stream reset handling)

### Manual Component Testing

**1. Start the simulator:**
```bash
./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --input-mode dummy \
    --dummy-label teststream \
    --dummy-frames 300 \
    --dummy-fps 30 \
    --dummy-width 640 \
    --dummy-height 360
```

**2. Test RTP output:**
```bash
# Terminal 1: Start receiver tester
./build/rtp_receiver_tester \
    --port 5004 \
    --expect-pt 96 \
    --packet-threshold 1 \
    --timeout-ms 10000

# Terminal 2: Start streamer
./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --rtp \
    --rtp-endpoint "127.0.0.1:5004" \
    --rtp-payload-type 96 \
    --rtp-sdp /tmp/test.sdp
```

**3. Test RTMP output (enhanced mode):**
```bash
# Terminal 1: Start RTMP stub tester
./build/rtmp_stub_tester \
    --mode h264 \
    --listen-host 127.0.0.1 \
    --listen-port 1935 \
    --video-threshold 1 \
    --timeout-ms 10000

# Terminal 2: Start streamer
./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --rtmp \
    --rtmp-url "rtmp://127.0.0.1:1935/live/test" \
    --rtmp-mode enhanced
```

## Compatibility Matrix

| Protocol | Codec | RTMP Mode | Status | Notes |
|----------|-------|-----------|--------|-------|
| RTP | H.264 | N/A | MANDATORY | Full support |
| RTP | H.265 | N/A | MANDATORY | Full support |
| RTMP | H.264 | enhanced | MANDATORY | Legacy codec-id 7 |
| RTMP | H.265 | enhanced | MANDATORY | FourCC `hvc1`, [Enhanced-RTMP spec](https://github.com/veovera/enhanced-rtmp) |
| RTMP | H.265 | domestic | MANDATORY | FLV codec-id 12, legacy CDN compatibility |
| RTMP | H.264 | domestic | INVALID | Rejected at startup with clear error |

**Legend:**
- **MANDATORY**: Must pass for release acceptance
- **INVALID**: Explicitly rejected, exits non-zero

## Runtime Configuration

### Input Options

| Flag | Description | Default |
|------|-------------|---------|
| `--shm-name NAME` | POSIX shared memory segment name | required |
| `--zmq-endpoint URI` | ZeroMQ PUB endpoint for frame sync | required |
| `--nats-url URL` | NATS server for control/status/body | `nats://localhost:4222` |
| `--queue-size N` | Ingest queue capacity (1 = latest-frame) | 1 |

### Codec Options

| Flag | Description |
|------|-------------|
| `--codec h264\|h265` | Video codec selection (required) |

### Output Options

| Flag | Description |
|------|-------------|
| `--rtp` | Enable RTP output |
| `--rtp-endpoint HOST:PORT` | RTP destination (required if --rtp) |
| `--rtp-payload-type PT` | Dynamic payload type [96,127] | 96 |
| `--rtp-sdp PATH` | SDP output path |
| `--rtmp` | Enable RTMP output |
| `--rtmp-url URL` | RTMP publish URL (required if --rtmp) |
| `--rtmp-mode enhanced\|domestic` | H.265 packaging mode (required for H.265) |

### Latency Knobs

| Flag | Description | Default |
|------|-------------|---------|
| `--gop N` | GOP size (keyframe interval) | 30 |
| `--b-frames N` | B-frame count (0 = lowest latency) | 0 |
| `--queue-size N` | Ingest queue depth | 1 |

### Operational Limits

| Flag | Description | Default |
|------|-------------|---------|
| `--ingest-max-frames N` | Process at most N frames then exit | 0 (unlimited) |
| `--ingest-idle-timeout-ms MS` | Exit if idle for MS milliseconds | 0 (disabled) |

## Architecture

### Data Flow

```
cv-mmap producer ──> SHM + ZMQ sync ──> Ingest Runtime
                                            │
                                            v
                                    ┌───────────────┐
                                    │ Bounded Queue │
                                    │ (size=1)      │
                                    └───────┬───────┘
                                            │
                                            v
                                    NVENC Pipeline
                                    (NVENC -> fallback)
                                            │
                                    ┌───────┴───────┐
                                    v               v
                              RTP Publisher    RTMP Publisher
                              (UDP unicast)    (TCP + FLV)
```

### Key Design Decisions

**Latest-Frame Semantics:** The ingest queue has size 1 by default. When a new frame arrives while the previous is still queued, the old frame is dropped. This prevents latency accumulation under backpressure.

**Coherent Snapshot:** Frame metadata is read twice around the payload copy. If `frame_count` or `timestamp_ns` changed, the frame is rejected as torn. This prevents consuming partially-updated frames.

**NVENC with Fallback:** The pipeline attempts NVENC first for hardware acceleration. If NVENC produces zero encoded access units after 60 frames, it falls back to software encoding (`x264enc` or `x265enc`).

**Dual-Mode H.265:** H.265 RTMP supports two packaging modes:
- **Enhanced-RTMP**: Uses FourCC `hvc1`, modern standard, supported by FFmpeg 6.0+, SRS 6.0+, ZLMediaKit
- **Domestic extension**: Uses FLV codec-id 12, legacy Chinese CDN compatibility

The mode must be explicitly selected via `--rtmp-mode` and cannot be mixed within a session.

## Environment Caveats

### Simulator Label Length
Simulator labels (`--label`) have a hard maximum of 24 bytes. Exceeding this causes immediate exit with code 2. Use compact deterministic labels like `acc_1_rtp_h264` instead of descriptive names.

### Deterministic Simulator Sizing
For reliable RTMP validation, use simulator frame sizes of at least 640x360. Smaller frames may trigger GStreamer caps negotiation failures before the first encoded access unit on some hosts.

### Build Path
Always use `downstream/cvmmap-streamer/build` for the build directory. Using the root `build/` folder causes cache collision with the main cv-mmap project.

### Fresh Configure
If you encounter configure errors referencing sibling repo paths, run:
```bash
cmake --fresh -B build -S .
```

## Optional Server Smoke Tests

Interoperability tests with SRS and ZLMediaKit are provided for reference but are **NOT** mandatory for acceptance. See:

- [SRS Smoke Test Profile](docs/smoke/srs.md)
- [ZLMediaKit Smoke Test Profile](docs/smoke/zlm.md)

If the server environment is unavailable, these tests should be skipped without failing the mandatory acceptance criteria.

## Project Structure

```
cvmmap-streamer/
├── CMakeLists.txt          # Build configuration
├── README.md               # This file
├── docs/
│   ├── smoke/
│   │   ├── srs.md          # SRS interoperability guide
│   │   └── zlm.md          # ZLMediaKit interoperability guide
│   ├── compat_matrix.md    # Detailed compatibility matrix
│   └── caveats.md          # Environment and operational caveats
├── include/cvmmap_streamer/# Public headers
│   ├── config/
│   │   └── runtime_config.hpp
│   ├── ipc/
│   │   └── cvmmap_contract.hpp
│   └── pipeline/
│       └── pipeline_types.hpp
├── scripts/
│   ├── acceptance_standalone.sh  # Mandatory acceptance runner
│   ├── fault_suite.sh            # Fault injection suite
│   └── *_helper.py               # Summary generators
└── src/
    ├── config/             # Runtime configuration
    ├── core/               # Ingest runtime and supervision
    ├── ipc/                # cv-mmap contract parsing
    ├── pipeline/           # NVENC encoding
    ├── protocol/           # RTP and RTMP publishers
    └── testers/            # Simulator and test stubs
```

## Evidence Artifacts

All test runs produce machine-readable evidence in `.sisyphus/evidence/`:

- `task-14-acceptance.txt` - Latest acceptance run metadata
- `task-14-acceptance-summary.json` - JSON summary of acceptance results
- `task-15-fault-suite.txt` - Latest fault suite run metadata
- `task-15-fault-suite-summary.json` - JSON summary of fault suite results

Each run creates timestamped subdirectories with full logs for every matrix row or fault scenario.

## Exit Codes

| Code | Meaning |
|------|---------|
| 0 | Success |
| 1 | Invalid arguments |
| 2 | Invalid arguments or configuration |
| 3 | RTP payload type mismatch |
| 4 | Packet/frame threshold not met |
| 5 | Pipeline initialization error (missing encoder) |
| 6 | RTMP mode mismatch (tester validation) |
| 7 | Protocol validation error |
| 124 | Timeout |

## References

- [Enhanced RTMP Specification](https://github.com/veovera/enhanced-rtmp)
- [cv-mmap IPC Contract](https://github.com/k2wanko/cv-mmap/blob/main/docs/cvmmap.ksy)
- SRS Documentation: https://ossrs.io/lts/en-us/docs/v7/doc/rtmp
- ZLMediaKit: https://github.com/ZLMediaKit/ZLMediaKit