A standalone C++ downstream project that reads frames from cv-mmap IPC, encodes with NVIDIA NVENC (with software fallback), and publishes RTMP + RTP streams with low-latency tuning on localhost.

Overview

This project consumes video frames from the cv-mmap shared memory interface and publishes them as encoded streams. It operates as a downstream consumer only, never writing to the cv-mmap shared memory.

Key Features:

Reads cv-mmap IPC frames via POSIX shared memory + ZeroMQ frame sync
Consumes cv-mmap control/status/body over NATS
NVENC H.264/H.265 encoding with deterministic software fallback
RTP UDP-unicast publisher with automatic SDP generation
RTMP publisher with dual H.265 modes (Enhanced-RTMP + domestic extension)
Embedded standalone testers for server-independent validation
Low-latency bounded queues with latest-frame semantics

Quickstart

Prerequisites

C++23 compatible compiler (GCC 13+, Clang 16+)
CMake 3.20+
GStreamer 1.20+ with development headers
ZeroMQ (cppzmq) with development headers
NATS server reachable at runtime
spdlog
NVIDIA GPU with NVENC support (optional, falls back to software encoding)

Arch Linux:

sudo pacman -S cmake gstreamer gst-plugins-base gst-plugins-good \
    gst-plugins-bad gst-plugins-ugly gst-libav cppzmq spdlog

Build

cvmmap-streamer uses CVMMAP_CNATS_PROVIDER to decide how cnats is resolved:

system (default): use an installed cnats package, typically from a top-level cv-mmap install under a standard prefix like /usr/local
workspace: use the local cv-mmap build-tree exports

cmake -B build -S .
cmake --build build

# Use a local cv-mmap build tree
cmake -B build -S . \
  -DCVMMAP_CNATS_PROVIDER=workspace \
  -DCVMMAP_LOCAL_ROOT=/path/to/cv-mmap
cmake --build build

Verify binaries exist:

ls -la build/{cvmmap_streamer,rtp_receiver_tester,rtmp_stub_tester}

ZED SVO/SVO2 To MP4

The repo also includes an offline conversion tool for the left ZED color stream:

CUDA_VISIBLE_DEVICES=GPU-9cc7b26e-90d4-0c49-4d4c-060e528ffba6 \
./build/bin/zed_svo_to_mp4 \
    --input /workspaces/data/kindergarten/bar/2026-03-18T11-59-41/2026-03-18T11-59-41_zed1.svo2 \
    --encoder-device auto \
    --preset balanced \
    --quality 20 \
    --start-frame 0 \
    --end-frame 89

By default the tool writes foo.mp4 next to foo.svo or foo.svo2, defaults to h265, and shows a tqdm-like progress bar when stderr is attached to a TTY. --encoder-device auto tries NVENC first and falls back to software (libx264 or libx265) if the hardware encoder is unavailable or cannot be opened.

Batch ZED SVO2 To MP4

Python dependencies for the batch wrapper are managed with uv:

uv sync

Use the wrapper to recurse through a folder, run zed_svo_to_mp4 on every matched .svo2, and show one aggregate tqdm progress bar:

uv run python scripts/zed_batch_svo_to_mp4.py \
    /workspaces/data/kindergarten/bar \
    --pattern '*.svo2' \
    --recursive \
    --jobs 2 \
    --encoder-device auto \
    --start-frame 0 \
    --end-frame 29 \
    --cuda-visible-devices GPU-9cc7b26e-90d4-0c49-4d4c-060e528ffba6

The batch tool mirrors the common encoder options from zed_svo_to_mp4, skips existing sibling .mp4 outputs by default, and continues after failures while returning a nonzero exit code if any conversion fails.

ZED SVO Grid To MP4

Use the grid converter to merge four synced ZED recordings into a 2x2 CCTV-style MP4 with a Unix timestamp overlay in the top-left corner:

./build/bin/zed_svo_grid_to_mp4 \
    --segment-dir /workspaces/data/kindergarten/bar/2026-03-18T11-59-41 \
    --encoder-device auto \
    --codec h265 \
    --duration-seconds 2

The tool syncs the four inputs using the same common-start timestamp rule as the ZED multi-camera playback sample, defaults to a 2x2 layout ordered as zed1 zed2 / zed3 zed4, and writes <segment>/<segment>_grid.mp4 unless --output is provided. By default each tile is scaled to 0.5x, so a four-camera 1920x1200 segment produces a 1920x1200 composite. Use repeated --input flags instead of --segment-dir when you want explicit row-major ordering.

Mandatory Acceptance (Standalone)

Run the full mandatory acceptance suite. This executes the complete protocol/codec matrix without requiring external servers.

./scripts/acceptance_standalone.sh

Expected result: Exit code 0 with summary showing total=5 pass=5 fail=0 skip=0

Individual matrix rows verified:

RTP + H.264
RTP + H.265
RTMP + H.264 (enhanced mode)
RTMP + H.265 enhanced mode
RTMP + H.265 domestic mode

Fault Suite Baseline

Run the fault injection and latency validation suite.

./scripts/fault_suite.sh

Expected result: Exit code 0 with all scenarios passing.

Scenarios tested:

Torn read handling (coherent snapshot validation)
Sink stall resilience (backpressure containment)
Reset storm recovery (stream reset handling)

Manual Component Testing

1. Start the simulator:

./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --input-mode dummy \
    --dummy-label teststream \
    --dummy-frames 300 \
    --dummy-fps 30 \
    --dummy-width 640 \
    --dummy-height 360

2. Test RTP output:

# Terminal 1: Start receiver tester
./build/rtp_receiver_tester \
    --port 5004 \
    --expect-pt 96 \
    --packet-threshold 1 \
    --timeout-ms 10000

# Terminal 2: Start streamer
./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --rtp \
    --rtp-endpoint "127.0.0.1:5004" \
    --rtp-payload-type 96 \
    --rtp-sdp /tmp/test.sdp

3. Test RTMP output (enhanced mode):

# Terminal 1: Start RTMP stub tester
./build/rtmp_stub_tester \
    --mode h264 \
    --listen-host 127.0.0.1 \
    --listen-port 1935 \
    --video-threshold 1 \
    --timeout-ms 10000

# Terminal 2: Start streamer
./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --rtmp \
    --rtmp-url "rtmp://127.0.0.1:1935/live/test" \
    --rtmp-mode enhanced

Compatibility Matrix

Protocol	Codec	RTMP Mode	Status	Notes
RTP	H.264	N/A	MANDATORY	Full support
RTP	H.265	N/A	MANDATORY	Full support
RTMP	H.264	enhanced	MANDATORY	Legacy codec-id 7
RTMP	H.265	enhanced	MANDATORY	FourCC `hvc1`, Enhanced-RTMP spec
RTMP	H.265	domestic	MANDATORY	FLV codec-id 12, legacy CDN compatibility
RTMP	H.264	domestic	INVALID	Rejected at startup with clear error

Legend:

MANDATORY: Must pass for release acceptance
INVALID: Explicitly rejected, exits non-zero

Runtime Configuration

Input Options

Flag	Description	Default
`--shm-name NAME`	POSIX shared memory segment name	required
`--zmq-endpoint URI`	ZeroMQ PUB endpoint for frame sync	required
`--nats-url URL`	NATS server for control/status/body	`nats://localhost:4222`
`--queue-size N`	Ingest queue capacity (1 = latest-frame)	1

Codec Options

Flag	Description
`--codec h264\|h265`	Video codec selection (required)

Output Options

Flag	Description
`--rtp`	Enable RTP output
`--rtp-endpoint HOST:PORT`	RTP destination (required if --rtp)
`--rtp-payload-type PT`	Dynamic payload type [96,127]
`--rtp-sdp PATH`	SDP output path
`--rtmp`	Enable RTMP output
`--rtmp-url URL`	RTMP publish URL (required if --rtmp)
`--rtmp-mode enhanced\|domestic`	H.265 packaging mode (required for H.265)

Latency Knobs

Flag	Description	Default
`--gop N`	GOP size (keyframe interval)	30
`--b-frames N`	B-frame count (0 = lowest latency)	0
`--queue-size N`	Ingest queue depth	1

Operational Limits

Flag	Description	Default
`--ingest-max-frames N`	Process at most N frames then exit	0 (unlimited)
`--ingest-idle-timeout-ms MS`	Exit if idle for MS milliseconds	0 (disabled)

Architecture

Data Flow

cv-mmap producer ──> SHM + ZMQ sync ──> Ingest Runtime
                                            │
                                            v
                                    ┌───────────────┐
                                    │ Bounded Queue │
                                    │ (size=1)      │
                                    └───────┬───────┘
                                            │
                                            v
                                    NVENC Pipeline
                                    (NVENC -> fallback)
                                            │
                                    ┌───────┴───────┐
                                    v               v
                              RTP Publisher    RTMP Publisher
                              (UDP unicast)    (TCP + FLV)

Key Design Decisions

Latest-Frame Semantics: The ingest queue has size 1 by default. When a new frame arrives while the previous is still queued, the old frame is dropped. This prevents latency accumulation under backpressure.

Coherent Snapshot: Frame metadata is read twice around the payload copy. If frame_count or timestamp_ns changed, the frame is rejected as torn. This prevents consuming partially-updated frames.

NVENC with Fallback: The pipeline attempts NVENC first for hardware acceleration. If NVENC produces zero encoded access units after 60 frames, it falls back to software encoding (x264enc or x265enc).

Dual-Mode H.265: H.265 RTMP supports two packaging modes:

Enhanced-RTMP: Uses FourCC hvc1, modern standard, supported by FFmpeg 6.0+, SRS 6.0+, ZLMediaKit
Domestic extension: Uses FLV codec-id 12, legacy Chinese CDN compatibility

The mode must be explicitly selected via --rtmp-mode and cannot be mixed within a session.

Environment Caveats

Simulator Label Length

Simulator labels (--label) have a hard maximum of 24 bytes. Exceeding this causes immediate exit with code 2. Use compact deterministic labels like acc_1_rtp_h264 instead of descriptive names.

Deterministic Simulator Sizing

For reliable RTMP validation, use simulator frame sizes of at least 640x360. Smaller frames may trigger GStreamer caps negotiation failures before the first encoded access unit on some hosts.

Build Path

Always use downstream/cvmmap-streamer/build for the build directory. Using the root build/ folder causes cache collision with the main cv-mmap project.

Fresh Configure

If you encounter configure errors referencing sibling repo paths, run:

cmake --fresh -B build -S .

Optional Server Smoke Tests

Interoperability tests with SRS and ZLMediaKit are provided for reference but are NOT mandatory for acceptance. See:

If the server environment is unavailable, these tests should be skipped without failing the mandatory acceptance criteria.

Project Structure

cvmmap-streamer/
├── CMakeLists.txt          # Build configuration
├── README.md               # This file
├── docs/
│   ├── smoke/
│   │   ├── srs.md          # SRS interoperability guide
│   │   └── zlm.md          # ZLMediaKit interoperability guide
│   ├── compat_matrix.md    # Detailed compatibility matrix
│   └── caveats.md          # Environment and operational caveats
├── include/cvmmap_streamer/# Public headers
│   ├── config/
│   │   └── runtime_config.hpp
│   ├── ipc/
│   │   └── cvmmap_contract.hpp
│   └── pipeline/
│       └── pipeline_types.hpp
├── scripts/
│   ├── acceptance_standalone.sh  # Mandatory acceptance runner
│   ├── fault_suite.sh            # Fault injection suite
│   └── *_helper.py               # Summary generators
└── src/
    ├── config/             # Runtime configuration
    ├── core/               # Ingest runtime and supervision
    ├── ipc/                # cv-mmap contract parsing
    ├── pipeline/           # NVENC encoding
    ├── protocol/           # RTP and RTMP publishers
    └── testers/            # Simulator and test stubs

Evidence Artifacts

All test runs produce machine-readable evidence in .sisyphus/evidence/:

task-14-acceptance.txt - Latest acceptance run metadata
task-14-acceptance-summary.json - JSON summary of acceptance results
task-15-fault-suite.txt - Latest fault suite run metadata
task-15-fault-suite-summary.json - JSON summary of fault suite results

Each run creates timestamped subdirectories with full logs for every matrix row or fault scenario.

Exit Codes

Code	Meaning
0	Success
1	Invalid arguments
2	Invalid arguments or configuration
3	RTP payload type mismatch
4	Packet/frame threshold not met
5	Pipeline initialization error (missing encoder)
6	RTMP mode mismatch (tester validation)
7	Protocol validation error
124	Timeout

References

Enhanced RTMP Specification
cv-mmap IPC Contract
SRS Documentation: https://ossrs.io/lts/en-us/docs/v7/doc/rtmp
ZLMediaKit: https://github.com/ZLMediaKit/ZLMediaKit