cv-mmap Streamer

A standalone C++ downstream project that reads frames from cv-mmap IPC, encodes with NVIDIA NVENC (with software fallback), and publishes RTMP + RTP streams with low-latency tuning on localhost.

Overview

This project consumes video frames from the cv-mmap shared memory interface and publishes them as encoded streams. It operates as a downstream consumer only, never writing to the cv-mmap shared memory.

Key Features:

• Reads cv-mmap IPC frames via POSIX shared memory + ZeroMQ frame sync
• Consumes cv-mmap control/status/body over NATS
• NVENC H.264/H.265 encoding with deterministic software fallback
• RTP UDP-unicast publisher with automatic SDP generation
• RTMP publisher with dual H.265 modes (Enhanced-RTMP + domestic extension)
• Embedded standalone testers for server-independent validation
• Low-latency bounded queues with latest-frame semantics

Quickstart

Prerequisites

• C++23 compatible compiler (GCC 13+, Clang 16+)
• CMake 3.20+
• GStreamer 1.20+ with development headers
• ZeroMQ (cppzmq) with development headers
• NATS server reachable at runtime
• spdlog
• NVIDIA GPU with NVENC support (optional, falls back to software encoding)

Arch Linux:

sudo pacman -S cmake gstreamer gst-plugins-base gst-plugins-good \
    gst-plugins-bad gst-plugins-ugly gst-libav cppzmq spdlog

Build

cvmmap-streamer uses CVMMAP_CNATS_PROVIDER to decide how cnats is resolved:

• system (default): use an installed cnats package, typically from a top-level cv-mmap install under a standard prefix like /usr/local
• workspace: use the local cv-mmap build-tree exports

cmake -B build -S .
cmake --build build

# Use a local cv-mmap build tree
cmake -B build -S . \
  -DCVMMAP_CNATS_PROVIDER=workspace \
  -DCVMMAP_LOCAL_ROOT=/path/to/cv-mmap
cmake --build build

Verify binaries exist:

ls -la build/{cvmmap_streamer,rtp_receiver_tester,rtmp_stub_tester}

Offline ZED Tooling

Offline ZED conversion, batch wrappers, dataset indexing, and MCAP inspection helpers moved to the sibling repository ../zed-offline-tools.

Use that repo for:

• zed_svo_to_mcap
• zed_svo_to_mp4
• zed_svo_grid_to_mp4
• mcap_video_bounds
• scripts/zed_batch_*
• scripts/zed_segment_time_index.py
• scripts/generate_playlist_config.py
• scripts/mcap_bundle_validator.py
• scripts/mcap_rgbd_example.py
• scripts/mcap_rgbd_viewer.py
• scripts/mcap_depth_alignment.py

This repo keeps the live downstream streamer/runtime plus the MCAP contract docs such as docs/mcap_layout.md, docs/mcap_legacy_single_camera_layout.md, and docs/mcap_body_tracking.md.

Mandatory Acceptance (Standalone)

Run the full mandatory acceptance suite. This executes the complete protocol/codec matrix without requiring external servers.

./scripts/acceptance_standalone.sh

Expected result: Exit code 0 with summary showing total=5 pass=5 fail=0 skip=0

Individual matrix rows verified:

1. RTP + H.264
2. RTP + H.265
3. RTMP + H.264 (enhanced mode)
4. RTMP + H.265 enhanced mode
5. RTMP + H.265 domestic mode

Fault Suite Baseline

Run the fault injection and latency validation suite.

./scripts/fault_suite.sh

Expected result: Exit code 0 with all scenarios passing.

Scenarios tested:

• Torn read handling (coherent snapshot validation)
• Sink stall resilience (backpressure containment)
• Reset storm recovery (stream reset handling)

Manual Component Testing

1. Start the simulator:

./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --input-mode dummy \
    --dummy-label teststream \
    --dummy-frames 300 \
    --dummy-fps 30 \
    --dummy-width 640 \
    --dummy-height 360

2. Test RTP output:

# Terminal 1: Start receiver tester
./build/rtp_receiver_tester \
    --port 5004 \
    --expect-pt 96 \
    --packet-threshold 1 \
    --timeout-ms 10000

# Terminal 2: Start streamer
./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --rtp \
    --rtp-endpoint "127.0.0.1:5004" \
    --rtp-payload-type 96 \
    --rtp-sdp /tmp/test.sdp

3. Test RTMP output (enhanced mode):

# Terminal 1: Start RTMP stub tester
./build/rtmp_stub_tester \
    --mode h264 \
    --listen-host 127.0.0.1 \
    --listen-port 1935 \
    --video-threshold 1 \
    --timeout-ms 10000

# Terminal 2: Start streamer
./build/cvmmap_streamer \
    --run-mode pipeline \
    --codec h264 \
    --shm-name test_stream \
    --zmq-endpoint "ipc:///tmp/test_sync.ipc" \
    --rtmp \
    --rtmp-url "rtmp://127.0.0.1:1935/live/test" \
    --rtmp-mode enhanced

Compatibility Matrix

Protocol	Codec	RTMP Mode	Status	Notes
RTP	H.264	N/A	MANDATORY	Full support
RTP	H.265	N/A	MANDATORY	Full support
RTMP	H.264	enhanced	MANDATORY	Legacy codec-id 7
RTMP	H.265	enhanced	MANDATORY	FourCC hvc1, Enhanced-RTMP spec
RTMP	H.265	domestic	MANDATORY	FLV codec-id 12, legacy CDN compatibility
RTMP	H.264	domestic	INVALID	Rejected at startup with clear error

Legend:

• MANDATORY: Must pass for release acceptance
• INVALID: Explicitly rejected, exits non-zero

Runtime Configuration

Input Options

Flag	Description	Default
--shm-name NAME	POSIX shared memory segment name	required
--zmq-endpoint URI	ZeroMQ PUB endpoint for frame sync	required
--nats-url URL	NATS server for control/status/body	nats://localhost:4222
--queue-size N	Ingest queue capacity (1 = latest-frame)	1

Codec Options

Flag	Description
--codec h264|h265	Video codec selection (required)

Output Options

Flag	Description
--rtp	Enable RTP output
--rtp-endpoint HOST:PORT	RTP destination (required if --rtp)
--rtp-payload-type PT	Dynamic payload type [96,127]
--rtp-sdp PATH	SDP output path
--rtmp	Enable RTMP output
--rtmp-url URL	RTMP publish URL (required if --rtmp)
--rtmp-mode enhanced|domestic	H.265 packaging mode (required for H.265)

Latency Knobs

Flag	Description	Default
--gop N	GOP size (keyframe interval)	30
--b-frames N	B-frame count (0 = lowest latency)	0
--queue-size N	Ingest queue depth	1

Operational Limits

Flag	Description	Default
--ingest-max-frames N	Process at most N frames then exit	0 (unlimited)
--ingest-idle-timeout-ms MS	Exit if idle for MS milliseconds; 0 disables the timeout	0 (disabled)

Architecture

Data Flow

cv-mmap producer ──> SHM + ZMQ sync ──> Ingest Runtime
                                            │
                                            v
                                    ┌───────────────┐
                                    │ Bounded Queue │
                                    │ (size=1)      │
                                    └───────┬───────┘
                                            │
                                            v
                                    NVENC Pipeline
                                    (NVENC -> fallback)
                                            │
                                    ┌───────┴───────┐
                                    v               v
                              RTP Publisher    RTMP Publisher
                              (UDP unicast)    (TCP + FLV)

Key Design Decisions

Latest-Frame Semantics: The ingest queue has size 1 by default. When a new frame arrives while the previous is still queued, the old frame is dropped. This prevents latency accumulation under backpressure.

Coherent Snapshot: Frame metadata is read twice around the payload copy. If frame_count or timestamp_ns changed, the frame is rejected as torn. This prevents consuming partially-updated frames.

NVENC with Fallback: The pipeline attempts NVENC first for hardware acceleration. If NVENC produces zero encoded access units after 60 frames, it falls back to software encoding (x264enc or x265enc).

Dual-Mode H.265: H.265 RTMP supports two packaging modes:

• Enhanced-RTMP: Uses FourCC hvc1, modern standard, supported by FFmpeg 6.0+, SRS 6.0+, ZLMediaKit
• Domestic extension: Uses FLV codec-id 12, legacy Chinese CDN compatibility

The mode must be explicitly selected via --rtmp-mode and cannot be mixed within a session.

Environment Caveats

Simulator Label Length

Simulator labels (--label) have a hard maximum of 24 bytes. Exceeding this causes immediate exit with code 2. Use compact deterministic labels like acc_1_rtp_h264 instead of descriptive names.

Deterministic Simulator Sizing

For reliable RTMP validation, use simulator frame sizes of at least 640x360. Smaller frames may trigger GStreamer caps negotiation failures before the first encoded access unit on some hosts.

Build Path

Always use downstream/cvmmap-streamer/build for the build directory. Using the root build/ folder causes cache collision with the main cv-mmap project.

Fresh Configure

If you encounter configure errors referencing sibling repo paths, run:

cmake --fresh -B build -S .

Optional Server Smoke Tests

Interoperability tests with SRS and ZLMediaKit are provided for reference but are NOT mandatory for acceptance. See:

• SRS Smoke Test Profile
• ZLMediaKit Smoke Test Profile

If the server environment is unavailable, these tests should be skipped without failing the mandatory acceptance criteria.

Project Structure

cvmmap-streamer/
├── CMakeLists.txt          # Build configuration
├── README.md               # This file
├── docs/
│   ├── smoke/
│   │   ├── srs.md          # SRS interoperability guide
│   │   └── zlm.md          # ZLMediaKit interoperability guide
│   ├── compat_matrix.md    # Detailed compatibility matrix
│   └── caveats.md          # Environment and operational caveats
├── include/cvmmap_streamer/# Public headers
│   ├── config/
│   │   └── runtime_config.hpp
│   ├── ipc/
│   │   └── cvmmap_contract.hpp
│   └── pipeline/
│       └── pipeline_types.hpp
├── scripts/
│   ├── acceptance_standalone.sh  # Mandatory acceptance runner
│   ├── fault_suite.sh            # Fault injection suite
│   └── *_helper.py               # Summary generators
└── src/
    ├── config/             # Runtime configuration
    ├── core/               # Ingest runtime and supervision
    ├── ipc/                # cv-mmap contract parsing
    ├── pipeline/           # NVENC encoding
    ├── protocol/           # RTP and RTMP publishers
    └── testers/            # Simulator and test stubs

Evidence Artifacts

All test runs produce machine-readable evidence in .sisyphus/evidence/:

• task-14-acceptance.txt - Latest acceptance run metadata
• task-14-acceptance-summary.json - JSON summary of acceptance results
• task-15-fault-suite.txt - Latest fault suite run metadata
• task-15-fault-suite-summary.json - JSON summary of fault suite results

Each run creates timestamped subdirectories with full logs for every matrix row or fault scenario.

Exit Codes

Code	Meaning
0	Success
1	Invalid arguments
2	Invalid arguments or configuration
3	RTP payload type mismatch
4	Packet/frame threshold not met
5	Pipeline initialization error (missing encoder)
6	RTMP mode mismatch (tester validation)
7	Protocol validation error
124	Timeout

References

• Enhanced RTMP Specification
• cv-mmap IPC Contract
• SRS Documentation: https://ossrs.io/lts/en-us/docs/v7/doc/rtmp
• ZLMediaKit: https://github.com/ZLMediaKit/ZLMediaKit