crosstyan
51d03d4279
Integrate a native Jetson Multimedia API encoder path and keep the existing encoded access-unit contract for RTMP, RTP, and recording consumers. This adds conditional Jetson MMAPI detection in CMake, builds the required NVIDIA sample common classes into a dedicated support library, and compiles the new backend only when the platform dependencies are present. The runtime selector now lets encoder.backend=auto prefer Jetson MM for NVIDIA hardware requests while keeping encoder.backend=ffmpeg as an explicit FFmpeg path. Device selection semantics are updated so nvidia requires the Jetson backend, auto can fall back to FFmpeg software, and software remains FFmpeg-only. The Jetson backend converts supported raw inputs through swscale, feeds NvVideoEncoder in YUV420M, emits Annex-B access units, and harvests decoder configuration from a warmup keyframe so downstream packetizers keep their existing contract. This also splits FFmpeg encoder option handling into a shared header, updates runtime config/help text and tester defaults, and refreshes compatibility/caveat documentation to reflect the new selection behavior. Build-tree runtime RPATH handling is tightened so GCC 15 builds keep the matching libstdc++ visible locally. Verification covered GCC 15 builds, RTP H.264/H.265 tester runs, RTMP H.264 stub publish, local live SRS smoke with cvmmap://zed1, and remote execution on 192.168.2.155 using the deployed bundle lib directory for the GCC 15 runtime.
108 lines
3.8 KiB
Markdown
108 lines
3.8 KiB
Markdown
# Operational Caveats
|
|
|
|
This document captures the current runtime constraints for `cvmmap-streamer` after the FFmpeg-only cleanup.
|
|
|
|
## Input Model
|
|
|
|
### cvmmap URI Only
|
|
|
|
The runtime ingests from a cvmmap URI, for example:
|
|
|
|
```bash
|
|
./build/cvmmap_streamer --run-mode pipeline --input-uri 'cvmmap://zed@/tmp/cvmmap'
|
|
```
|
|
|
|
Legacy flags such as `--shm-name`, `--zmq-endpoint`, `--input-mode`, and the dummy-input options are no longer supported.
|
|
|
|
## Encoder Path
|
|
|
|
### Jetson Hardware Encode Uses The Jetson Multimedia API Backend
|
|
|
|
The public backend surface is still:
|
|
|
|
- `--encoder-backend auto`
|
|
- `--encoder-backend ffmpeg`
|
|
|
|
`--encoder-backend ffmpeg` keeps the existing FFmpeg encoder path.
|
|
`--encoder-backend auto` may now select the in-repo Jetson Multimedia API backend on Jetson builds when hardware encoding is requested.
|
|
|
|
The removed GStreamer backend is still unavailable.
|
|
|
|
### NVIDIA Device Mode On Jetson No Longer Depends On FFmpeg Hardware Encoders
|
|
|
|
On Jetson builds with `CVMMAP_STREAMER_ENABLE_JETSON_MM` enabled, `--encoder-device nvidia` uses the Jetson Multimedia API encoder directly rather than desktop `*_nvenc` or Jetson FFmpeg `*_v4l2m2m` encoder exposure.
|
|
|
|
Selection behavior is now:
|
|
|
|
- `--encoder-device software` -> FFmpeg software encode (`libx264`/`libx265`)
|
|
- `--encoder-device nvidia` -> Jetson Multimedia API backend only; fail if unavailable
|
|
- `--encoder-device auto` -> try Jetson Multimedia API first on Jetson builds, then fall back to FFmpeg software
|
|
|
|
Useful local checks on Jetson:
|
|
|
|
```bash
|
|
test -e /dev/v4l2-nvenc
|
|
gst-inspect-1.0 nvv4l2h264enc nvv4l2h265enc
|
|
```
|
|
|
|
FFmpeg encoder enumeration is no longer the authoritative Jetson hardware-encode check for this repo.
|
|
|
|
### Low-Latency Defaults
|
|
|
|
The current low-latency defaults are:
|
|
|
|
- `gop=30`
|
|
- `b_frames=0`
|
|
- encoder-family-specific low-latency options where FFmpeg exposes them
|
|
|
|
This favors immediacy over compression efficiency.
|
|
|
|
## Transport Caveats
|
|
|
|
### RTMP Is Enhanced RTMP Only
|
|
|
|
The repo now supports:
|
|
|
|
- `libavformat` RTMP output
|
|
- `ffmpeg_process` RTMP output
|
|
|
|
The removed custom RTMP packetizer and `domestic` mode are no longer available. RTMP is always Enhanced RTMP.
|
|
|
|
### RTP SDP Is Optional
|
|
|
|
The RTP publisher can write an SDP sidecar when `--rtp-sdp` is set. This is only for generic receivers that need an out-of-band description. If the consumer is preconfigured, skip the SDP output.
|
|
|
|
### No Direct RTSP/WebRTC Publishing
|
|
|
|
This repo does not contain a direct RTSP publisher or a direct WebRTC/WHEP publisher.
|
|
|
|
If you want browser playback or WHEP, publish RTMP to a media server such as SRS and let that server expose WHEP/WebRTC.
|
|
|
|
## Media Scope
|
|
|
|
### No Audio Support
|
|
|
|
This project is video-only. It publishes or records encoded video access units only.
|
|
|
|
If you need audio transport, muxing, or A/V sync, that is outside the current repo scope.
|
|
|
|
## Recording Caveats
|
|
|
|
### MCAP Stores Encoded Video Frames
|
|
|
|
MCAP recording stores one `foxglove.CompressedVideo` message per encoded access unit. Replay depends on encoded keyframes carrying decoder configuration; this is handled by the current writer on keyframes.
|
|
|
|
### MCAP Body Recording Is Raw Packet Capture
|
|
|
|
When the producer exposes ZED body tracking, MCAP recording stores one raw `cvmmap.body_tracking.v1` message per body PUB packet on `/camera/body` by default. These payload bytes are not normalized by `cvmmap-streamer`; downstream consumers should parse them with the cv-mmap body-tracking v1 contract.
|
|
|
|
## External Server Caveats
|
|
|
|
### Local SRS Defaults Can Hit `ulimit`
|
|
|
|
On this machine, stock SRS configs can fail at startup if `max_connections` exceeds the local `ulimit -n`. The checked-in smoke profile uses `max_connections 64` to avoid that local failure mode.
|
|
|
|
### Server Latency Still Matters
|
|
|
|
Low encoder latency does not guarantee low end-to-end latency. SRS, ZLMediaKit, HTTP-FLV players, and browser buffering can all add delay on top of the encoder path.
|