90 lines
2.6 KiB
Markdown
90 lines
2.6 KiB
Markdown
# Encoded Passthrough From `cv-mmap` ABI v2.1
|
|
|
|
## Summary
|
|
|
|
`cvmmap-streamer` can now consume encoded access units directly from a `cv-mmap` shared-memory snapshot when the producer exposes the ABI v2.1 encoded plane.
|
|
|
|
This is intended for producers such as `cv-mmap` with `video.backend = "udp_rtp"`:
|
|
|
|
- `cv-mmap` receives RTP H.265
|
|
- `cv-mmap` parses once and publishes raw BGR plus encoded AU in the same snapshot
|
|
- `cvmmap-streamer` can forward or mux the encoded AU directly
|
|
|
|
The video no longer needs a decode -> encode round trip inside the streamer.
|
|
|
|
## Input selection
|
|
|
|
`input.video_source` controls how the pipeline uses the snapshot:
|
|
|
|
```toml
|
|
[input]
|
|
uri = "cvmmap://default"
|
|
nats_url = "nats://localhost:4222"
|
|
video_source = "auto"
|
|
```
|
|
|
|
Valid values:
|
|
|
|
- `auto`: prefer encoded passthrough when the encoded AU plane is present, otherwise use raw encoding
|
|
- `raw`: always use the raw left plane and the local encoder backend
|
|
- `encoded`: require the encoded AU plane; fail if it is missing
|
|
|
|
CLI equivalent:
|
|
|
|
```text
|
|
--input-video-source auto|raw|encoded
|
|
```
|
|
|
|
## Encoded passthrough behavior
|
|
|
|
When `video_source` resolves to encoded mode:
|
|
|
|
- no encoder backend is created
|
|
- the streamer reads codec, bitstream format, keyframe flag, frame rate, and stream PTS from the SHM metadata
|
|
- width and height still come from the raw left-plane frame metadata
|
|
- the encoded AU is forwarded as-is
|
|
|
|
Current assumptions:
|
|
|
|
- codec is H.265 or H.264
|
|
- bitstream format is Annex B
|
|
- one SHM encoded plane contains one AU
|
|
- time base is treated as `1/1e9`
|
|
|
|
## Outputs
|
|
|
|
### RTP
|
|
|
|
RTP output accepts passthrough access units directly.
|
|
|
|
- in encoded mode, SDP signaling and RTP packetization follow the codec from SHM metadata
|
|
- `config.encoder.codec` is ignored for RTP while passthrough is active
|
|
- raw mode still uses the configured local encoder and its codec
|
|
|
|
### RTMP
|
|
|
|
RTMP output accepts passthrough access units directly.
|
|
|
|
- If decoder config is present, it is used.
|
|
- If decoder config is absent, the output relies on in-band parameter sets from keyframes.
|
|
|
|
For `cv-mmap udp_rtp`, this works because the producer republishes keyframes with VPS/SPS/PPS in-band.
|
|
|
|
### MCAP
|
|
|
|
MCAP recording writes the compressed video directly.
|
|
|
|
- no local encoder is used
|
|
- keyframes are written exactly from the encoded AU payload
|
|
- depth and body-tracking recording continue to use the same raw/depth/NATS paths as before
|
|
|
|
## Fallback to raw mode
|
|
|
|
If the producer does not expose the encoded plane, `auto` mode falls back to the existing raw pipeline:
|
|
|
|
- read raw BGR from SHM
|
|
- push frames through the configured encoder backend
|
|
- publish or record the encoder output
|
|
|
|
This keeps existing producers compatible.
|