Add encoded SHM passthrough support
This commit is contained in:
@@ -0,0 +1,81 @@
|
||||
# Encoded Passthrough From `cv-mmap` ABI v2.1
|
||||
|
||||
## Summary
|
||||
|
||||
`cvmmap-streamer` can now consume encoded access units directly from a `cv-mmap` shared-memory snapshot when the producer exposes the ABI v2.1 encoded plane.
|
||||
|
||||
This is intended for producers such as `cv-mmap` with `video.backend = "udp_rtp"`:
|
||||
|
||||
- `cv-mmap` receives RTP H.265
|
||||
- `cv-mmap` parses once and publishes raw BGR plus encoded AU in the same snapshot
|
||||
- `cvmmap-streamer` can forward or mux the encoded AU directly
|
||||
|
||||
The video no longer needs a decode -> encode round trip inside the streamer.
|
||||
|
||||
## Input selection
|
||||
|
||||
`input.video_source` controls how the pipeline uses the snapshot:
|
||||
|
||||
```toml
|
||||
[input]
|
||||
uri = "cvmmap://default"
|
||||
nats_url = "nats://localhost:4222"
|
||||
video_source = "auto"
|
||||
```
|
||||
|
||||
Valid values:
|
||||
|
||||
- `auto`: prefer encoded passthrough when the encoded AU plane is present, otherwise use raw encoding
|
||||
- `raw`: always use the raw left plane and the local encoder backend
|
||||
- `encoded`: require the encoded AU plane; fail if it is missing
|
||||
|
||||
CLI equivalent:
|
||||
|
||||
```text
|
||||
--input-video-source auto|raw|encoded
|
||||
```
|
||||
|
||||
## Encoded passthrough behavior
|
||||
|
||||
When `video_source` resolves to encoded mode:
|
||||
|
||||
- no encoder backend is created
|
||||
- the streamer reads codec, bitstream format, keyframe flag, frame rate, and stream PTS from the SHM metadata
|
||||
- width and height still come from the raw left-plane frame metadata
|
||||
- the encoded AU is forwarded as-is
|
||||
|
||||
Current assumptions:
|
||||
|
||||
- codec is H.265 or H.264
|
||||
- bitstream format is Annex B
|
||||
- one SHM encoded plane contains one AU
|
||||
- time base is treated as `1/1e9`
|
||||
|
||||
## Outputs
|
||||
|
||||
### RTMP
|
||||
|
||||
RTMP output accepts passthrough access units directly.
|
||||
|
||||
- If decoder config is present, it is used.
|
||||
- If decoder config is absent, the output relies on in-band parameter sets from keyframes.
|
||||
|
||||
For `cv-mmap udp_rtp`, this works because the producer republishes keyframes with VPS/SPS/PPS in-band.
|
||||
|
||||
### MCAP
|
||||
|
||||
MCAP recording writes the compressed video directly.
|
||||
|
||||
- no local encoder is used
|
||||
- keyframes are written exactly from the encoded AU payload
|
||||
- depth and body-tracking recording continue to use the same raw/depth/NATS paths as before
|
||||
|
||||
## Fallback to raw mode
|
||||
|
||||
If the producer does not expose the encoded plane, `auto` mode falls back to the existing raw pipeline:
|
||||
|
||||
- read raw BGR from SHM
|
||||
- push frames through the configured encoder backend
|
||||
- publish or record the encoder output
|
||||
|
||||
This keeps existing producers compatible.
|
||||
Reference in New Issue
Block a user