crosstyan
ffd246e508
Namespace single-source zed_svo_to_mcap outputs with a derived camera label so one-camera exports use the same copy-layout topic shape as current multi-camera copy files. Use the per-camera frame and pose reference identifiers for those single-source outputs, document copy as the current one-camera wire shape, and clarify that legacy /camera/* files are retained only as a legacy contract. Add a small mcap_rgbd_example helper plus mcap_recipes guide for summarizing bundled/copy MCAPs and exporting one RGB/depth sample, and update the validator and viewer wording/behavior to match the bundled-vs-copy semantics.
163 lines
8.2 KiB
Markdown
163 lines
8.2 KiB
Markdown
# MCAP Layout
|
||
|
||
`cvmmap-streamer` currently deals with three active MCAP layouts plus one legacy single-camera layout:
|
||
|
||
- legacy single-camera `/camera/*` MCAP export
|
||
- bundled multi-camera timeline export
|
||
- multi-camera copy export
|
||
- single-source `zed_svo_to_mcap` output, which now uses the one-camera `copy` shape
|
||
|
||
This document covers the topic layout, schema types, and timestamp semantics for both.
|
||
|
||
## Legacy Single-Camera Layout
|
||
|
||
Default topics:
|
||
|
||
| Topic | Schema / Encoding | Notes |
|
||
|------|------|------|
|
||
| `/camera/video` | `foxglove.CompressedVideo` | H.264 or H.265 Annex B access units |
|
||
| `/camera/depth` | `cvmmap_streamer.DepthMap` | RVL-compressed depth payload |
|
||
| `/camera/calibration` | `foxglove.CameraCalibration` | Video intrinsics |
|
||
| `/camera/depth_calibration` | `foxglove.CameraCalibration` | Written only when depth resolution differs from video |
|
||
| `/camera/pose` | `foxglove.PoseInFrame` | Optional; only when pose export is enabled and tracking is valid |
|
||
| `/camera/body` | raw `cvmmap.body_tracking.v1` | Optional raw body-tracking packets; see [mcap_body_tracking.md](./mcap_body_tracking.md) |
|
||
|
||
Single-camera export preserves the original per-frame source timestamp on video, depth, and pose messages.
|
||
|
||
This layout is still used by the generic sink/testers in the repository, but it is no longer the default shape written by `zed_svo_to_mcap` for a single `.svo` or `.svo2` input.
|
||
|
||
## Bundled Multi-Camera Layout
|
||
|
||
Bundled export namespaces each camera stream and adds one bundle manifest topic:
|
||
|
||
| Topic | Schema / Encoding | Notes |
|
||
|------|------|------|
|
||
| `/bundle` | `cvmmap_streamer.BundleManifest` | One message per emitted bundle |
|
||
| `/zedN/video` | `foxglove.CompressedVideo` | Per-camera encoded video |
|
||
| `/zedN/depth` | `cvmmap_streamer.DepthMap` | Per-camera depth |
|
||
| `/zedN/calibration` | `foxglove.CameraCalibration` | Per-camera video intrinsics |
|
||
| `/zedN/depth_calibration` | `foxglove.CameraCalibration` | Written only when exported depth resolution differs from video |
|
||
| `/zedN/pose` | `foxglove.PoseInFrame` | Optional per-camera pose |
|
||
| `/zedN/body` | raw `cvmmap.body_tracking.v1` | Optional raw body packets; see [mcap_body_tracking.md](./mcap_body_tracking.md) |
|
||
|
||
`nearest` is the default bundled policy. It emits one bundle on a common timeline, but it keeps the original per-camera timestamps on `/zedN/video`, `/zedN/depth`, and optional `/zedN/pose`.
|
||
|
||
`strict` is still available. In strict mode, bundle membership is thresholded by timestamp skew and `--sync-tolerance-ms` applies. In `nearest` mode, `--sync-tolerance-ms` is not used.
|
||
|
||
Because `nearest` emits on the slowest common timeline, faster cameras can legitimately end up with the same message count as slower cameras in bundled output.
|
||
|
||
## Multi-Camera Copy Layout
|
||
|
||
`copy` export also namespaces each camera stream under `/zedN/...`, but it does not emit `/bundle`.
|
||
|
||
| Topic | Schema / Encoding | Notes |
|
||
|------|------|------|
|
||
| `/zedN/video` | `foxglove.CompressedVideo` | Per-camera encoded video |
|
||
| `/zedN/depth` | `cvmmap_streamer.DepthMap` | Per-camera depth |
|
||
| `/zedN/calibration` | `foxglove.CameraCalibration` | Per-camera video intrinsics |
|
||
| `/zedN/depth_calibration` | `foxglove.CameraCalibration` | Written only when exported depth resolution differs from video |
|
||
| `/zedN/pose` | `foxglove.PoseInFrame` | Optional per-camera pose |
|
||
| `/zedN/body` | raw `cvmmap.body_tracking.v1` | Optional raw body packets; see [mcap_body_tracking.md](./mcap_body_tracking.md) |
|
||
|
||
`copy` preserves each camera topic exactly as an independent stream inside one MCAP:
|
||
|
||
- no `/bundle` topic
|
||
- no shared bundle index
|
||
- no resampling to a common timeline
|
||
- original per-camera timestamps and cadence are preserved
|
||
|
||
`--copy-range common` trims each camera to the common overlap window without resampling. `--copy-range full` preserves each camera’s full readable timestamp range from the grouped segment.
|
||
|
||
Single-source `zed_svo_to_mcap` now writes this same wire shape with one derived label, for example:
|
||
|
||
- `/zed4/video`, `/zed4/depth`, `/zed4/calibration` for `*_zed4.svo2`
|
||
- `/cam1/video`, `/cam1/depth`, `/cam1/calibration` when the filename has no `zedN` suffix
|
||
|
||
## Bundle Manifest Contract
|
||
|
||
`/bundle` is the authoritative grouping contract for bundled MCAP files. Consumers should not infer grouping from identical MCAP `logTime` values or from matching per-camera timestamps.
|
||
|
||
`cvmmap_streamer.BundleManifest` contains:
|
||
|
||
- `timestamp`: the nominal bundle timestamp on the common timeline
|
||
- `bundle_index`: zero-based emitted bundle index
|
||
- `policy`: `BUNDLE_POLICY_NEAREST` or `BUNDLE_POLICY_STRICT`
|
||
- `members`: one entry per camera label in the bundle
|
||
|
||
Each bundle member contains:
|
||
|
||
- `camera_label`
|
||
- `timestamp`: the original camera sample timestamp when a payload is present
|
||
- `delta_ns`: `member.timestamp - bundle.timestamp` when a payload is present
|
||
- `status`
|
||
- `corrupted_frames_skipped`
|
||
|
||
Current member statuses:
|
||
|
||
- `BUNDLE_MEMBER_STATUS_PRESENT`
|
||
- `BUNDLE_MEMBER_STATUS_CORRUPTED_GAP`
|
||
|
||
`CORRUPTED_GAP` means the exporter skipped one or more unreadable ZED frames and intentionally emitted no video, depth, or pose payload for that camera for this bundle. `corrupted_frames_skipped` reports the size of the skipped run that led to the next recovered readable frame.
|
||
|
||
## Timestamp Semantics
|
||
|
||
The bundled MCAP contract intentionally separates bundle time from per-camera sample time:
|
||
|
||
- `/bundle.logTime` and `/bundle.publishTime` use the nominal bundle timestamp
|
||
- `/zedN/video`, `/zedN/depth`, and `/zedN/pose` use the original camera sample timestamp
|
||
- calibration and depth-calibration messages use the timestamp of the first emitted sample on that camera
|
||
|
||
This means a single bundle can legitimately contain different per-camera timestamps, especially in `nearest` mode.
|
||
|
||
`copy` has no separate bundle time. Its `/zedN/video`, `/zedN/depth`, and `/zedN/pose` messages all use the original per-camera sample timestamp directly.
|
||
|
||
## Corruption And Partial Bundles
|
||
|
||
Bundled `nearest` export is resilient to ZED `CORRUPTED FRAME` runs:
|
||
|
||
- unreadable tail frames are treated as end-of-stream
|
||
- mid-stream corruption is skipped until a readable frame is found
|
||
- bundles inside the unreadable gap still exist
|
||
- the affected camera is marked `CORRUPTED_GAP` in `/bundle`
|
||
- no `/zedN/video`, `/zedN/depth`, or `/zedN/pose` message is written for that camera for those bundles
|
||
|
||
Bundled `strict` export stays strict:
|
||
|
||
- corruption is skipped internally until recovery
|
||
- only fully present, threshold-qualified groups are emitted
|
||
|
||
`copy` is also resilient to ZED `CORRUPTED FRAME` runs:
|
||
|
||
- unreadable tail frames are treated as end-of-stream
|
||
- mid-stream corruption is skipped until a readable frame is found
|
||
- there is no placeholder or manifest entry because `copy` has no grouping contract
|
||
- the affected camera topic simply resumes at the recovered readable frame
|
||
|
||
## Validation Expectations
|
||
|
||
For single-camera MCAP files, the current validation contract is:
|
||
|
||
- `/camera/video` must exist and contain at least one message
|
||
- `/camera/depth` must exist and contain at least one message
|
||
- `/camera/calibration` must exist exactly once
|
||
- `/camera/video` and `/camera/depth` message counts must match
|
||
- `/camera/depth_calibration` may appear zero or one time
|
||
- `/camera/pose` is optional, but it may not outnumber `/camera/video`
|
||
|
||
For bundled MCAP files, the current validation contract is:
|
||
|
||
- `/bundle` must exist
|
||
- every bundle must contain one member per camera label
|
||
- for each camera, the count of `BUNDLE_MEMBER_STATUS_PRESENT` members must match the number of `/zedN/video` messages
|
||
- for each camera, the count of `BUNDLE_MEMBER_STATUS_PRESENT` members must match the number of `/zedN/depth` messages
|
||
|
||
That is why a partially written MCAP with topic presence but mismatched counts is treated as invalid.
|
||
|
||
For multi-camera `copy` MCAP files, the current validation contract is:
|
||
|
||
- `/bundle` must not exist
|
||
- each camera must have `/zedN/video`, `/zedN/depth`, and `/zedN/calibration`
|
||
- for each camera, `/zedN/video` and `/zedN/depth` message counts must match
|
||
|
||
The repository-level Python helper [scripts/mcap_bundle_validator.py](../scripts/mcap_bundle_validator.py) understands bundled, copy, and legacy `/camera/*` layouts and reports which one it found before applying the corresponding validation rules.
|