cvmmap-streamer/docs/mcap_layout.md

MCAP Layout

cvmmap-streamer currently deals with three active MCAP layout shapes:

• bundled multi-camera timeline export
• multi-camera copy export
• single-source zed_svo_to_mcap output, which now uses the one-camera copy shape

This document covers the current bundled and copy contracts.

Legacy /camera/* is documented separately in mcap_legacy_single_camera_layout.md. Conceptually, it is compatible with one-camera copy if you treat the camera label as the literal camera instead of zedN or another derived label.

Bundled Multi-Camera Layout

Bundled export namespaces each camera stream and adds one bundle manifest topic:

Topic	Schema / Encoding	Notes
/bundle	cvmmap_streamer.BundleManifest	One message per emitted bundle
/zedN/video	foxglove.CompressedVideo	Per-camera encoded video
/zedN/depth	cvmmap_streamer.DepthMap	Per-camera depth
/zedN/calibration	foxglove.CameraCalibration	Per-camera video intrinsics
/zedN/depth_calibration	foxglove.CameraCalibration	Written only when exported depth resolution differs from video
/zedN/pose	foxglove.PoseInFrame	Optional per-camera pose
/zedN/body	raw cvmmap.body_tracking.v1	Optional raw body packets; see mcap_body_tracking.md

nearest is the default bundled policy. It emits one bundle on a common timeline, but it keeps the original per-camera timestamps on /zedN/video, /zedN/depth, and optional /zedN/pose.

strict is still available. In strict mode, bundle membership is thresholded by timestamp skew and --sync-tolerance-ms applies. In nearest mode, --sync-tolerance-ms is not used.

Because nearest emits on the slowest common timeline, faster cameras can legitimately end up with the same message count as slower cameras in bundled output.

Multi-Camera Copy Layout

copy export also namespaces each camera stream under /zedN/..., but it does not emit /bundle.

Topic	Schema / Encoding	Notes
/zedN/video	foxglove.CompressedVideo	Per-camera encoded video
/zedN/depth	cvmmap_streamer.DepthMap	Per-camera depth
/zedN/calibration	foxglove.CameraCalibration	Per-camera video intrinsics
/zedN/depth_calibration	foxglove.CameraCalibration	Written only when exported depth resolution differs from video
/zedN/pose	foxglove.PoseInFrame	Optional per-camera pose
/zedN/body	raw cvmmap.body_tracking.v1	Optional raw body packets; see mcap_body_tracking.md

copy preserves each camera topic exactly as an independent stream inside one MCAP:

• no /bundle topic
• no shared bundle index
• no resampling to a common timeline
• original per-camera timestamps and cadence are preserved

--copy-range common trims each camera to the common overlap window without resampling. --copy-range full preserves each camera’s full readable timestamp range from the grouped segment.

Single-source zed_svo_to_mcap now writes this same wire shape with one derived label, for example:

• /zed4/video, /zed4/depth, /zed4/calibration for *_zed4.svo2
• /cam1/video, /cam1/depth, /cam1/calibration when the filename has no zedN suffix

Bundle Manifest Contract

/bundle is the authoritative grouping contract for bundled MCAP files. Consumers should not infer grouping from identical MCAP logTime values or from matching per-camera timestamps.

cvmmap_streamer.BundleManifest contains:

• timestamp: the nominal bundle timestamp on the common timeline
• bundle_index: zero-based emitted bundle index
• policy: BUNDLE_POLICY_NEAREST or BUNDLE_POLICY_STRICT
• members: one entry per camera label in the bundle

Each bundle member contains:

• camera_label
• timestamp: the original camera sample timestamp when a payload is present
• delta_ns: member.timestamp - bundle.timestamp when a payload is present
• status
• corrupted_frames_skipped

Current member statuses:

• BUNDLE_MEMBER_STATUS_PRESENT
• BUNDLE_MEMBER_STATUS_CORRUPTED_GAP

CORRUPTED_GAP means the exporter skipped one or more unreadable ZED frames and intentionally emitted no video, depth, or pose payload for that camera for this bundle. corrupted_frames_skipped reports the size of the skipped run that led to the next recovered readable frame.

Timestamp Semantics

The bundled MCAP contract intentionally separates bundle time from per-camera sample time:

• /bundle.logTime and /bundle.publishTime use the nominal bundle timestamp
• /zedN/video, /zedN/depth, and /zedN/pose use the original camera sample timestamp
• calibration and depth-calibration messages use the timestamp of the first emitted sample on that camera

This means a single bundle can legitimately contain different per-camera timestamps, especially in nearest mode.

copy has no separate bundle time. Its /zedN/video, /zedN/depth, and /zedN/pose messages all use the original per-camera sample timestamp directly.

Corruption And Partial Bundles

Bundled nearest export is resilient to ZED CORRUPTED FRAME runs:

• unreadable tail frames are treated as end-of-stream
• mid-stream corruption is skipped until a readable frame is found
• bundles inside the unreadable gap still exist
• the affected camera is marked CORRUPTED_GAP in /bundle
• no /zedN/video, /zedN/depth, or /zedN/pose message is written for that camera for those bundles

Bundled strict export stays strict:

• corruption is skipped internally until recovery
• only fully present, threshold-qualified groups are emitted

copy is also resilient to ZED CORRUPTED FRAME runs:

• unreadable tail frames are treated as end-of-stream
• mid-stream corruption is skipped until a readable frame is found
• there is no placeholder or manifest entry because copy has no grouping contract
• the affected camera topic simply resumes at the recovered readable frame

Validation Expectations

For bundled MCAP files, the current validation contract is:

• /bundle must exist
• every bundle must contain one member per camera label
• for each camera, the count of BUNDLE_MEMBER_STATUS_PRESENT members must match the number of /zedN/video messages
• for each camera, the count of BUNDLE_MEMBER_STATUS_PRESENT members must match the number of /zedN/depth messages

That is why a partially written MCAP with topic presence but mismatched counts is treated as invalid.

For multi-camera copy MCAP files, the current validation contract is:

• /bundle must not exist
• each camera must have /zedN/video, /zedN/depth, and /zedN/calibration
• for each camera, /zedN/video and /zedN/depth message counts must match

Legacy /camera/* validation expectations are documented in mcap_legacy_single_camera_layout.md.

The repository-level Python helper scripts/mcap_bundle_validator.py understands bundled, copy, and legacy /camera/* layouts and reports which one it found before applying the corresponding validation rules.