cvmmap-streamer/docs/mcap_layout.md

MCAP Layout

cvmmap-streamer currently deals with three active MCAP layouts plus one legacy single-camera layout:

• legacy single-camera /camera/* MCAP export
• bundled multi-camera timeline export
• multi-camera copy export
• single-source zed_svo_to_mcap output, which now uses the one-camera copy shape

This document covers the topic layout, schema types, and timestamp semantics for both.

Legacy Single-Camera Layout

Default topics:

Topic	Schema / Encoding	Notes
/camera/video	foxglove.CompressedVideo	H.264 or H.265 Annex B access units
/camera/depth	cvmmap_streamer.DepthMap	RVL-compressed depth payload
/camera/calibration	foxglove.CameraCalibration	Video intrinsics
/camera/depth_calibration	foxglove.CameraCalibration	Written only when depth resolution differs from video
/camera/pose	foxglove.PoseInFrame	Optional; only when pose export is enabled and tracking is valid
/camera/body	raw cvmmap.body_tracking.v1	Optional raw body-tracking packets; see mcap_body_tracking.md

Single-camera export preserves the original per-frame source timestamp on video, depth, and pose messages.

This layout is still used by the generic sink/testers in the repository, but it is no longer the default shape written by zed_svo_to_mcap for a single .svo or .svo2 input.

Bundled Multi-Camera Layout

Bundled export namespaces each camera stream and adds one bundle manifest topic:

Topic	Schema / Encoding	Notes
/bundle	cvmmap_streamer.BundleManifest	One message per emitted bundle
/zedN/video	foxglove.CompressedVideo	Per-camera encoded video
/zedN/depth	cvmmap_streamer.DepthMap	Per-camera depth
/zedN/calibration	foxglove.CameraCalibration	Per-camera video intrinsics
/zedN/depth_calibration	foxglove.CameraCalibration	Written only when exported depth resolution differs from video
/zedN/pose	foxglove.PoseInFrame	Optional per-camera pose
/zedN/body	raw cvmmap.body_tracking.v1	Optional raw body packets; see mcap_body_tracking.md

nearest is the default bundled policy. It emits one bundle on a common timeline, but it keeps the original per-camera timestamps on /zedN/video, /zedN/depth, and optional /zedN/pose.

strict is still available. In strict mode, bundle membership is thresholded by timestamp skew and --sync-tolerance-ms applies. In nearest mode, --sync-tolerance-ms is not used.

Because nearest emits on the slowest common timeline, faster cameras can legitimately end up with the same message count as slower cameras in bundled output.

Multi-Camera Copy Layout

copy export also namespaces each camera stream under /zedN/..., but it does not emit /bundle.

Topic	Schema / Encoding	Notes
/zedN/video	foxglove.CompressedVideo	Per-camera encoded video
/zedN/depth	cvmmap_streamer.DepthMap	Per-camera depth
/zedN/calibration	foxglove.CameraCalibration	Per-camera video intrinsics
/zedN/depth_calibration	foxglove.CameraCalibration	Written only when exported depth resolution differs from video
/zedN/pose	foxglove.PoseInFrame	Optional per-camera pose
/zedN/body	raw cvmmap.body_tracking.v1	Optional raw body packets; see mcap_body_tracking.md

copy preserves each camera topic exactly as an independent stream inside one MCAP:

• no /bundle topic
• no shared bundle index
• no resampling to a common timeline
• original per-camera timestamps and cadence are preserved

--copy-range common trims each camera to the common overlap window without resampling. --copy-range full preserves each camera’s full readable timestamp range from the grouped segment.

Single-source zed_svo_to_mcap now writes this same wire shape with one derived label, for example:

• /zed4/video, /zed4/depth, /zed4/calibration for *_zed4.svo2
• /cam1/video, /cam1/depth, /cam1/calibration when the filename has no zedN suffix

Bundle Manifest Contract

/bundle is the authoritative grouping contract for bundled MCAP files. Consumers should not infer grouping from identical MCAP logTime values or from matching per-camera timestamps.

cvmmap_streamer.BundleManifest contains:

• timestamp: the nominal bundle timestamp on the common timeline
• bundle_index: zero-based emitted bundle index
• policy: BUNDLE_POLICY_NEAREST or BUNDLE_POLICY_STRICT
• members: one entry per camera label in the bundle

Each bundle member contains:

• camera_label
• timestamp: the original camera sample timestamp when a payload is present
• delta_ns: member.timestamp - bundle.timestamp when a payload is present
• status
• corrupted_frames_skipped

Current member statuses:

• BUNDLE_MEMBER_STATUS_PRESENT
• BUNDLE_MEMBER_STATUS_CORRUPTED_GAP

CORRUPTED_GAP means the exporter skipped one or more unreadable ZED frames and intentionally emitted no video, depth, or pose payload for that camera for this bundle. corrupted_frames_skipped reports the size of the skipped run that led to the next recovered readable frame.

Timestamp Semantics

The bundled MCAP contract intentionally separates bundle time from per-camera sample time:

• /bundle.logTime and /bundle.publishTime use the nominal bundle timestamp
• /zedN/video, /zedN/depth, and /zedN/pose use the original camera sample timestamp
• calibration and depth-calibration messages use the timestamp of the first emitted sample on that camera

This means a single bundle can legitimately contain different per-camera timestamps, especially in nearest mode.

copy has no separate bundle time. Its /zedN/video, /zedN/depth, and /zedN/pose messages all use the original per-camera sample timestamp directly.

Corruption And Partial Bundles

Bundled nearest export is resilient to ZED CORRUPTED FRAME runs:

• unreadable tail frames are treated as end-of-stream
• mid-stream corruption is skipped until a readable frame is found
• bundles inside the unreadable gap still exist
• the affected camera is marked CORRUPTED_GAP in /bundle
• no /zedN/video, /zedN/depth, or /zedN/pose message is written for that camera for those bundles

Bundled strict export stays strict:

• corruption is skipped internally until recovery
• only fully present, threshold-qualified groups are emitted

copy is also resilient to ZED CORRUPTED FRAME runs:

• unreadable tail frames are treated as end-of-stream
• mid-stream corruption is skipped until a readable frame is found
• there is no placeholder or manifest entry because copy has no grouping contract
• the affected camera topic simply resumes at the recovered readable frame

Validation Expectations

For single-camera MCAP files, the current validation contract is:

• /camera/video must exist and contain at least one message
• /camera/depth must exist and contain at least one message
• /camera/calibration must exist exactly once
• /camera/video and /camera/depth message counts must match
• /camera/depth_calibration may appear zero or one time
• /camera/pose is optional, but it may not outnumber /camera/video

For bundled MCAP files, the current validation contract is:

• /bundle must exist
• every bundle must contain one member per camera label
• for each camera, the count of BUNDLE_MEMBER_STATUS_PRESENT members must match the number of /zedN/video messages
• for each camera, the count of BUNDLE_MEMBER_STATUS_PRESENT members must match the number of /zedN/depth messages

That is why a partially written MCAP with topic presence but mismatched counts is treated as invalid.

For multi-camera copy MCAP files, the current validation contract is:

• /bundle must not exist
• each camera must have /zedN/video, /zedN/depth, and /zedN/calibration
• for each camera, /zedN/video and /zedN/depth message counts must match

The repository-level Python helper scripts/mcap_bundle_validator.py understands bundled, copy, and legacy /camera/* layouts and reports which one it found before applying the corresponding validation rules.