cvmmap-streamer/docs/mcap_recipes.md

# MCAP Recipes

This guide is the simple, non-GUI path for inspecting RGB+depth MCAP files.

Use it when you want to:

- confirm whether an MCAP is bundled, `copy`, or legacy `/camera/*`
- inspect camera labels, message counts, and timestamp ranges
- export one RGB frame and one decoded depth sample as a concrete example
- understand how `/bundle` changes the meaning of timestamps and sample grouping

For the current bundled/copy layout contract, see [mcap_layout.md](./mcap_layout.md). The older `/camera/*` wire shape is documented separately in [mcap_legacy_single_camera_layout.md](./mcap_legacy_single_camera_layout.md).

## Quick Summary

The repository includes a small example helper:

```bash
uv run python scripts/mcap_rgbd_example.py --help
```

It has two commands:

- `summary`: print layout, per-camera counts, and timestamp ranges
- `export-sample`: write one RGB image plus one depth array/preview

`summary` works with the base Python dependencies:

```bash
uv sync
```

`export-sample` also needs:

- `ffmpeg` on `PATH`
- the optional depth decoder binding:

```bash
uv sync --extra viewer
```

## The Practical Cases

For this helper, there are really two operational cases:

- `bundled`: multiple namespaced camera topics plus `/bundle`
- single-camera stream with no `/bundle`

That second case can appear in two wire shapes:

- `copy`: namespaced topics such as `/zed4/video`
- legacy single-camera: `/camera/video`

Current single-source `zed_svo_to_mcap` output uses the one-camera `copy` shape by default, so even a one-camera file usually looks like namespaced `/{label}/*` topics with no `/bundle`.

The helper treats legacy `/camera/*` as compatible with `copy` by using the implicit camera label `camera`.

## Recipe: Summarize One MCAP

```bash
uv run python scripts/mcap_rgbd_example.py summary <MCAP_PATH>
```

What the summary prints:

- layout and validation status
- camera labels
- per-camera `video`, `depth`, `pose`, `calibration`, `depth_calibration`, and `body` counts
- per-camera video/depth timestamp ranges
- for bundled files only:
  - bundle count
  - bundle timestamp range
  - bundle policy counts
  - per-camera present/corrupted-gap/unknown bundle-member counts

This is the fastest way to answer:

- “is this file bundled, copy, or legacy single-camera?”
- “which camera labels are inside?”
- “do video and depth counts match?”
- “what timestamp range does each camera cover?”

## Recipe: Export One RGB + Depth Sample

```bash
uv run python scripts/mcap_rgbd_example.py export-sample \
    <MCAP_PATH> \
    --output-dir /tmp/mcap_sample
```

For multi-camera or namespaced one-camera files, choose the camera explicitly when needed:

```bash
uv run python scripts/mcap_rgbd_example.py export-sample \
    <MCAP_PATH> \
    --camera-label zed2 \
    --sample-index 25 \
    --output-dir /tmp/mcap_sample_zed2
```

Outputs:

- `rgb.png`
- `depth.npy`
- `depth_preview.png`
- `sample_metadata.json`

`sample_index` is always zero-based per-camera RGB+depth sample order.

That means:

- legacy `/camera/*`: sample `N` is `/camera/video[N]` + `/camera/depth[N]`
- `copy`: sample `N` is `/{label}/video[N]` + `/{label}/depth[N]`
- `bundled`: sample `N` is the `N`th present sample for that camera, not bundle index `N`

In bundled files, `sample_metadata.json` also records the matched `/bundle` member metadata for the selected camera sample.

## Recipe: Understand Bundled vs Non-Bundled Timing

Bundled files intentionally separate bundle time from camera sample time:

- `/bundle.timestamp` is the nominal common-timeline bundle timestamp
- `/zedN/video` and `/zedN/depth` keep the original per-camera sample timestamps

Copy and legacy single-camera files do not have bundle time at all:

- there is no `/bundle`
- each camera topic keeps its own original cadence and timestamps

If you care about grouping, use `/bundle` in bundled files.
For `copy` and legacy single-camera files, treat each camera stream independently.

## Recipe: Inspect `/bundle` In Python

The helper script is intentionally small, but sometimes it is easier to inspect `/bundle` directly.
This snippet shows how to print bundle membership for one camera:

```python
from pathlib import Path

import zed_batch_svo_to_mcap as batch


path = Path("<MCAP_PATH>").expanduser().resolve()
camera_label = "zed1"
reader_module = batch.load_mcap_reader()

with path.open("rb") as stream:
    reader = reader_module.make_reader(stream)
    for schema, channel, message in reader.iter_messages():
        if channel.topic != "/bundle":
            continue
        if schema is None or schema.name != "cvmmap_streamer.BundleManifest":
            continue

        bundle_class, present_value = batch.load_bundle_manifest_type(schema.data)
        bundle = bundle_class()
        bundle.ParseFromString(message.data)

        for member in bundle.members:
            if str(member.camera_label) != camera_label:
                continue
            status_value = int(getattr(member, "status", 0))
            status_field = member.DESCRIPTOR.fields_by_name.get("status")
            status_enum = status_field.enum_type if status_field is not None else None
            status_name = (
                status_enum.values_by_number.get(status_value).name
                if status_enum is not None and status_enum.values_by_number.get(status_value) is not None
                else str(status_value)
            )
            print(bundle.bundle_index, status_name)
            break
```

This is the important mental model:

- `bundled`: follow `/bundle` for grouping
- `copy`: treat each namespaced camera as an independent stream
- legacy `/camera/*`: same model as one-camera `copy`, with the implicit label `camera`