crosstyan/cvmmap-streamer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(zed): remove extracted offline helper tooling

Browse Source 
Drop the offline ZED helper implementations that were moved into zed-offline-tools.\n\nThis removes the standalone conversion binaries, batch/index/inspection scripts, related configs and tests, and the tool-specific support code that no longer belongs in cvmmap-streamer.\n\nThe build files and docs are updated to point at the standalone repo while keeping the streamer runtime surface intact.
This commit is contained in:
crosstyan
2026-04-13 07:50:41 +00:00
parent 30cd956c5c
commit ddea6b0e3d
28 changed files with 16 additions and 12881 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/depth_alignment.md
-86
View File
@@ -1,86 +0,0 @@
# Depth Alignment
Exported ZED MCAP files can carry RGB video and depth at different raster sizes.
For the current kindergarten `zed4` exports, the common pair is:
- video: `1920x1200`
- depth: `960x512`
That means RGB and depth do not share aspect ratio. The files stay alignable because the exporter writes two separate calibration topics:
- `/{label}/calibration` for video
- `/{label}/depth_calibration` for depth
See [mcap_layout.md](./mcap_layout.md) for the topic contract.
## What The Mapping Means
The correct way to align depth onto RGB is to use the two calibration matrices, not to assume matching pixel grids.
For the same camera, with zero distortion and identity rectification, the mapping reduces to a 2D affine transform:
```text
u_rgb = (fx_rgb / fx_depth) * u_depth + (cx_rgb - (fx_rgb / fx_depth) * cx_depth)
v_rgb = (fy_rgb / fy_depth) * v_depth + (cy_rgb - (fy_rgb / fy_depth) * cy_depth)
```
and the inverse:
```text
u_depth = (fx_depth / fx_rgb) * u_rgb + (cx_depth - (fx_depth / fx_rgb) * cx_rgb)
v_depth = (fy_depth / fy_rgb) * v_rgb + (cy_depth - (fy_depth / fy_rgb) * cy_rgb)
```
For the sampled kindergarten `zed4` files, those offsets are effectively zero, so the mapping becomes an anisotropic resize:
```text
u_rgb ~= 2.0 * u_depth
v_rgb ~= 2.34375 * v_depth
```
This is why the practical overlay behavior is a stretch, not a crop.
It is still better to derive the mapping from the two calibration topics than to hardcode `2.0` and `2.34375`, because the exact calibration can vary by camera and export settings.
## Helper Script
Use the alignment helper to inspect the calibration pair and optionally export an example overlay:
```bash
uv run --extra viewer python scripts/mcap_depth_alignment.py \
/workspaces/data/kindergarten/bar/2026-03-18T11-59-41/2026-03-18T11-59-41_zed4.mcap \
--camera-label zed4
```
To export example images:
```bash
uv run --extra viewer python scripts/mcap_depth_alignment.py \
/workspaces/data/kindergarten/bar/2026-03-18T11-59-41/2026-03-18T11-59-41_zed4.mcap \
--camera-label zed4 \
--frame-index 400 \
--output-dir /tmp/zed4_alignment_demo
```
That command writes:
- `rgb_frame.png`
- `depth_native_colorized.png`
- `depth_aligned_to_rgb_colorized.png`
- `depth_overlay_on_rgb.png`
- `rgb_aligned_to_depth.png`
## What The Helper Actually Does
The script:
1. reads `/{label}/calibration` and `/{label}/depth_calibration`
2. computes the affine mapping implied by the two intrinsic matrices
3. decodes one RGB frame and one depth frame from the MCAP
4. warps depth into RGB space with `cv2.warpAffine`
5. optionally warps RGB into depth space with the inverse mapping
For the current exported ZED MCAP contract, that is the right simple alignment path.
If a future export starts carrying non-zero distortion or non-identity rectification, consumers should switch from this affine shortcut to a full camera-model reprojection path.
docs/mcap_layout.md
+1 -1
View File
@@ -137,4 +137,4 @@ For multi-camera `copy` MCAP files, the current validation contract is:
Legacy `/camera/*` validation expectations are documented in [mcap_legacy_single_camera_layout.md](./mcap_legacy_single_camera_layout.md).
The repository-level Python helper [scripts/mcap_bundle_validator.py](../scripts/mcap_bundle_validator.py) understands bundled, copy, and legacy `/camera/*` layouts and reports which one it found before applying the corresponding validation rules.
The standalone helper [zed-offline-tools/scripts/mcap_bundle_validator.py](../../zed-offline-tools/scripts/mcap_bundle_validator.py) understands bundled, copy, and legacy `/camera/*` layouts and reports which one it found before applying the corresponding validation rules.
docs/mcap_recipes.md
-179
View File
@@ -1,179 +0,0 @@
# MCAP Recipes
This guide is the simple, non-GUI path for inspecting RGB+depth MCAP files.
Use it when you want to:
- confirm whether an MCAP is bundled, `copy`, or legacy `/camera/*`
- inspect camera labels, message counts, and timestamp ranges
- export one RGB frame and one decoded depth sample as a concrete example
- understand how `/bundle` changes the meaning of timestamps and sample grouping
For the current bundled/copy layout contract, see [mcap_layout.md](./mcap_layout.md). The older `/camera/*` wire shape is documented separately in [mcap_legacy_single_camera_layout.md](./mcap_legacy_single_camera_layout.md).
## Quick Summary
The repository includes a small example helper:
```bash
uv run python scripts/mcap_rgbd_example.py --help
```
It has two commands:
- `summary`: print layout, per-camera counts, and timestamp ranges
- `export-sample`: write one RGB image plus one depth array/preview
`summary` works with the base Python dependencies:
```bash
uv sync
```
`export-sample` also needs:
- `ffmpeg` on `PATH`
- the optional depth decoder binding:
```bash
uv sync --extra viewer
```
## The Practical Cases
For this helper, there are really two operational cases:
- `bundled`: multiple namespaced camera topics plus `/bundle`
- single-camera stream with no `/bundle`
That second case can appear in two wire shapes:
- `copy`: namespaced topics such as `/zed4/video`
- legacy single-camera: `/camera/video`
Current single-source `zed_svo_to_mcap` output uses the one-camera `copy` shape by default, so even a one-camera file usually looks like namespaced `/{label}/*` topics with no `/bundle`.
The helper treats legacy `/camera/*` as compatible with `copy` by using the implicit camera label `camera`.
## Recipe: Summarize One MCAP
```bash
uv run python scripts/mcap_rgbd_example.py summary <MCAP_PATH>
```
What the summary prints:
- layout and validation status
- camera labels
- per-camera `video`, `depth`, `pose`, `calibration`, `depth_calibration`, and `body` counts
- per-camera video/depth timestamp ranges
- for bundled files only:
- bundle count
- bundle timestamp range
- bundle policy counts
- per-camera present/corrupted-gap/unknown bundle-member counts
This is the fastest way to answer:
- “is this file bundled, copy, or legacy single-camera?”
- “which camera labels are inside?”
- “do video and depth counts match?”
- “what timestamp range does each camera cover?”
## Recipe: Export One RGB + Depth Sample
```bash
uv run python scripts/mcap_rgbd_example.py export-sample \
<MCAP_PATH> \
--output-dir /tmp/mcap_sample
```
For multi-camera or namespaced one-camera files, choose the camera explicitly when needed:
```bash
uv run python scripts/mcap_rgbd_example.py export-sample \
<MCAP_PATH> \
--camera-label zed2 \
--sample-index 25 \
--output-dir /tmp/mcap_sample_zed2
```
Outputs:
- `rgb.png`
- `depth.npy`
- `depth_preview.png`
- `sample_metadata.json`
`sample_index` is always zero-based per-camera RGB+depth sample order.
That means:
- legacy `/camera/*`: sample `N` is `/camera/video[N]` + `/camera/depth[N]`
- `copy`: sample `N` is `/{label}/video[N]` + `/{label}/depth[N]`
- `bundled`: sample `N` is the `N`th present sample for that camera, not bundle index `N`
In bundled files, `sample_metadata.json` also records the matched `/bundle` member metadata for the selected camera sample.
## Recipe: Understand Bundled vs Non-Bundled Timing
Bundled files intentionally separate bundle time from camera sample time:
- `/bundle.timestamp` is the nominal common-timeline bundle timestamp
- `/zedN/video` and `/zedN/depth` keep the original per-camera sample timestamps
Copy and legacy single-camera files do not have bundle time at all:
- there is no `/bundle`
- each camera topic keeps its own original cadence and timestamps
If you care about grouping, use `/bundle` in bundled files.
For `copy` and legacy single-camera files, treat each camera stream independently.
## Recipe: Inspect `/bundle` In Python
The helper script is intentionally small, but sometimes it is easier to inspect `/bundle` directly.
This snippet shows how to print bundle membership for one camera:
```python
from pathlib import Path
import zed_batch_svo_to_mcap as batch
path = Path("<MCAP_PATH>").expanduser().resolve()
camera_label = "zed1"
reader_module = batch.load_mcap_reader()
with path.open("rb") as stream:
reader = reader_module.make_reader(stream)
for schema, channel, message in reader.iter_messages():
if channel.topic != "/bundle":
continue
if schema is None or schema.name != "cvmmap_streamer.BundleManifest":
continue
bundle_class, present_value = batch.load_bundle_manifest_type(schema.data)
bundle = bundle_class()
bundle.ParseFromString(message.data)
for member in bundle.members:
if str(member.camera_label) != camera_label:
continue
status_value = int(getattr(member, "status", 0))
status_field = member.DESCRIPTOR.fields_by_name.get("status")
status_enum = status_field.enum_type if status_field is not None else None
status_name = (
status_enum.values_by_number.get(status_value).name
if status_enum is not None and status_enum.values_by_number.get(status_value) is not None
else str(status_value)
)
print(bundle.bundle_index, status_name)
break
```
This is the important mental model:
- `bundled`: follow `/bundle` for grouping
- `copy`: treat each namespaced camera as an independent stream
- legacy `/camera/*`: same model as one-camera `copy`, with the implicit label `camera`
docs/zed_segment_time_index.md
-97
View File
@@ -1,97 +0,0 @@
# ZED Segment Time Index
`scripts/zed_segment_time_index.py` builds and queries an embedded DuckDB index for bundled ZED segment folders.
Default artifact name:
```text
<DATASET_ROOT>/segment_time_index.duckdb
```
Primary commands:
```bash
uv run python scripts/zed_segment_time_index.py build <DATASET_ROOT>
uv run python scripts/zed_segment_time_index.py query <DATASET_ROOT> --at 2026-03-18T12-00-23
uv run python scripts/zed_segment_time_index.py query <DATASET_ROOT> --start 2026-03-18T12-00-23 --end 2026-03-18T12-00-30
```
## Data Source Rules
- Segment discovery is recursive and follows the same multi-camera layout assumptions as the batch ZED tooling.
- A directory is considered a valid segment when it contains at least two unique `*_zedN.svo` or `*_zedN.svo2` files and no duplicate camera labels.
- Timing is sourced from the segment MCAP, not from the SVO/SVO2 files.
- A valid segment is skipped when it has no `.mcap` file or more than one `.mcap` file in the segment directory.
## MCAP Bounds Extraction
`build/bin/mcap_video_bounds` scans `foxglove.CompressedVideo` messages in one MCAP and emits:
- `start_ns`
- `end_ns`
- `duration_ns`
- `video_message_count`
- `start_iso_utc`
- `end_iso_utc`
The helper prefers the protobuf `CompressedVideo.timestamp` field and falls back to MCAP `logTime` when that field is zero.
## DuckDB Layout
The database contains two tables: `meta` and `segments`.
### `meta`
Key-value metadata for the index:
- `schema_version`: current schema version, currently `1`
- `dataset_root`: absolute dataset root used when the index was built
- `built_at_utc`: build timestamp in UTC
- `default_timezone`: inferred dataset wall-clock timezone used when querying with `--timezone dataset`
### `segments`
One row per indexed segment.
| Column | Type | Meaning |
|---|---|---|
| `segment_dir` | `VARCHAR` | Absolute path to the segment directory |
| `relative_segment_dir` | `VARCHAR` | Path relative to the dataset root |
| `group_path` | `VARCHAR` | Parent path of the segment within the dataset |
| `activity` | `VARCHAR` | First path component under the dataset root |
| `segment_name` | `VARCHAR` | Segment directory basename |
| `mcap_path` | `VARCHAR` | Absolute MCAP path used for timing |
| `start_ns` | `BIGINT` | Earliest video timestamp in nanoseconds since Unix epoch |
| `end_ns` | `BIGINT` | Latest video timestamp in nanoseconds since Unix epoch |
| `duration_ns` | `BIGINT` | `end_ns - start_ns` |
| `start_iso_utc` | `VARCHAR` | UTC rendering of `start_ns` |
| `end_iso_utc` | `VARCHAR` | UTC rendering of `end_ns` |
| `camera_count` | `INTEGER` | Number of discovered camera inputs in the segment directory |
| `camera_labels` | `VARCHAR` | Comma-separated camera labels, for example `zed1,zed2,zed3,zed4` |
| `video_message_count` | `BIGINT` | Number of `foxglove.CompressedVideo` messages observed in the MCAP |
| `index_source` | `VARCHAR` | Current extractor label, currently `mcap_video_bounds` |
Indexes are created on `start_ns` and `end_ns`.
## Query Semantics
- `--at` performs an overlap lookup, not just an exact nanosecond equality check.
- Query precision follows the precision supplied by the user.
- A second-precision value like `2026-03-18T12-00-23` is treated as the whole second `[12:00:23.000, 12:00:23.999999999]`.
- Integer epochs are widened similarly by their apparent unit:
- 10 digits or fewer: seconds
- 11-13 digits: milliseconds
- 14-16 digits: microseconds
- 17+ digits: nanoseconds
- `--start/--end` returns every segment whose `[start_ns, end_ns]` overlaps the requested interval.
## Timezone Behavior
- Query default is `--timezone dataset`.
- `dataset` resolves to the `default_timezone` stored in `meta`.
- If inference is unavailable, the script falls back to `local`.
- Explicit values are also accepted:
- `local`
- `UTC`
- fixed offsets such as `UTC+08:00`
- IANA zone names such as `Asia/Shanghai`