Commit Graph

17 Commits

Author SHA1 Message Date
crosstyan ffd246e508 feat(mcap): switch single-source exports to copy layout
Namespace single-source zed_svo_to_mcap outputs with a derived camera label so one-camera exports use the same copy-layout topic shape as current multi-camera copy files.

Use the per-camera frame and pose reference identifiers for those single-source outputs, document copy as the current one-camera wire shape, and clarify that legacy /camera/* files are retained only as a legacy contract.

Add a small mcap_rgbd_example helper plus mcap_recipes guide for summarizing bundled/copy MCAPs and exporting one RGB/depth sample, and update the validator and viewer wording/behavior to match the bundled-vs-copy semantics.
2026-03-24 18:44:44 +08:00
crosstyan 8597976678 feat(mcap): add copy mode for multi-camera exports
Add an opt-in multi-camera copy layout for zed_svo_to_mcap.

copy mode preserves each camera's original timestamps and cadence,
writes namespaced /zedN topics without /bundle, and supports
common-overlap or full-range export windows.

Update the batch wrapper, Python validator, RGBD viewer, and
documentation so copy-layout MCAP files are treated as a first-class
format rather than invalid bundled output.

Validation:
- cmake --build build --target zed_svo_to_mcap -j4
- uv run python -m py_compile scripts/zed_batch_svo_to_mcap.py scripts/mcap_bundle_validator.py scripts/mcap_rgbd_viewer.py
- build/bin/zed_svo_to_mcap --segment-dir /workspaces/data/kindergarten/bar/2026-03-18T11-59-41 --output /tmp/bar_11-59-41_copy_common.mcap --encoder-device nvidia --depth-mode neural_plus --depth-size optimal --bundle-policy copy --copy-range common
- uv run python scripts/mcap_bundle_validator.py /tmp/bar_11-59-41_copy_common.mcap
- uv run --extra viewer python scripts/mcap_rgbd_viewer.py /tmp/bar_11-59-41_copy_common.mcap --summary-only
2026-03-24 18:44:44 +08:00
crosstyan 6d50b29eff feat(mcap): add Python layout validator
Document the bundled and single-camera MCAP topic contract in docs/mcap_layout.md and link it from the README.

Add scripts/mcap_bundle_validator.py to summarize and validate both bundled /bundle-based MCAPs and single-camera /camera/* MCAPs from Python.

Validate bundled files against bundle-member presence counts and single-camera files against topic/schema expectations plus video/depth/calibration count rules.
2026-03-24 16:03:16 +08:00
crosstyan 807a73b480 feat(zed): recover corrupted frame gaps in MCAP export
Make ZED MCAP export skip corrupted frame runs until recovery and

treat unreadable tail frames as end-of-stream instead of hard

failing conversion.

Update bundled nearest-mode export to emit partial bundles during

corruption gaps, extend BundleManifest with explicit member status

and skipped-frame counts, and only write payload messages for

present cameras.

Tighten batch probing so bundled MCAP validation checks /bundle

coverage and per-camera message counts, and improve failure

excerpts to include stderr tail output.

Also add a local cppzmq CMake fallback, refresh the multi-record

tester for the new bundle schema, and document the mixed NVENC

limitations in the README.
2026-03-24 16:03:16 +08:00
crosstyan a0b9c95d5b feat(zed): improve MCAP export batching and defaults
Default ZED MCAP export to neural_plus depth across the CLI and Python wrappers, and add tail-frame handling plus better corrupted-frame diagnostics in zed_svo_to_mcap.

Add mixed hardware/software worker pools to the batch MCAP wrapper, replace tqdm with progress-table on TTYs, keep text event logging and heartbeats for non-TTY runs, and document the NVENC session-limit rationale for mixed mode in the README.

Also refresh Python dependencies for the batch tooling and move the OpenSSL lookup in CMake so the local workspace build remains compatible with the vendored cnats setup.
2026-03-24 16:02:15 +08:00
crosstyan 7808b89a03 feat: support bundled frame-window offsets in zed_svo_to_mcap
Implement bundled multi-camera --start-frame semantics as synced-group\nindices from the common synced start, inclusive with --end-frame.\n\nUpdate zed_svo_to_mcap to skip synced groups before writing, keep\nsingle-camera raw SVO frame semantics unchanged, and adjust exact\nprogress totals for selected bundled windows.\n\nReplace the expensive bundled exact pre-count path with approximate\ntime-window progress when --end-frame is not set, and update the\nshared TTY progress bar helper to support fraction-based rendering.\n\nExpose --start-frame in the batch MCAP wrapper and document the\nbundled frame-window semantics and approximate progress behavior in\nthe README.
2026-03-24 16:01:46 +08:00
crosstyan 1369f5235d feat: add shared TTY progress bars for ZED offline tools
Extract the tqdm-like stderr progress bar into a shared helper and reuse it across zed_svo_to_mp4, zed_svo_grid_to_mp4, and zed_svo_to_mcap.

For zed_svo_to_mcap, single-source exports now report exact frame totals and bundled multi-camera exports report exact synced-group totals on TTY. When bundled mode runs without --end-frame, the tool performs a counting pass first so the progress total remains exact instead of estimated.

Also document the bundled MCAP progress behavior in the README and record the current third_party dependency state in third_party/README. That note now makes it explicit that CLI11 and proxy are the active submodules, while tomlplusplus and mcap are vendored source drops, and adds a TODO to revisit converting mcap into a submodule later.

Verified with the Debug build during implementation, including single-camera and bundled zed_svo_to_mcap runs that rendered clean progress output to a TTY.
2026-03-24 16:01:46 +08:00
crosstyan 1ef318bde4 build(streamer): make ZED SDK optional 2026-03-20 17:34:30 +08:00
crosstyan 1691274e85 feat: add batch MCAP export tooling for ZED segments
Add a Python batch wrapper around zed_svo_to_mcap for multi-camera
segment exports. The new script supports dataset discovery, repeated
segment-dir inputs, CSV-driven ordering, skip/probe/report flows, dry-run,
and CUDA environment passthrough so kindergarten-style datasets can be
converted into one bundled MCAP per segment.

Extend zed_svo_to_mcap so bundled multi-camera mode accepts --end-frame
with synced-group semantics. In this mode the value is interpreted as the
last emitted synced frame-group index from the common synced start, while
--start-frame remains unsupported.

Vendor a minimal pose-config TOML and a sample segments CSV into this repo
so the MCAP workflow is self-contained. Update the README to document the
batch MCAP flow, use portable placeholders instead of machine-specific
absolute paths, and describe the expected dataset layout explicitly.
2026-03-20 17:25:29 +08:00
crosstyan 0cf0b2566d Add ffprobe checks for grid batch outputs 2026-03-20 17:25:29 +08:00
crosstyan c2e5cc561f Add batch wrapper for ZED grid MP4 conversion 2026-03-20 17:25:29 +08:00
crosstyan 2671ac7ba9 Add synced ZED SVO grid exporter 2026-03-20 17:25:29 +08:00
crosstyan ae19b881b0 feat: add mcap recorder control and cnats providers
Register an MCAP recorder service on the streamer control subjects, reuse the shared recording request and status model, and expose the zed recording preview/conversion helper.

This also replaces the temporary cnats boolean with the explicit CVMMAP_CNATS_PROVIDER modes and documents the supported system and workspace build paths.
2026-03-18 11:53:04 +08:00
crosstyan ee53d1958e refactor(streamer): consume control and body over NATS 2026-03-16 17:07:45 +08:00
crosstyan eed6695758 test(suites): run acceptance and fault flows on dummy backend 2026-03-06 08:50:32 +08:00
crosstyan d5df65927b fix(standalone): decouple evidence paths and harden gate scripts
Switch acceptance/fault/release scripts to project-local .sisyphus evidence roots and remove parent-repo path assumptions.

Also harden deterministic behavior with run-id-derived port allocation and tuned fault thresholds so release gate pass and injected-failure flows remain stable in standalone execution.
2026-03-05 23:52:37 +08:00
crosstyan 56e874ab6d feat(downstream): add cvmmap downstream runtime implementation
This commit introduces the full downstream runtime implementation needed to ingest, transform, and publish streams.

It preserves the original upstream request boundary by packaging the entire cvmmap-streamer module (build config, public API, protocol and IPC glue, and simulator/tester entrypoints) in one coherent core unit.

Keeping this group isolated enables reviewers to validate runtime behavior and correctness without mixing test evidence or process documentation changes.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-03-05 20:31:58 +08:00