Integrate a native Jetson Multimedia API encoder path and keep the existing encoded access-unit contract for RTMP, RTP, and recording consumers.
This adds conditional Jetson MMAPI detection in CMake, builds the required NVIDIA sample common classes into a dedicated support library, and compiles the new backend only when the platform dependencies are present.
The runtime selector now lets encoder.backend=auto prefer Jetson MM for NVIDIA hardware requests while keeping encoder.backend=ffmpeg as an explicit FFmpeg path. Device selection semantics are updated so nvidia requires the Jetson backend, auto can fall back to FFmpeg software, and software remains FFmpeg-only.
The Jetson backend converts supported raw inputs through swscale, feeds NvVideoEncoder in YUV420M, emits Annex-B access units, and harvests decoder configuration from a warmup keyframe so downstream packetizers keep their existing contract.
This also splits FFmpeg encoder option handling into a shared header, updates runtime config/help text and tester defaults, and refreshes compatibility/caveat documentation to reflect the new selection behavior.
Build-tree runtime RPATH handling is tightened so GCC 15 builds keep the matching libstdc++ visible locally. Verification covered GCC 15 builds, RTP H.264/H.265 tester runs, RTMP H.264 stub publish, local live SRS smoke with cvmmap://zed1, and remote execution on 192.168.2.155 using the deployed bundle lib directory for the GCC 15 runtime.
Drop the offline ZED helper implementations that were moved into zed-offline-tools.\n\nThis removes the standalone conversion binaries, batch/index/inspection scripts, related configs and tests, and the tool-specific support code that no longer belongs in cvmmap-streamer.\n\nThe build files and docs are updated to point at the standalone repo while keeping the streamer runtime surface intact.
Track libavformat RTMP session initialization state so teardown only writes a trailer after the muxer header succeeds. This avoids calling av_write_trailer() on partially initialized sessions when avio_open2() fails with Connection refused.
Add a fault-suite regression for libavformat RTMP connection refusal and update the summary helper to compute all_pass from the actual manifest size instead of a hardcoded seven-row expectation.
Verified by rebuilding cvmmap_streamer and rtmp_output_tester, reproducing the refused-connection path without a crash, and running ./scripts/fault_suite.sh successfully (8/8).
Replace the deprecated <nats.h> include with <nats/nats.h> in the streamer request/reply service implementation to avoid the cnats deprecation warning during builds.
Introduce a dedicated streamer-side recording control plane instead of sharing the producer recorder API.
- register streamer-owned recorder endpoints as a NATS micro service
- add explicit MP4 and MCAP recorder control protobufs and subject helpers
- wire recorder lifecycle handling into the pipeline runtime
- add MP4 writer and depth-alignment support files used by the new recording flow
Refactor runtime option parsing to use structured CLI11 option bindings,
validators, and grouped help output instead of the previous raw-string
collection and manual field-by-field parsing.
This introduces reusable canonicalization helpers, endpoint validation,
non-empty validators, and boolean-assignment checks so invalid CLI input is
rejected earlier and help/default rendering comes directly from the option
definitions.
The static help text is updated to advertise the new keep-stream-on-reset
flag added by the streaming work.
Preserve RTP/RTMP session continuity across upstream stream_reset events by
forcing a keyframe on restart, remapping live timestamps, and keeping live
outputs open when the runtime requests reset continuity.
Disable idle auto-exit by default by changing ingest_idle_timeout_ms to 0,
removing validation that rejected 0, and only enforcing idle shutdown when a
positive timeout is configured in pipeline and ingest loops.
Also suppress libavformat FLV trailer header backfill attempts on RTMP sockets
and update the RTP output tester for the newer publisher create signature.
Docs are updated to state that 0 disables the idle timeout.
Move the legacy /camera/* contract into its own reference page so the main MCAP layout doc can stay focused on current bundled and copy-layout behavior.
Document the compatibility model explicitly: legacy single-camera is operationally equivalent to one-camera copy when the effective camera label is treated as the literal `camera`.
Update the mcap_rgbd_example helper and recipe docs to accept legacy /camera/* inputs under that compatibility rule instead of rejecting them.
Namespace single-source zed_svo_to_mcap outputs with a derived camera label so one-camera exports use the same copy-layout topic shape as current multi-camera copy files.
Use the per-camera frame and pose reference identifiers for those single-source outputs, document copy as the current one-camera wire shape, and clarify that legacy /camera/* files are retained only as a legacy contract.
Add a small mcap_rgbd_example helper plus mcap_recipes guide for summarizing bundled/copy MCAPs and exporting one RGB/depth sample, and update the validator and viewer wording/behavior to match the bundled-vs-copy semantics.
Add an opt-in multi-camera copy layout for zed_svo_to_mcap.
copy mode preserves each camera's original timestamps and cadence,
writes namespaced /zedN topics without /bundle, and supports
common-overlap or full-range export windows.
Update the batch wrapper, Python validator, RGBD viewer, and
documentation so copy-layout MCAP files are treated as a first-class
format rather than invalid bundled output.
Validation:
- cmake --build build --target zed_svo_to_mcap -j4
- uv run python -m py_compile scripts/zed_batch_svo_to_mcap.py scripts/mcap_bundle_validator.py scripts/mcap_rgbd_viewer.py
- build/bin/zed_svo_to_mcap --segment-dir /workspaces/data/kindergarten/bar/2026-03-18T11-59-41 --output /tmp/bar_11-59-41_copy_common.mcap --encoder-device nvidia --depth-mode neural_plus --depth-size optimal --bundle-policy copy --copy-range common
- uv run python scripts/mcap_bundle_validator.py /tmp/bar_11-59-41_copy_common.mcap
- uv run --extra viewer python scripts/mcap_rgbd_viewer.py /tmp/bar_11-59-41_copy_common.mcap --summary-only
Document the bundled and single-camera MCAP topic contract in docs/mcap_layout.md and link it from the README.
Add scripts/mcap_bundle_validator.py to summarize and validate both bundled /bundle-based MCAPs and single-camera /camera/* MCAPs from Python.
Validate bundled files against bundle-member presence counts and single-camera files against topic/schema expectations plus video/depth/calibration count rules.
Make ZED MCAP export skip corrupted frame runs until recovery and
treat unreadable tail frames as end-of-stream instead of hard
failing conversion.
Update bundled nearest-mode export to emit partial bundles during
corruption gaps, extend BundleManifest with explicit member status
and skipped-frame counts, and only write payload messages for
present cameras.
Tighten batch probing so bundled MCAP validation checks /bundle
coverage and per-camera message counts, and improve failure
excerpts to include stderr tail output.
Also add a local cppzmq CMake fallback, refresh the multi-record
tester for the new bundle schema, and document the mixed NVENC
limitations in the README.
Add a new mcap_video_bounds helper binary plus a zed_segment_time_index.py CLI that builds and queries an embedded DuckDB index for bundled ZED segment recordings.
The index stores segment folders, MCAP paths, video time bounds, durations, camera labels, and dataset metadata, and reuses the existing recursive multi-camera segment discovery logic so nested kindergarten layouts are indexed correctly.
Infer a dataset default timezone from folder names versus MCAP timestamps, and make point queries precision-aware so second-level folder timestamps like 2026-03-18T12-00-23 resolve to the matching segment instead of missing due to subsecond start offsets.
Verification:
- uv add 'duckdb>=1.0'
- cmake --build build --target mcap_video_bounds
- uv run python -m unittest tests.test_zed_segment_time_index
- uv run python scripts/zed_segment_time_index.py build /workspaces/data/kindergarten --jobs 8
- uv run python scripts/zed_segment_time_index.py query /workspaces/data/kindergarten --at 2026-03-18T12-00-23
Default ZED MCAP export to neural_plus depth across the CLI and Python wrappers, and add tail-frame handling plus better corrupted-frame diagnostics in zed_svo_to_mcap.
Add mixed hardware/software worker pools to the batch MCAP wrapper, replace tqdm with progress-table on TTYs, keep text event logging and heartbeats for non-TTY runs, and document the NVENC session-limit rationale for mixed mode in the README.
Also refresh Python dependencies for the batch tooling and move the OpenSSL lookup in CMake so the local workspace build remains compatible with the vendored cnats setup.
Restrict ZED depth export to neural modes and default offline MCAP conversion to the SDK-selected optimal depth size instead of camera resolution.
Add a configurable --depth-size option, propagate the actual returned depth dimensions into DepthMap metadata, and emit a dedicated depth calibration topic when depth resolution differs from video.
Update the batch and recording helper scripts to use the new neural-only depth mode surface and pass through depth sizing.
Verification:
- cmake --build build --target zed_svo_to_mcap mcap_multi_record_tester mcap_pose_record_tester -j4
- build/bin/mcap_multi_record_tester
- build/bin/mcap_pose_record_tester
- build/bin/zed_svo_to_mcap --input /workspaces/data/kindergarten/jump/experiment/2/2026-03-18T11-27-15/2026-03-18T11-27-15_zed4.svo2 --output /tmp/zed4_neural_optimal_test.mcap --codec h264 --encoder-device software --mcap-compression zstd --depth-mode neural --depth-size optimal --start-frame 0 --end-frame 9
Implement bundled multi-camera --start-frame semantics as synced-group\nindices from the common synced start, inclusive with --end-frame.\n\nUpdate zed_svo_to_mcap to skip synced groups before writing, keep\nsingle-camera raw SVO frame semantics unchanged, and adjust exact\nprogress totals for selected bundled windows.\n\nReplace the expensive bundled exact pre-count path with approximate\ntime-window progress when --end-frame is not set, and update the\nshared TTY progress bar helper to support fraction-based rendering.\n\nExpose --start-frame in the batch MCAP wrapper and document the\nbundled frame-window semantics and approximate progress behavior in\nthe README.
Extract the tqdm-like stderr progress bar into a shared helper and reuse it across zed_svo_to_mp4, zed_svo_grid_to_mp4, and zed_svo_to_mcap.
For zed_svo_to_mcap, single-source exports now report exact frame totals and bundled multi-camera exports report exact synced-group totals on TTY. When bundled mode runs without --end-frame, the tool performs a counting pass first so the progress total remains exact instead of estimated.
Also document the bundled MCAP progress behavior in the README and record the current third_party dependency state in third_party/README. That note now makes it explicit that CLI11 and proxy are the active submodules, while tomlplusplus and mcap are vendored source drops, and adds a TODO to revisit converting mcap into a submodule later.
Verified with the Debug build during implementation, including single-camera and bundled zed_svo_to_mcap runs that rendered clean progress output to a TTY.
Replace the vendored MCAP Zstd chunk writer's ZSTD_compress2-based path with ZSTD_compressCCtx and keep the selected compression level on the wrapper state.
The previous implementation crashed in Debug and real zed_svo_to_mcap runs on this machine when the first compressed chunk was flushed with --mcap-compression zstd. The same exports succeeded with none and lz4, which narrowed the failure to the shared vendored MCAP Zstd path rather than exporter sync logic.
Also extend mcap_multi_record_tester to accept a compression argument so the Zstd path can be exercised directly during regression testing.
Verified with:
- ./build-debug/bin/mcap_multi_record_tester /tmp/mcap_multi_zstd_test.mcap zstd
- ./build-debug/bin/zed_svo_to_mcap --input jump/experiment/2/2026-03-18T11-27-15/2026-03-18T11-27-15_zed1.svo2 --output /tmp/zed1_single_zstd_fixed.mcap --mcap-compression zstd --end-frame 1
- ./build-debug/bin/zed_svo_to_mcap --segment-dir jump/experiment/2/2026-03-18T11-27-15 --output /tmp/multi_zstd_fixed.mcap --mcap-compression zstd --end-frame 1
Add a Python batch wrapper around zed_svo_to_mcap for multi-camera
segment exports. The new script supports dataset discovery, repeated
segment-dir inputs, CSV-driven ordering, skip/probe/report flows, dry-run,
and CUDA environment passthrough so kindergarten-style datasets can be
converted into one bundled MCAP per segment.
Extend zed_svo_to_mcap so bundled multi-camera mode accepts --end-frame
with synced-group semantics. In this mode the value is interpreted as the
last emitted synced frame-group index from the common synced start, while
--start-frame remains unsupported.
Vendor a minimal pose-config TOML and a sample segments CSV into this repo
so the MCAP workflow is self-contained. Update the README to document the
batch MCAP flow, use portable placeholders instead of machine-specific
absolute paths, and describe the expected dataset layout explicitly.
Extend zed_svo_to_mcap to export multiple SVO inputs or a kindergarten segment directory into a single MCAP with per-camera namespaced topics.
Add a MultiMcapRecordSink so one writer can manage independent video, depth, calibration, pose, and body channels for multiple cameras while reusing the existing protobuf schemas and encoded access-unit flow.
Implement strict timestamp synchronization for bundled exports by computing the common time window, seeking each SVO to the shared start, dropping unsynced frames, and only emitting groups that fall within the configured tolerance.
Load per-camera positional tracking settings from a cv-mmap style TOML pose config, preserve single-camera behavior, and add graceful SIGINT or SIGTERM shutdown so interrupted exports flush encoders and finalize readable MCAP files.
Add mcap_multi_record_tester coverage and switch zed_svo_to_mcap defaults to H.265 plus QUALITY depth mode.
Register an MCAP recorder service on the streamer control subjects, reuse the shared recording request and status model, and expose the zed recording preview/conversion helper.
This also replaces the temporary cnats boolean with the explicit CVMMAP_CNATS_PROVIDER modes and documents the supported system and workspace build paths.
crosstyan