crosstyan/zed-playground
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(calibrate): integrate multi-frame depth pooling with --depth-pool-size flag

Browse Source
This commit is contained in:
crosstyan
2026-02-07 08:10:01 +00:00
parent dad1f2a69f
commit 4fc8de4bdc
6 changed files with 774 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files
py_workspace/.sisyphus/notepads/multi-frame-depth-pooling/learnings.md
+41
View File
@@ -0,0 +1,41 @@
## Depth Pooling Implementation
- Implemented `pool_depth_maps` in `aruco/depth_pool.py`.
- Uses `np.nanmedian` for robust per-pixel depth pooling.
- Supports confidence gating (lower is better) and `min_valid_count` threshold.
- Handles N=1 case by returning a masked copy.
- Vectorized implementation using `np.stack` and boolean masking for performance.
## 2026-02-07: Depth Pooling Test Implementation
- Implemented comprehensive unit tests for `pool_depth_maps` in `tests/test_depth_pool.py`.
- Verified handling of:
- Empty input and shape mismatches (ValueError).
- Single map behavior (masked copy, min_valid_count check).
- Median pooling logic with multiple maps.
- Invalid depth values (<=0, non-finite).
- Confidence gating (ZED semantics: lower is better).
- min_valid_count enforcement across multiple frames.
- Type checking with basedpyright confirmed clean (after fixing unused call results and Optional handling in tests).
## Task 4: CLI Option Wiring
- Added `--depth-pool-size` (1-10, default 1) to `calibrate_extrinsics.py`.
- Wired the option through `main` to `apply_depth_verify_refine_postprocess`.
- Maintained backward compatibility by defaulting to 1.
- Extended `verification_frames` to store a list of top-N frames per camera, sorted by score descending.
- Maintained backward compatibility by using the first frame in the list for current verification and benchmark logic.
- Added `depth_pool_size` parameter to `main` and passed it to `apply_depth_verify_refine_postprocess`.
## 2026-02-07: Multi-Frame Depth Pooling Integration
- Integrated `pool_depth_maps` into `calibrate_extrinsics.py`.
- Added `--depth-pool-size` CLI option (default 1).
- Implemented fallback logic: if pooled depth has < 50% valid points compared to best single frame, fallback to single frame.
- Added `depth_pool` metadata to JSON output.
- Verified N=1 equivalence with regression test `tests/test_depth_pool_integration.py`.
- Verified E2E smoke test:
- Pool=1 vs Pool=5 showed mixed results on small sample (20 frames):
- Camera 41831756: -0.0004m (Improved)
- Camera 44289123: +0.0004m (Worse)
- Camera 44435674: -0.0003m (Improved)
- Camera 46195029: +0.0036m (Worse)
- This variance is expected on small samples; pooling is intended for stability over larger datasets.
- Runtime warning `All-NaN slice encountered` observed in `nanmedian` when some pixels are invalid in all frames; this is handled by `nanmedian` returning NaN, which is correct behavior for us.