chore: checkpoint ground-plane calibration refinement work

2026-02-09 10:02:48 +00:00
parent 915c7973d1
commit 511994e3a8
19 changed files with 4601 additions and 41 deletions
@@ -0,0 +1,11 @@
+
+## Depth Data Saving Integration
+- Integrated `--save-depth` flag into `calibrate_extrinsics.py`.
+- Uses `aruco.depth_save.save_depth_data` to persist HDF5 files.
+- Captures:
+  - Intrinsics and resolution.
+  - Pooled depth and confidence maps.
+  - Pool metadata (RMSE comparison, fallback reasons).
+  - Raw candidate frames (depth, confidence, score, frame index).
+- Logic is guarded: only runs if `verify_depth` or `refine_depth` is enabled.
+- Added integration test `tests/test_depth_save_integration.py` using mocks to verify data flow without writing actual HDF5 files during testing.
@@ -5,3 +5,11 @@
 ## [2026-02-09] Final Integration
 - No regressions found in the full test suite.
 - basedpyright warnings are mostly related to missing stubs for third-party libraries (h5py, open3d, plotly) and deprecated type hints in older Python patterns, which are acceptable given the project's current state and consistency with existing code.
+
+## Working Tree Cleanup
+- Restored deleted legacy plan files in .sisyphus/plans/
+- Restored unintended modifications to apply_calibration_to_fusion_config.py
+- Restored unintended modifications to ../zed_settings/inside_shared_manual.json
+- Verified that implementation files (aruco/ground_plane.py, calibrate_extrinsics.py, refine_ground_plane.py, tests/test_ground_plane.py) remain intact.
+## Issues Encountered
+- Initial implementation placed `ground_refine` directly under camera nodes, which could break schema-strict consumers like `calibrate_extrinsics.py` output expectations.
@@ -37,3 +37,6 @@
 - Clarified the "Consensus-Relative Correction" strategy vs. absolute alignment.
 - Added explicit tuning guidance for `stride`, `ransac-dist-thresh`, and `max-rotation-deg` based on implementation constraints.

+## Schema Compatibility Fix
+- Moved per-camera ground refinement diagnostics to `_meta.ground_refined.per_camera` to maintain compatibility with consumers expecting only `pose` in camera nodes.
+- Preserved `<camera_sn>.pose` contract.
@@ -0,0 +1,18 @@
+# Decisions from Task 5 (Fix): Per-Camera Correction
+
+## Architecture
+- **Per-Camera Correction Logic**: Instead of computing a consensus plane and deriving a single global correction, the system now:
+  1. Detects a floor plane for each camera.
+  2. Computes a correction transform for *that specific camera* to align its observed floor to `target_y`.
+  3. Applies the correction to that camera's extrinsics.
+  4. Skips cameras where no plane is detected.
+
+## Metrics
+- **Detailed Tracking**: `GroundPlaneMetrics` now includes:
+  - `camera_corrections`: Map of serial -> correction matrix.
+  - `skipped_cameras`: List of serials that were skipped.
+  - `rotation_deg` / `translation_m`: Max values across all applied corrections (for summary).
+
+## Rationale
+- **Robustness**: This approach allows cameras with good floor visibility to be corrected even if others fail. It also handles cases where cameras might have different initial misalignments (e.g., one tilted up, one tilted down).
+- **Independence**: Each camera is corrected based on its own data, reducing dependency on a potentially noisy consensus if some cameras are outliers.
@@ -0,0 +1,12 @@
+# Learnings from Task 5 (Fix): Per-Camera Correction
+
+## Patterns
+- **Per-Camera vs Global Correction**: The initial implementation applied a single global correction based on a consensus plane. The requirement was for per-camera correction. This was fixed by iterating through each camera's detected plane and computing a specific correction for that camera to align it to the target Y.
+- **Metrics Granularity**: `GroundPlaneMetrics` was updated to track per-camera corrections (`camera_corrections`) and skipped cameras (`skipped_cameras`), providing better visibility into the process.
+
+## Testing
+- **Partial Success Scenarios**: Added a test case `test_refine_ground_from_depth_partial_success` where one camera has a valid plane and another doesn't. This verified that the valid camera gets corrected while the invalid one is skipped and tracked in metrics.
+- **Verification of Per-Camera Logic**: The test explicitly checks that `metrics.camera_corrections` contains the expected cameras and that the applied transform is correct for the specific camera.
+
+## Issues
+- **Ambiguity in "Relative to Consensus"**: The plan mentioned "relative to consensus", which could be interpreted as aligning cameras to the consensus plane. However, "per-camera refinement" usually implies correcting each camera's error independently. I chose to align each camera's observed plane to the target Y directly, which satisfies the goal of placing the floor at the correct height for all cameras, effectively making them consistent with the target (and thus each other).