feat(cli): add depth verify/refine outputs and tests
- Retrieve depth + confidence measures from SVOReader when depth enabled - Compute depth residual metrics and attach to output JSON - Optionally write per-corner residual CSV via --report-csv - Post-process refinement: optimize final pose and report pre/post metrics - Add unit tests for depth verification and refinement modules - Add basedpyright dev dependency for diagnostics
This commit is contained in:
@@ -0,0 +1,81 @@
|
||||
## Task 4 Complete: CLI Flags Added
|
||||
|
||||
Successfully added all required CLI flags:
|
||||
- --verify-depth / --no-verify-depth
|
||||
- --refine-depth / --no-refine-depth
|
||||
- --depth-mode [NEURAL|ULTRA|PERFORMANCE|NONE]
|
||||
- --depth-confidence-threshold
|
||||
- --report-csv
|
||||
|
||||
All flags appear in --help output.
|
||||
|
||||
## Next Steps
|
||||
|
||||
Tasks 5 and 6 require integrating depth verification and refinement into the main workflow:
|
||||
- Task 5: Add depth verification logic to main() function
|
||||
- Task 6: Add depth refinement logic to main() function
|
||||
- Task 7: Create unit tests for depth modules
|
||||
|
||||
The integration needs to:
|
||||
1. Parse depth_mode string to sl.DEPTH_MODE enum
|
||||
2. Pass depth_mode to SVOReader when verify_depth or refine_depth is enabled
|
||||
3. After computing extrinsics, run depth verification
|
||||
4. If refine_depth is enabled, run optimization and update results
|
||||
5. Add verification stats to output JSON
|
||||
6. Optionally write CSV report
|
||||
|
||||
## Task 5 Complete: Depth Verification Integration
|
||||
|
||||
- **Refactored `depth_verify.py`**:
|
||||
- Implemented `compute_marker_corner_residuals` to return detailed (marker_id, corner_idx, residual) tuples.
|
||||
- Added confidence map filtering logic: filters out points where ZED confidence > threshold (lower is better in ZED SDK).
|
||||
- Fixed `compute_depth_residual` to handle projection errors and bounds checking robustly.
|
||||
- Return type `DepthVerificationResult` now includes list of all residuals for CSV reporting.
|
||||
|
||||
- **Updated `svo_sync.py`**:
|
||||
- Added `confidence_map` to `FrameData` dataclass.
|
||||
- Updated `SVOReader` to retrieve both depth and confidence measures when depth mode is enabled.
|
||||
|
||||
- **Integrated into `calibrate_extrinsics.py`**:
|
||||
- Stores a "verification sample" (last valid frame with depth) during the processing loop.
|
||||
- Post-process verification: After `T_mean` is computed, runs verification using the stored sample and the final consensus pose.
|
||||
- Adds `depth_verify` block to the output JSON with RMSE, mean absolute error, and validity counts.
|
||||
- Writes detailed CSV report if `--report-csv` is provided.
|
||||
|
||||
- **Verification**:
|
||||
- `uv run pytest -q` passes (10 tests).
|
||||
- LSP checks pass for modified files.
|
||||
### Type Fixes in aruco/svo_sync.py
|
||||
- Updated type annotations to use modern Python 3.12 syntax:
|
||||
- Replaced `List`, `Dict`, `Optional` with `list`, `dict`, and `| None`.
|
||||
- Added missing type arguments to generic classes (e.g., `dict[str, Any]`).
|
||||
- Added explicit class attribute annotations for `SVOReader`.
|
||||
- Verified that `lsp_diagnostics` no longer reports `reportMissingTypeArgument` or `reportDeprecated` for this file.
|
||||
- Confirmed that `uv run pytest` passes after changes.
|
||||
|
||||
## Depth Refinement Integration
|
||||
- **Pattern**: Post-process refinement is superior to per-frame refinement for extrinsic calibration.
|
||||
- **Why**: Per-frame refinement is noisy and computationally expensive. Refining the robust mean pose against a high-quality frame (or average of frames) yields more stable results.
|
||||
- **Implementation**: Store a representative frame (e.g., last valid frame with depth) during the loop, then run refinement once on the final aggregated pose.
|
||||
- **Verification**: Always verify *before* and *after* refinement to quantify improvement.
|
||||
- **Metrics**: RMSE of depth residuals, delta rotation/translation.
|
||||
- **Guardrails**: Skip refinement if insufficient valid depth points are available (e.g., < 4 points).
|
||||
|
||||
## Task 6 Complete: Depth Refinement Integration
|
||||
|
||||
- **Refinement Logic**:
|
||||
- Implemented post-process refinement: verifies initial pose, refines using L-BFGS-B optimization, then verifies refined pose.
|
||||
- Updates JSON output with `refine_depth` stats and `depth_verify_post` metrics.
|
||||
- Calculates and reports RMSE improvement.
|
||||
- Uses refined pose for final CSV report if enabled.
|
||||
|
||||
- **Code Quality**:
|
||||
- Cleaned up loop logic to remove per-frame refinement attempts (inefficient).
|
||||
- Added proper type hints (`List`, `Dict`, `Any`, `Optional`, `Tuple`) to satisfy LSP.
|
||||
- Removed agent memo comments.
|
||||
- Verified with `pytest` (18 passed) and `calibrate_extrinsics.py --help`.
|
||||
|
||||
- **Key Implementation Details**:
|
||||
- Refinement only runs if sufficient valid depth points (>4) exist.
|
||||
- Refined pose (`T_refined`) replaces the original RANSAC mean pose in the output JSON if refinement is successful.
|
||||
- Original RANSAC stats remain in `stats` field, while refinement deltas are in `refine_depth`.
|
||||
@@ -531,7 +531,7 @@ Critical Path: Task 1 → Task 2 → Task 4 → Task 5 → Task 6
|
||||
|
||||
---
|
||||
|
||||
- [ ] 6. Integrate refinement into CLI workflow
|
||||
- [x] 6. Integrate refinement into CLI workflow
|
||||
|
||||
**What to do**:
|
||||
- Modify `py_workspace/calibrate_extrinsics.py`
|
||||
@@ -602,7 +602,7 @@ Critical Path: Task 1 → Task 2 → Task 4 → Task 5 → Task 6
|
||||
|
||||
---
|
||||
|
||||
- [ ] 7. Add unit tests for depth modules
|
||||
- [x] 7. Add unit tests for depth modules
|
||||
|
||||
**What to do**:
|
||||
- Create `py_workspace/tests/test_depth_verify.py`
|
||||
|
||||
Reference in New Issue
Block a user