crosstyan
e8a8156e0b
- Ensure default extrinsics calibration runs with depth mode NONE - Depth is only enabled when --verify-depth or --refine-depth is set - Document opt-in behavior in notepad
83 lines
4.5 KiB
Markdown
83 lines
4.5 KiB
Markdown
## Task 4 Complete: CLI Flags Added
|
|
|
|
Successfully added all required CLI flags:
|
|
- --verify-depth / --no-verify-depth
|
|
- --refine-depth / --no-refine-depth
|
|
- --depth-mode [NEURAL|ULTRA|PERFORMANCE|NONE]
|
|
- --depth-confidence-threshold
|
|
- --report-csv
|
|
|
|
All flags appear in --help output.
|
|
|
|
## Next Steps
|
|
|
|
Tasks 5 and 6 require integrating depth verification and refinement into the main workflow:
|
|
- Task 5: Add depth verification logic to main() function
|
|
- Task 6: Add depth refinement logic to main() function
|
|
- Task 7: Create unit tests for depth modules
|
|
|
|
The integration needs to:
|
|
1. Parse depth_mode string to sl.DEPTH_MODE enum
|
|
2. Pass depth_mode to SVOReader when verify_depth or refine_depth is enabled
|
|
3. After computing extrinsics, run depth verification
|
|
4. If refine_depth is enabled, run optimization and update results
|
|
5. Add verification stats to output JSON
|
|
6. Optionally write CSV report
|
|
|
|
## Task 5 Complete: Depth Verification Integration
|
|
|
|
- **Refactored `depth_verify.py`**:
|
|
- Implemented `compute_marker_corner_residuals` to return detailed (marker_id, corner_idx, residual) tuples.
|
|
- Added confidence map filtering logic: filters out points where ZED confidence > threshold (lower is better in ZED SDK).
|
|
- Fixed `compute_depth_residual` to handle projection errors and bounds checking robustly.
|
|
- Return type `DepthVerificationResult` now includes list of all residuals for CSV reporting.
|
|
|
|
- **Updated `svo_sync.py`**:
|
|
- Added `confidence_map` to `FrameData` dataclass.
|
|
- Updated `SVOReader` to retrieve both depth and confidence measures when depth mode is enabled.
|
|
|
|
- **Integrated into `calibrate_extrinsics.py`**:
|
|
- Stores a "verification sample" (last valid frame with depth) during the processing loop.
|
|
- Post-process verification: After `T_mean` is computed, runs verification using the stored sample and the final consensus pose.
|
|
- Adds `depth_verify` block to the output JSON with RMSE, mean absolute error, and validity counts.
|
|
- Writes detailed CSV report if `--report-csv` is provided.
|
|
|
|
- **Verification**:
|
|
- `uv run pytest -q` passes (10 tests).
|
|
- LSP checks pass for modified files.
|
|
### Type Fixes in aruco/svo_sync.py
|
|
- Updated type annotations to use modern Python 3.12 syntax:
|
|
- Replaced `List`, `Dict`, `Optional` with `list`, `dict`, and `| None`.
|
|
- Added missing type arguments to generic classes (e.g., `dict[str, Any]`).
|
|
- Added explicit class attribute annotations for `SVOReader`.
|
|
- Verified that `lsp_diagnostics` no longer reports `reportMissingTypeArgument` or `reportDeprecated` for this file.
|
|
- Confirmed that `uv run pytest` passes after changes.
|
|
|
|
## Depth Refinement Integration
|
|
- **Pattern**: Post-process refinement is superior to per-frame refinement for extrinsic calibration.
|
|
- **Why**: Per-frame refinement is noisy and computationally expensive. Refining the robust mean pose against a high-quality frame (or average of frames) yields more stable results.
|
|
- **Implementation**: Store a representative frame (e.g., last valid frame with depth) during the loop, then run refinement once on the final aggregated pose.
|
|
- **Verification**: Always verify *before* and *after* refinement to quantify improvement.
|
|
- **Metrics**: RMSE of depth residuals, delta rotation/translation.
|
|
- **Guardrails**: Skip refinement if insufficient valid depth points are available (e.g., < 4 points).
|
|
|
|
## Task 6 Complete: Depth Refinement Integration
|
|
|
|
- **Refinement Logic**:
|
|
- Implemented post-process refinement: verifies initial pose, refines using L-BFGS-B optimization, then verifies refined pose.
|
|
- Updates JSON output with `refine_depth` stats and `depth_verify_post` metrics.
|
|
- Calculates and reports RMSE improvement.
|
|
- Uses refined pose for final CSV report if enabled.
|
|
|
|
- **Code Quality**:
|
|
- Cleaned up loop logic to remove per-frame refinement attempts (inefficient).
|
|
- Added proper type hints (`List`, `Dict`, `Any`, `Optional`, `Tuple`) to satisfy LSP.
|
|
- Removed agent memo comments.
|
|
- Verified with `pytest` (18 passed) and `calibrate_extrinsics.py --help`.
|
|
|
|
- **Key Implementation Details**:
|
|
- Refinement only runs if sufficient valid depth points (>4) exist.
|
|
- Refined pose (`T_refined`) replaces the original RANSAC mean pose in the output JSON if refinement is successful.
|
|
- Original RANSAC stats remain in `stats` field, while refinement deltas are in `refine_depth`.
|
|
- Opt-in depth behavior: Depth computation is now explicitly disabled in `calibrate_extrinsics.py` unless `--verify-depth` or `--refine-depth` is set. This prevents unnecessary GPU/CPU overhead during standard extrinsic calibration.
|