Files
zed-playground/py_workspace/.sisyphus/notepads/ground-plane-alignment/learnings.md
T

84 lines
4.7 KiB
Markdown

- Implemented core alignment utilities in aruco/alignment.py.
- Used Rodrigues' rotation formula for vector alignment with explicit handling for parallel and anti-parallel cases.
- Implemented `FACE_MARKER_MAP` and `get_face_normal_from_geometry` to support multi-marker face normal averaging.
- Implemented `detect_ground_face` using dot-product scoring against camera up-vector with `loguru` debug logging.
- Integrated ground-plane alignment into `calibrate_extrinsics.py` with CLI-toggled heuristic and explicit face/marker selection.
## SVO Directory Expansion
- Implemented directory expansion for `--svo` argument.
- Iterates through provided paths, checks if directory, and finds `.svo` and `.svo2` files.
- Maintains backward compatibility for single file paths.
- Sorts found files to ensure deterministic processing order.
## ArUco Dictionary Selection
- Added `--aruco-dictionary` CLI option mapping string names to `cv2.aruco` constants.
- Defaults to `DICT_4X4_50` but supports all standard dictionaries including AprilTags.
- Passed to `create_detector` to allow flexibility for different marker sets.
## Minimum Markers Configuration
- Added `--min-markers` CLI option (default 1).
- Passed to `estimate_pose_from_detections` to filter out poses with insufficient marker support.
- Useful for improving robustness or allowing single-marker poses when necessary.
## Logging Improvements
- Added `loguru` debug logs for:
- Number of detected markers per frame.
- Pose acceptance/rejection with specific reasons (reprojection error, marker count).
## Dynamic Face Mapping
- Implemented `load_face_mapping` in `aruco/marker_geometry.py` to read face definitions from parquet metadata.
- Parquet file must contain `name` (string) and `ids` (list of ints) columns.
- `calibrate_extrinsics.py` now loads this map at runtime and passes it to alignment functions.
- `aruco/alignment.py` functions (`get_face_normal_from_geometry`, `detect_ground_face`) now accept an optional `face_marker_map` argument.
## Strict Data-Driven Alignment
- Removed implicit fallback to `FACE_MARKER_MAP` in `aruco/alignment.py`.
- `calibrate_extrinsics.py` now explicitly checks for loaded face mapping.
- If mapping is missing (e.g., old parquet without `name`/`ids`), alignment is skipped with a warning instead of using hardcoded defaults.
- This enforces the requirement that ground alignment configuration must come from the marker definition file.
- Alignment tests verify that `rotation_align_vectors` correctly handles identity, 90-degree, and anti-parallel cases.
- `detect_ground_face` and `get_face_normal_from_geometry` are now data-driven, requiring an explicit `face_marker_map` at runtime.
- Unit tests use mock geometry to verify normal computation and face selection logic without requiring real SVO/parquet data.
- **Parquet Schema**: The marker configuration parquet file (`standard_box_markers_600mm.parquet`) uses a schema with `name` (string), `ids` (list<int64>), and `corners` (list<list<list<float64>>>).
- **Dual Loading Strategy**: The system loads this single file in two ways:
1. `load_marker_geometry`: Flattens `ids` and `corners` to build a global map of Marker ID -> 3D Corners.
2. `load_face_mapping`: Uses `name` and `ids` to group markers by face (e.g., "bottom"), which is critical for ground plane alignment.
## Runtime Stability
- Fixed `AttributeError: 'FrameData' object has no attribute 'confidence_map'` by explicitly adding it to the dataclass and populating it in `SVOReader`.
- Added `--debug` flag to control log verbosity, defaulting to cleaner INFO level output.
## Consistency Hardening
- Removed "using default fallback" messaging from `calibrate_extrinsics.py` to align with the strict data-driven requirement.
## Fast Iteration
- Added `--max-samples` CLI option to `calibrate_extrinsics.py` to allow processing a limited number of samples (e.g., 1 or 3) instead of the full SVO.
## Test Configuration
- Configured `pytest` in `pyproject.toml` to explicitly target the `tests/` directory and ignore `loguru`, `tmp`, and `libs`.
## Debug Visibility
## Documentation
## Type Annotation Hardening
- Integrated `jaxtyping` for shape-aware array annotations (e.g., `Float[np.ndarray, "4 4"]`).
- Used `TYPE_CHECKING` blocks to define these aliases, ensuring they are available for static analysis (like `basedpyright`) while falling back to standard `np.ndarray` at runtime if `jaxtyping` backends are missing.
## Depth Units
- ZED SDK `retrieve_measure(sl.MEASURE.DEPTH)` returns values in the unit defined in `InitParameters.coordinate_units`.
- The default unit is `MILLIMETER`.
- Since our extrinsics and marker geometry are in meters, we must explicitly convert the retrieved depth map to meters (divide by 1000.0) to avoid massive scale mismatches during verification and refinement.