zed-playground/py_workspace/.sisyphus/notepads/ground-plane-alignment/learnings.md

• Implemented core alignment utilities in aruco/alignment.py.
• Used Rodrigues' rotation formula for vector alignment with explicit handling for parallel and anti-parallel cases.
• Implemented FACE_MARKER_MAP and get_face_normal_from_geometry to support multi-marker face normal averaging.
• Implemented detect_ground_face using dot-product scoring against camera up-vector with loguru debug logging.
• Integrated ground-plane alignment into calibrate_extrinsics.py with CLI-toggled heuristic and explicit face/marker selection.

SVO Directory Expansion

• Implemented directory expansion for --svo argument.
• Iterates through provided paths, checks if directory, and finds .svo and .svo2 files.
• Maintains backward compatibility for single file paths.
• Sorts found files to ensure deterministic processing order.

ArUco Dictionary Selection

• Added --aruco-dictionary CLI option mapping string names to cv2.aruco constants.
• Defaults to DICT_4X4_50 but supports all standard dictionaries including AprilTags.
• Passed to create_detector to allow flexibility for different marker sets.

Minimum Markers Configuration

• Added --min-markers CLI option (default 1).
• Passed to estimate_pose_from_detections to filter out poses with insufficient marker support.
• Useful for improving robustness or allowing single-marker poses when necessary.

Logging Improvements

• Added loguru debug logs for:
  • Number of detected markers per frame.
  • Pose acceptance/rejection with specific reasons (reprojection error, marker count).

Dynamic Face Mapping

• Implemented load_face_mapping in aruco/marker_geometry.py to read face definitions from parquet metadata.
• Parquet file must contain name (string) and ids (list of ints) columns.
• calibrate_extrinsics.py now loads this map at runtime and passes it to alignment functions.
• aruco/alignment.py functions (get_face_normal_from_geometry, detect_ground_face) now accept an optional face_marker_map argument.

Strict Data-Driven Alignment

• Removed implicit fallback to FACE_MARKER_MAP in aruco/alignment.py.

• calibrate_extrinsics.py now explicitly checks for loaded face mapping.

• If mapping is missing (e.g., old parquet without name/ids), alignment is skipped with a warning instead of using hardcoded defaults.

• This enforces the requirement that ground alignment configuration must come from the marker definition file.

• Alignment tests verify that rotation_align_vectors correctly handles identity, 90-degree, and anti-parallel cases.

• detect_ground_face and get_face_normal_from_geometry are now data-driven, requiring an explicit face_marker_map at runtime.

• Unit tests use mock geometry to verify normal computation and face selection logic without requiring real SVO/parquet data.

• Parquet Schema: The marker configuration parquet file (standard_box_markers_600mm.parquet) uses a schema with name (string), ids (list), and corners (list<list<list>>).

• Dual Loading Strategy: The system loads this single file in two ways:

  1. load_marker_geometry: Flattens ids and corners to build a global map of Marker ID -> 3D Corners.
  2. load_face_mapping: Uses name and ids to group markers by face (e.g., "bottom"), which is critical for ground plane alignment.

Runtime Stability

• Fixed AttributeError: 'FrameData' object has no attribute 'confidence_map' by explicitly adding it to the dataclass and populating it in SVOReader.
• Added --debug flag to control log verbosity, defaulting to cleaner INFO level output.

Consistency Hardening

• Removed "using default fallback" messaging from calibrate_extrinsics.py to align with the strict data-driven requirement.

Fast Iteration

• Added --max-samples CLI option to calibrate_extrinsics.py to allow processing a limited number of samples (e.g., 1 or 3) instead of the full SVO.

Test Configuration

• Configured pytest in pyproject.toml to explicitly target the tests/ directory and ignore loguru, tmp, and libs.

Debug Visibility

Documentation

Type Annotation Hardening

• Integrated jaxtyping for shape-aware array annotations (e.g., Float[np.ndarray, "4 4"]).
• Used TYPE_CHECKING blocks to define these aliases, ensuring they are available for static analysis (like basedpyright) while falling back to standard np.ndarray at runtime if jaxtyping backends are missing.

Depth Units

• ZED SDK retrieve_measure(sl.MEASURE.DEPTH) returns values in the unit defined in InitParameters.coordinate_units.
• The default unit is MILLIMETER.
• Since our extrinsics and marker geometry are in meters, we must explicitly convert the retrieved depth map to meters (divide by 1000.0) to avoid massive scale mismatches during verification and refinement.