zed-playground/py_workspace/.sisyphus/notepads/icp-registration/learnings.md

Notes

• Open3D registration_generalized_icp is more robust for noisy depth data but requires normal estimation.

• Multi-scale ICP significantly improves convergence range by starting with large voxels (4x base).

• Information matrix from get_information_matrix_from_point_clouds is essential for weighting edges in the pose graph.

• Initial relative transform from extrinsics is crucial for ICP convergence when cameras are far apart in camera frame.

• Pose graph optimization should only include the connected component reachable from the reference camera to avoid singular systems.

• Transforming point clouds to camera frame before pairwise ICP allows using the initial extrinsic-derived relative transform as a meaningful starting guess.

• Pose graph construction must strictly filter for the connected component reachable from the reference camera to ensure a well-constrained optimization problem.

• Aligned build_pose_graph signature with plan (returns PoseGraph only).

• Implemented disconnected camera logging within build_pose_graph.

• Re-derived optimized_serials in refine_with_icp to maintain node-to-serial mapping consistency.

• Open3D PoseGraphEdge(source, target, T) expects T to be T_{target\_source}.

• When monkeypatching for tests, ensure all internal calls are accounted for, especially when production code has bugs that need to be worked around or highlighted.

• Integrated ICP refinement into refine_ground_plane.py CLI, enabling optional global registration after ground plane alignment.

• Added _meta.icp_refined block to output JSON to track ICP configuration and success metrics.

ICP Registration

• GICP method in requires normals, which are estimated internally if not provided.
• Synthetic tests for ICP should use deterministic seeds for point cloud generation to ensure stability.

ICP Registration

• GICP method in pairwise_icp requires normals, which are estimated internally if not provided.
• Synthetic tests for ICP should use deterministic seeds for point cloud generation to ensure stability.

Balanced SN442 Profile

• A balanced profile was established to handle cameras (like SN44289123) that show significant floor disconnect (~5.5cm translation, ~1.5° rotation).
• Permissive RANSAC threshold (0.05m) and min inlier ratio (0.01) allow recovery when strict defaults fail.
• Safety limits were increased to --max-rotation-deg 15 and --max-translation-m 1.0 to accommodate observed disconnects.
• GICP with 0.04m voxel size provides robust inter-camera alignment following ground plane correction.