zed-playground/py_workspace/.sisyphus/notepads/icp-registration/issues.md

Notes

• basedpyright reports many warnings for open3d due to missing type stubs; these are suppressed/ignored as they don't indicate logic errors.

• Synthetic smoke testing requires mocking unproject_depth_to_points or providing valid depth/K pairs that produce overlapping points.

• Open3D PoseGraph requires careful indexing; ensuring reference camera is at index 0 and fixed helps stabilize global optimization.

• Gravity constraint regularization is now correctly applied relative to the original extrinsic-derived transform, preserving the RANSAC-leveled orientation.

• [BUG] build_pose_graph in aruco/icp_registration.py uses result.transformation (which is T_{21}) as the edge from idx2 to idx1 (which expects T_{12}). This causes global optimization to move cameras in the wrong direction and likely exceed safety bounds.

• Pairwise ICP convergence depends on min_overlap_area and min_fitness; cameras failing these criteria are excluded from global optimization and logged as warnings.