crosstyan
9c861105f7
- Add comprehensive work plan for ArUco-based multi-camera calibration - Add recording_multi.py for multi-camera SVO recording - Add streaming_receiver.py for network streaming - Add svo_playback.py for synchronized playback - Add zed_network_utils.py for camera configuration - Add AGENTS.md with project context
55 lines
3.1 KiB
Markdown
55 lines
3.1 KiB
Markdown
## 2026-02-04 Init
|
|
- Repo is script-first, but `aruco/` imports work via Python namespace package even without `__init__.py`.
|
|
- No existing test suite. `pytest` is not installed; will need to be added (likely as dev dependency) before tests can run.
|
|
- Existing CLI patterns use `click` (e.g., `streaming_receiver.py`).
|
|
|
|
## Pose Math Utilities
|
|
- Created `aruco/pose_math.py` for common SE(3) operations.
|
|
- Standardized on 4x4 homogeneous matrices for composition and inversion.
|
|
- Inversion uses the efficient property: [R | t; 0 | 1]^-1 = [R^T | -R^T * t; 0 | 1].
|
|
- Reprojection error calculation uses `cv2.projectPoints` and mean Euclidean distance.
|
|
|
|
- ArUco marker geometry loading and validation logic implemented in `aruco/marker_geometry.py`.
|
|
- Use `awkward` and `pyarrow` for efficient parquet loading of multi-dimensional arrays (corners).
|
|
- Reshaping `ak.to_numpy` output is necessary when dealing with nested structures like corners (4x3).
|
|
|
|
## SVO Sync
|
|
- Added `aruco/svo_sync.py` with `SVOReader` and `FrameData` to open multiple SVO2 files, align starts by timestamp, and grab frames.
|
|
- Verified with real local SVO2 files: able to open, sync, and grab frames for 2 cameras.
|
|
|
|
## ArUco Detector
|
|
- Added `aruco/detector.py` implementing:
|
|
- ArUcoDetector creation (DICT_4X4_50)
|
|
- marker detection (BGR or grayscale input)
|
|
- ZED intrinsics -> K matrix extraction
|
|
- multi-marker solvePnP pose estimation + reprojection error
|
|
- Verified pose estimation with synthetic projected points and with a real SVO-opened camera for intrinsics.
|
|
- Implemented `PoseAccumulator` in `aruco/pose_averaging.py` for robust SE(3) pose averaging.
|
|
- Added RANSAC-based filtering for rotation and translation consensus.
|
|
- Implemented quaternion eigen-mean fallback for rotation averaging when `scipy` is unavailable.
|
|
- Used median for robust translation averaging.
|
|
|
|
## Task 6: ArUco Preview Helpers
|
|
- Implemented `aruco/preview.py` with `draw_detected_markers`, `draw_pose_axes`, and `show_preview`.
|
|
- Ensured grayscale images are converted to BGR before drawing to support colored overlays.
|
|
- Used `cv2.drawFrameAxes` for pose visualization.
|
|
- `show_preview` handles multiple windows based on camera serial numbers.
|
|
|
|
- Removed *.parquet from .gitignore to allow versioning of marker geometry data.
|
|
|
|
## Unit Testing
|
|
- Added pytest as a dev dependency.
|
|
- Implemented synthetic tests for pose math and averaging.
|
|
- Discovered that OpenCV's `projectPoints` is strict about `tvec` being floating-point; ensured tests use `dtype=np.float64`.
|
|
- Verified that `PoseAccumulator` correctly filters outliers using RANSAC and computes robust means.
|
|
|
|
## Pytest sys.path Pitfall
|
|
When running pytest via a console script (e.g., `uv run pytest`), the current working directory is not always automatically added to `sys.path`. This can lead to `ModuleNotFoundError` for local packages like `aruco`.
|
|
**Fix:** Create a `tests/conftest.py` file that explicitly inserts the project root into `sys.path`:
|
|
```python
|
|
import sys
|
|
import os
|
|
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), \"..\")))
|
|
```
|
|
This ensures deterministic import behavior regardless of how pytest is invoked.
|