feat(calibration): robust depth refinement pipeline with diagnostics and benchmarking
This commit is contained in:
+184
-33
@@ -1,37 +1,188 @@
|
||||
# Python Agent Context
|
||||
# AGENTS.md — Python Workspace Guide
|
||||
|
||||
## Environment
|
||||
- **Directory**: `/workspaces/zed-playground/py_workspace`
|
||||
- **Package Manager**: `uv`
|
||||
- **Python Version**: 3.12+ (Managed by `uv`)
|
||||
- **Dependencies**: Defined in `pyproject.toml`
|
||||
- `pyzed`: ZED SDK Python wrapper
|
||||
- `opencv-python`: GUI and image processing
|
||||
- `click`: CLI argument parsing
|
||||
- `numpy`, `cupy-cuda12x`: Data manipulation
|
||||
This file defines coding-agent guidance for:
|
||||
`/workspaces/zed-playground/py_workspace`
|
||||
|
||||
## Workflow & Commands
|
||||
- **Run Scripts**: Always use `uv run` to ensure correct environment.
|
||||
```bash
|
||||
uv run streaming_receiver.py --help
|
||||
uv run recording_multi.py
|
||||
```
|
||||
- **New Dependencies**: Add with `uv add <package>` (e.g., `uv add requests`).
|
||||
Use this as the primary reference for Python work in this repository.
|
||||
|
||||
## Architecture & Patterns
|
||||
- **Network Camera Handling**:
|
||||
- Use `zed_network_utils.py` for all network config parsing.
|
||||
- Config file: `/workspaces/zed-playground/zed_settings/inside_network.json`
|
||||
- **Threading Model**:
|
||||
- **Main Thread**: MUST handle all OpenCV GUI (`cv2.imshow`, `cv2.waitKey`).
|
||||
- **Worker Threads**: Handle `camera.grab()` and data retrieval.
|
||||
- **Communication**: Use `queue.Queue` to pass frames from workers to main.
|
||||
- **ZED API Patterns**:
|
||||
- Streaming Input: `init_params.set_from_stream(ip, port)`
|
||||
- Serial Number: Use `camera.get_camera_information().serial_number`.
|
||||
---
|
||||
|
||||
## Documentation & References
|
||||
- **Python API Docs**: `/usr/local/zed/doc/API/html/python/index.html`
|
||||
- **ZED SDK General Docs**: `/usr/local/zed/doc/`
|
||||
- **C++ Headers (Reference)**: `/usr/local/zed/include/sl/`
|
||||
- Useful for understanding underlying enum values or behaviors not fully detailed in Python docstrings.
|
||||
## 1) Scope & Environment
|
||||
|
||||
- Package manager: **uv**
|
||||
- Python: **3.12+**
|
||||
- Project file: `pyproject.toml`
|
||||
- Main package/work area: top-level scripts + `aruco/` + `tests/`
|
||||
- Non-primary/vendor-like areas (avoid unless explicitly asked):
|
||||
- `loguru/`
|
||||
- `tmp/`
|
||||
- `libs/`
|
||||
|
||||
Core dependencies include:
|
||||
- `pyzed`, `opencv-python`, `click`, `numpy`, `scipy`
|
||||
- `loguru`, `awkward`, `jaxtyping`, `pyarrow`, `pandas`
|
||||
|
||||
Dev dependencies:
|
||||
- `pytest`, `basedpyright`
|
||||
|
||||
---
|
||||
|
||||
## 2) Build / Run / Lint / Test Commands
|
||||
|
||||
Run commands from:
|
||||
`/workspaces/zed-playground/py_workspace`
|
||||
|
||||
Environment sync:
|
||||
```bash
|
||||
uv sync
|
||||
uv run python -V
|
||||
```
|
||||
|
||||
Run common scripts:
|
||||
```bash
|
||||
uv run streaming_receiver.py --help
|
||||
uv run recording_multi.py
|
||||
uv run calibrate_extrinsics.py --help
|
||||
```
|
||||
|
||||
Type-check / lint-equivalent:
|
||||
```bash
|
||||
uv run basedpyright
|
||||
```
|
||||
|
||||
Run all tests:
|
||||
```bash
|
||||
uv run pytest
|
||||
```
|
||||
|
||||
Run a single test file:
|
||||
```bash
|
||||
uv run pytest tests/test_depth_refine.py
|
||||
```
|
||||
|
||||
Run a single test function:
|
||||
```bash
|
||||
uv run pytest tests/test_depth_refine.py::test_refine_extrinsics_with_depth_with_offset
|
||||
```
|
||||
|
||||
Run subset by keyword:
|
||||
```bash
|
||||
uv run pytest -k "depth and refine"
|
||||
```
|
||||
|
||||
Useful options:
|
||||
```bash
|
||||
uv run pytest -x -vv
|
||||
```
|
||||
|
||||
Notes from `pyproject.toml`:
|
||||
- `testpaths = ["tests"]`
|
||||
- `norecursedirs = ["loguru", "tmp", "libs"]`
|
||||
|
||||
---
|
||||
|
||||
## 3) Rules Files (Cursor / Copilot)
|
||||
|
||||
Latest scan in this workspace found:
|
||||
- No `.cursorrules`
|
||||
- No `.cursor/rules/`
|
||||
- No `.github/copilot-instructions.md`
|
||||
|
||||
If these files appear later, treat them as higher-priority local instructions.
|
||||
|
||||
---
|
||||
|
||||
## 4) Python Code Style Conventions
|
||||
|
||||
### Imports
|
||||
- Group imports: standard library → third-party → local modules.
|
||||
- Use ZED import style:
|
||||
- `import pyzed.sl as sl`
|
||||
- In package modules (`aruco/*`), prefer relative imports:
|
||||
- `from .pose_math import ...`
|
||||
- In top-level scripts, absolute imports are common:
|
||||
- `from aruco.detector import ...`
|
||||
|
||||
### Formatting & structure
|
||||
- 4-space indentation.
|
||||
- PEP8-style layout.
|
||||
- Keep functions focused and composable.
|
||||
- Prefer explicit CLI options over positional ambiguity.
|
||||
|
||||
### Typing
|
||||
- Add type hints on public and most internal functions.
|
||||
- Existing code uses both:
|
||||
- `typing.Optional/List/Dict/Tuple`
|
||||
- modern `|` unions
|
||||
Stay consistent with the surrounding file.
|
||||
- When arrays/matrices are central, use `jaxtyping` shape aliases (with `TYPE_CHECKING` guards) where already established.
|
||||
- Avoid broad `Any` unless unavoidable at library boundaries (OpenCV/pyzed interop).
|
||||
|
||||
### Naming
|
||||
- `snake_case`: functions, variables, modules.
|
||||
- `PascalCase`: classes.
|
||||
- `UPPER_SNAKE_CASE`: constants.
|
||||
|
||||
### Docstrings
|
||||
- Use concise purpose + `Args` / `Returns`.
|
||||
- Document expected array shapes and units for geometry/math functions.
|
||||
|
||||
### Logging & output
|
||||
- User-facing CLI output: `click.echo`.
|
||||
- Diagnostic logs: `loguru` (`logger.debug/info/warning`).
|
||||
- Keep verbose logs behind a `--debug` flag.
|
||||
|
||||
### Error handling
|
||||
- Raise specific exceptions (`ValueError`, `FileNotFoundError`, etc.) with actionable messages.
|
||||
- For CLI fatal paths, use `click.UsageError` or `SystemExit(1)` patterns.
|
||||
- Validate early (shape/range/None) before expensive compute.
|
||||
|
||||
---
|
||||
|
||||
## 5) Testing Conventions
|
||||
|
||||
- Framework: `pytest`
|
||||
- Numerical checks: use `numpy.testing.assert_allclose` where appropriate.
|
||||
- Exception checks: `pytest.raises(..., match=...)`.
|
||||
- Place/add tests under `tests/`.
|
||||
- For `aruco/*` behavior changes, update related tests (`test_depth_*`, `test_alignment`, etc.).
|
||||
|
||||
---
|
||||
|
||||
## 6) Project-Specific ZED Guidance
|
||||
|
||||
### Streaming vs Fusion architecture
|
||||
- Streaming API sends compressed video; host computes depth/tracking.
|
||||
- Fusion API sends metadata; host does lightweight fusion.
|
||||
- Do not assume built-in depth-map streaming parity with metadata fusion.
|
||||
|
||||
### Units
|
||||
- Keep units explicit and consistent end-to-end.
|
||||
- Marker parquet geometry is meter-based in this workspace.
|
||||
- Be careful with ZED depth unit configuration and conversions.
|
||||
|
||||
### Threading
|
||||
- OpenCV GUI (`cv2.imshow`, `cv2.waitKey`) should stay on main thread.
|
||||
- Use worker thread(s) for grab/retrieve and queue handoff patterns.
|
||||
|
||||
### Network config
|
||||
- Follow `zed_network_utils.py` and `zed_settings/inside_network.json` patterns.
|
||||
|
||||
---
|
||||
|
||||
## 7) Agent Workflow Checklist
|
||||
|
||||
Before editing:
|
||||
1. Identify target file/module and nearest existing pattern.
|
||||
2. Confirm expected command(s) from this guide.
|
||||
3. Check for relevant existing tests.
|
||||
|
||||
After editing:
|
||||
1. Run focused tests first.
|
||||
2. Run broader test selection as needed.
|
||||
3. Run `uv run basedpyright` on final pass.
|
||||
4. Keep changes minimal and avoid unrelated churn.
|
||||
|
||||
If uncertain:
|
||||
- Prefer small, verifiable changes.
|
||||
- Document assumptions in PR/commit notes.
|
||||
|
||||
Reference in New Issue
Block a user