Refactor code structure for improved readability and maintainability

.python-version

@@ -0,0 +1 @@
+3.13

.vscode/settings.json

@@ -1,6 +1,6 @@
 {
     "python.analysis.typeCheckingMode": "standard",
     "python.analysis.autoImportCompletions": true,
-    "python-envs.defaultEnvManager": "ms-python.python:system",
+    "python-envs.defaultEnvManager": "ms-python.python:uv",
     "python-envs.pythonProjects": []
 }

AGENTS.md

@@ -0,0 +1,180 @@
+# AGENTS.md
+
+Guide for coding agents working in this repository.
+
+## Project Overview
+
+- Domain: Computer vision experiments with ArUco / ChArUco and camera calibration
+- Language: Python
+- Python version: 3.13+ (see `.python-version`, `pyproject.toml`)
+- Env/deps manager: `uv`
+- Test runner: `pytest`
+- Lint/format: `ruff`
+- Packaging mode: workspace scripts (`[tool.uv] package = false`)
+
+## High-Signal Files
+
+- `find_aruco_points.py`: live marker detection + frame overlays
+- `find_extrinsic_object.py`: pose estimation with known object points
+- `cali.py`: charuco calibration and parquet output
+- `capture.py`: webcam frame capture helper
+- `run_capture.py`: multi-port gstreamer recorder CLI (`click`)
+- `scripts/uv_to_object_points.py`: UV -> 3D conversion script
+- `test_cam_props.py`: camera property probe test/script
+- Shell helpers: `gen.sh`, `cvt_all_pdfs.sh`, `dump_and_play.sh`
+
+## Setup
+
+```bash
+uv sync
+```
+
+Creates `.venv` and installs dependencies from `pyproject.toml`.
+
+## Build / Lint / Test Commands
+
+No compile step. “Build” usually means running generation/util scripts.
+
+### Lint
+
+```bash
+uv run ruff check .
+uv run ruff check . --fix
+```
+
+### Format
+
+```bash
+uv run ruff format .
+```
+
+### Tests
+
+Full suite:
+
+```bash
+uv run pytest -q
+```
+
+Single test file (important):
+
+```bash
+uv run pytest test_cam_props.py -q
+```
+
+Single test function:
+
+```bash
+uv run pytest test_cam_props.py::test_props -q
+```
+
+Keyword filter:
+
+```bash
+uv run pytest -k "props" -q
+```
+
+### Script sanity checks
+
+```bash
+uv run python -m py_compile *.py scripts/*.py
+uv run python run_capture.py --help
+uv run python scripts/uv_to_object_points.py --help
+```
+
+## Runtime/Tooling Notes
+
+- Prefer `uv run python <script>.py` for all local execution.
+- `scripts/uv_to_object_points.py` also supports script-mode execution directly.
+- Shell scripts require system tools:
+  - `gen.sh`: expects `MarkerPrinter.py` from OpenCV contrib generator context
+  - `cvt_all_pdfs.sh`: needs ImageMagick (`magick`)
+  - `dump_and_play.sh`: needs `gst-launch-1.0`
+
+## Code Style (Observed Conventions)
+
+Follow existing style in touched files; keep edits narrow.
+
+### Imports
+
+- Keep imports at top of module.
+- Common pattern: stdlib + third-party; ordering is not perfectly strict.
+- Do not do broad import reordering unless asked.
+
+### Formatting
+
+- 4-space indentation
+- Predominantly double quotes
+- Script-oriented functions; avoid unnecessary abstractions
+
+### Types
+
+- Type hints are common in core numeric/geometry scripts.
+- Existing usage includes:
+  - builtin generics (`list[int]`, `tuple[float, float]`)
+  - `TypedDict`
+  - `typing.cast`
+  - `numpy.typing` and jaxtyping aliases
+- Preserve/improve types when touching typed code.
+
+### Naming
+
+- `snake_case`: functions, variables
+- `PascalCase`: classes
+- `UPPER_SNAKE_CASE`: constants/config
+
+### Error Handling / Logging
+
+- `loguru` is the preferred logger.
+- Use `logger.warning(...)` for recoverable detection/runtime issues.
+- Raise explicit exceptions for invalid inputs in utility code.
+
+### CLI / Entrypoints
+
+- `click` is used for CLI scripts.
+- Use `if __name__ == "__main__":` entrypoints.
+- Keep side effects in `main()` when possible.
+
+### CV / Numeric Practices
+
+- Be explicit about array shapes where relevant.
+- Normalize/reshape OpenCV outputs before downstream operations.
+- Keep calibration/dictionary constants near top-level config.
+
+## Testing Guidance
+
+- Repo is hardware-heavy; avoid adding camera-dependent tests unless requested.
+- Prefer extracting pure logic and testing that logic.
+- Use pytest naming: `test_*.py`, `test_*`.
+
+## Dependency Management (uv)
+
+```bash
+uv add <package>
+uv add --dev <package>
+uv remove <package>
+uv sync
+```
+
+Prefer checking in both `pyproject.toml` and `uv.lock` for reproducibility.
+
+## Cursor / Copilot Rules Check
+
+- `.cursor/rules/`: not present
+- `.cursorrules`: not present
+- `.github/copilot-instructions.md`: not present
+
+No repository-specific Cursor/Copilot rule files currently exist.
+
+## Agent Workflow Checklist
+
+Before coding:
+1. Read this file and target scripts.
+2. Run `uv sync` if env may be stale.
+3. Check whether task depends on camera/hardware.
+
+After coding:
+1. Run focused checks first.
+2. Run `uv run ruff check .`.
+3. Run `uv run pytest -q` (or explain hardware-related skips).
+4. Keep edits minimal and task-scoped.

find_aruco_points_with_image.ipynb

@@ -37,8 +37,10 @@
     "# 7x7\n",
     "# DICTIONARY: Final[int] = aruco.DICT_7X7_1000\n",
     "DICTIONARY: Final[int] = aruco.DICT_APRILTAG_36H11\n",
-    "# 400mm\n",
+    "# real-world box side length (e.g. 600mm)\n",
-    "MARKER_LENGTH: Final[float] = 0.4"
+    "BOX_SIZE_MM: Final[float] = 600.0\n",
+    "# standard_box.glb spans approximately [-1, 1] so side length is 2 mesh units\n",
+    "UNIT_BOX_SIDE_MESH_UNITS: Final[float] = 2.0"
    ]
   },
   {
@@ -342,10 +344,29 @@
    ],
    "source": [
     "m = trimesh.load_mesh(\"sample/standard_box.glb\")\n",
+    "\n",
+    "def scale_mesh_for_box_size_mm(\n",
+    "    mesh: trimesh.Trimesh, box_size_mm: float, unit_box_side: float = 2.0\n",
+    ") -> trimesh.Trimesh:\n",
+    "    if box_size_mm <= 0:\n",
+    "        raise ValueError(\"box_size_mm must be positive\")\n",
+    "    if unit_box_side <= 0:\n",
+    "        raise ValueError(\"unit_box_side must be positive\")\n",
+    "    scale = (box_size_mm / 1000.0) / unit_box_side\n",
+    "    scaled = mesh.copy()\n",
+    "    scaled.vertices = scaled.vertices * scale\n",
+    "    return scaled\n",
+    "\n",
     "def marker_to_3d_coords(marker: Marker, mesh: trimesh.Trimesh):\n",
     "    uv_points = marker.corners\n",
     "    return interpolate_uvs_to_3d_trimesh(uv_points, mesh)\n",
     "\n",
+    "m = scale_mesh_for_box_size_mm(\n",
+    "    mesh=cast(trimesh.Trimesh, m),\n",
+    "    box_size_mm=BOX_SIZE_MM,\n",
+    "    unit_box_side=UNIT_BOX_SIDE_MESH_UNITS,\n",
+    ")\n",
+    "\n",
     "id_to_3d_coords = {marker.id: marker_to_3d_coords(marker, m) for marker in output_markers}\n",
     "# note that the glb is Y up\n",
     "# when visualizing with matplotlib, it's Z up\n",
@@ -485,12 +506,13 @@
     "    markers.append(MarkerFace(name=name, ids=np.array(face.marker_ids), corners=corners))\n",
     "display(markers)\n",
     "\n",
-    "ak.to_parquet(markers, \"output/standard_box_markers.parquet\")"
+    "output_parquet = Path(f\"output/standard_box_markers_{int(BOX_SIZE_MM)}mm.parquet\")\n",
+    "ak.to_parquet(markers, str(output_parquet))"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {

pyproject.toml

@@ -0,0 +1,30 @@
+[project]
+name = "charuco-board-exp"
+version = "0.1.0"
+description = "ChArUco and ArUco calibration/pose experiments"
+readme = "README.md"
+requires-python = ">=3.13"
+dependencies = [
+  "awkward>=2.8.4",
+  "click>=8.1.8",
+  "jaxtyping>=0.3.2",
+  "loguru>=0.7.3",
+  "numpy>=2.2.3",
+  "opencv-python>=4.11.0.86",
+  "orjson>=3.10.15",
+  "trimesh>=4.6.4",
+]
+
+[dependency-groups]
+dev = [
+  "jupyterlab>=4.5.3",
+  "pytest>=8.3.4",
+  "ruff>=0.9.6",
+]
+
+[tool.uv]
+package = false
+
+[tool.pytest.ini_options]
+python_files = ["test_*.py"]
+testpaths = ["."]

scripts/uv_to_object_points.py

@@ -0,0 +1,255 @@
+#!/usr/bin/env -S uv run --script
+# /// script
+# requires-python = ">=3.13"
+# dependencies = [
+#   "numpy",
+#   "opencv-python",
+#   "trimesh",
+#   "awkward",
+#   "orjson",
+#   "click",
+# ]
+# ///
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, cast
+
+import awkward as ak
+import click
+import cv2
+import numpy as np
+import orjson
+import trimesh
+from cv2 import aruco
+from numpy.typing import NDArray
+
+
+@dataclass
+class Marker:
+    id: int
+    center: NDArray[np.float64]
+    corners: NDArray[np.float64]
+
+
+def normalize_point(
+    point: NDArray[Any], width: int, height: int
+) -> NDArray[np.float64]:
+    return cast(
+        NDArray[np.float64], point / np.array([width, height], dtype=np.float64)
+    )
+
+
+def flip_y(point: NDArray[Any], y_max: float = 1.0) -> NDArray[np.float64]:
+    return np.array([point[0], y_max - point[1]], dtype=np.float64)
+
+
+def detect_markers_as_uv(
+    input_image: Path,
+    dictionary: int,
+) -> list[Marker]:
+    frame = cv2.imread(str(input_image))
+    if frame is None:
+        raise FileNotFoundError(f"Failed to read image: {input_image}")
+
+    detector = aruco.ArucoDetector(
+        dictionary=aruco.getPredefinedDictionary(dictionary),
+        detectorParams=aruco.DetectorParameters(),
+    )
+    grey = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
+    markers, ids, _ = detector.detectMarkers(grey)
+    if ids is None:
+        return []
+
+    markers = np.reshape(markers, (-1, 4, 2))
+    ids = np.reshape(ids, (-1, 1))
+    image_width = frame.shape[1]
+    image_height = frame.shape[0]
+
+    output_markers: list[Marker] = []
+    for m, marker_id in zip(markers, ids):
+        center = np.mean(m, axis=0)
+        output_markers.append(
+            Marker(
+                id=int(marker_id[0]),
+                center=flip_y(normalize_point(center, image_width, image_height)),
+                corners=np.array(
+                    [
+                        flip_y(normalize_point(corner, image_width, image_height))
+                        for corner in m
+                    ],
+                    dtype=np.float64,
+                ),
+            )
+        )
+
+    return output_markers
+
+
+def interpolate_uvs_to_3d(
+    uv_points: NDArray[np.float64],
+    vertices: NDArray[np.float64],
+    uvs: NDArray[np.float64],
+    faces: NDArray[np.int64],
+    epsilon: float = 1e-6,
+) -> NDArray[np.float64]:
+    results = np.full((uv_points.shape[0], 3), np.nan, dtype=np.float64)
+    for point_index, uv_point in enumerate(uv_points):
+        for face in faces:
+            uv_tri = uvs[face]
+            v_tri = vertices[face]
+            matrix = np.array(
+                [
+                    [uv_tri[0, 0] - uv_tri[2, 0], uv_tri[1, 0] - uv_tri[2, 0]],
+                    [uv_tri[0, 1] - uv_tri[2, 1], uv_tri[1, 1] - uv_tri[2, 1]],
+                ],
+                dtype=np.float64,
+            )
+            rhs = uv_point - uv_tri[2]
+            try:
+                w0, w1 = np.linalg.solve(matrix, rhs)
+            except np.linalg.LinAlgError:
+                continue
+            w2 = 1.0 - w0 - w1
+            if min(w0, w1, w2) >= -epsilon:
+                results[point_index] = w0 * v_tri[0] + w1 * v_tri[1] + w2 * v_tri[2]
+                break
+    return results
+
+
+def interpolate_uvs_to_3d_trimesh(
+    uv_points: NDArray[np.float64],
+    mesh: trimesh.Trimesh,
+    epsilon: float = 1e-6,
+) -> NDArray[np.float64]:
+    if mesh.visual is None:
+        raise ValueError("Mesh has no visual")
+    uv_data = cast(Any, mesh.visual).uv
+    if uv_data is None:
+        raise ValueError("Mesh has no UV")
+    return interpolate_uvs_to_3d(
+        uv_points=uv_points,
+        vertices=cast(NDArray[np.float64], mesh.vertices),
+        uvs=cast(NDArray[np.float64], uv_data),
+        faces=cast(NDArray[np.int64], mesh.faces),
+        epsilon=epsilon,
+    )
+
+
+def scale_mesh_for_box_size_mm(
+    mesh: trimesh.Trimesh,
+    box_size_mm: float,
+    unit_box_side: float = 2.0,
+) -> trimesh.Trimesh:
+    if box_size_mm <= 0:
+        raise ValueError("box_size_mm must be positive")
+    if unit_box_side <= 0:
+        raise ValueError("unit_box_side must be positive")
+
+    scale = (box_size_mm / 1000.0) / unit_box_side
+    scaled = mesh.copy()
+    scaled.vertices = cast(NDArray[np.float64], scaled.vertices * scale)
+    return scaled
+
+
+def marker_to_3d_coords(marker: Marker, mesh: trimesh.Trimesh) -> NDArray[np.float64]:
+    return interpolate_uvs_to_3d_trimesh(marker.corners, mesh)
+
+
+def parse_dictionary(value: str) -> int:
+    if not hasattr(aruco, value):
+        raise ValueError(f"Unknown aruco dictionary name: {value}")
+    return int(getattr(aruco, value))
+
+
+@click.command(
+    help="Convert draw_uv marker detections into 3D object points with real-world box sizing"
+)
+@click.option(
+    "--input-image",
+    type=click.Path(path_type=Path),
+    default=Path("merged_uv_layout.png"),
+    show_default=True,
+)
+@click.option(
+    "--mesh",
+    type=click.Path(path_type=Path),
+    default=Path("sample/standard_box.glb"),
+    show_default=True,
+)
+@click.option(
+    "--dictionary", type=str, default="DICT_APRILTAG_36H11", show_default=True
+)
+@click.option("--box-size-mm", type=float, default=600.0, show_default=True)
+@click.option("--unit-box-side", type=float, default=2.0, show_default=True)
+@click.option(
+    "--output-json",
+    type=click.Path(path_type=Path),
+    default=Path("output/aruco_2d_uv_coords_normalized.json"),
+    show_default=True,
+)
+@click.option(
+    "--output-parquet",
+    type=click.Path(path_type=Path),
+    default=Path("output/standard_box_markers.parquet"),
+    show_default=True,
+)
+def main(
+    input_image: Path,
+    mesh: Path,
+    dictionary: str,
+    box_size_mm: float,
+    unit_box_side: float,
+    output_json: Path,
+    output_parquet: Path,
+) -> None:
+    dictionary_value = parse_dictionary(dictionary)
+    output_markers = detect_markers_as_uv(input_image, dictionary_value)
+
+    output_json.parent.mkdir(parents=True, exist_ok=True)
+    output_json.write_bytes(
+        orjson.dumps(output_markers, option=orjson.OPT_SERIALIZE_NUMPY)
+    )
+
+    loaded = trimesh.load_mesh(mesh)
+    if isinstance(loaded, trimesh.Scene):
+        if not loaded.geometry:
+            raise ValueError("Scene has no geometry")
+        mesh = list(loaded.geometry.values())[0]
+    else:
+        mesh = loaded
+    if not isinstance(mesh, trimesh.Trimesh):
+        raise TypeError("Expected Trimesh or Scene with Trimesh geometry")
+
+    mesh = scale_mesh_for_box_size_mm(mesh, box_size_mm, unit_box_side)
+    id_to_3d_coords = {
+        marker.id: marker_to_3d_coords(marker, mesh) for marker in output_markers
+    }
+
+    face_to_ids = {
+        "bottom": [21],
+        "back": [22],
+        "top": [23],
+        "front": [24],
+        "right": [26],
+        "left": [25],
+    }
+    rows: list[dict[str, Any]] = []
+    for name, marker_ids in face_to_ids.items():
+        corners = np.array([id_to_3d_coords[marker_id] for marker_id in marker_ids])
+        rows.append(
+            {
+                "name": name,
+                "ids": np.array(marker_ids),
+                "corners": corners,
+            }
+        )
+
+    output_parquet.parent.mkdir(parents=True, exist_ok=True)
+    ak.to_parquet(rows, str(output_parquet))
+
+
+if __name__ == "__main__":
+    main()