OpenGait/.sisyphus/notepads/demo-visualizer/learnings.md

Task 1: CLI Flag Addition

• Added --visualize boolean flag to opengait/demo/__main__.py
• Uses argparse with action="store_true" as requested
• Passes visualize=args.visualize to ScoliosisPipeline constructor
• Note: ScoliosisPipeline does not yet accept visualize parameter (Task 3 will add this)
• All existing CLI options preserved with same defaults as pipeline.py click definitions

Task 1 Final State

• Single if name == "main" block
• Uses inspect.signature to conditionally pass visualize
• All CLI options preserved with correct defaults
• --visualize flag present and functional

Task 2: OpenCVVisualizer Implementation

Completed

• Created opengait/demo/visualizer.py with OpenCVVisualizer class
• Internal state self.mask_mode = 0 (0: Both, 1: Raw, 2: Normalized)
• Method update(frame, bbox, track_id, mask_raw, silhouette, label, confidence, fps)
• Two windows: main stream (bbox + text overlay) and segmentation (mode-dependent view)
• Key handling: m cycles mask mode, q returns False to signal quit
• close() method calls cv2.destroyAllWindows()

Key Implementation Details

• Silhouette shape assumed 64x44 (from preprocess.py SIL_HEIGHT/SIL_WIDTH)
• Upscaled to 256x176 for display using INTER_NEAREST to preserve pixelation
• Handles missing inputs gracefully (bbox/mask_raw/silhouette can be None)
• Converts grayscale arrays to BGR for consistent display
• Mode indicator text shown in segmentation window for operator clarity

Type Checking Notes

• OpenCV MatLike type conflicts with NDArray annotations
• These are acceptable warnings - runtime behavior is correct
• Used _ = cv2.function() pattern to suppress unused return value warnings

Task 2 Retry: Type Annotation Fixes

Problem

LSP diagnostics showed basedpyright errors due to OpenCV MatLike type conflicting with strict NDArray[np.uint8] annotations.

Solution

• Added ImageArray type alias = NDArray[np.uint8]
• Used typing.cast() to convert OpenCV return values (MatLike) to ImageArray
• Applied cast() on all cv2.cvtColor() and cv2.resize() calls that return MatLike

Changes Made

• Added: from typing import cast
• Added: ImageArray = NDArray[np.uint8] type alias
• Modified: All methods returning NDArray[np.uint8] now use cast() for OpenCV calls
• Modified: Parameter types changed from NDArray[np.uint8] to ImageArray for consistency

Verification

• lsp_diagnostics: 0 errors (4 warnings about Any types remain - acceptable)
• Import test: uv run python -c "from opengait.demo.visualizer import OpenCVVisualizer; print(ok)" -> ok
• No TODO/FIXME/HACK placeholders introduced

Task 3: Pipeline Integration - COMPLETED

Implementation Summary

• Added _visualizer: object | None attribute to ScoliosisPipeline class
• Updated __init__ to accept visualize: bool = False parameter
• Conditionally instantiates OpenCVVisualizer when visualize=True
• Updated _select_silhouette to return 4-tuple: (silhouette, mask_raw, bbox, track_id)
• Updated process_frame to:
  • Unpack 4-tuple from _select_silhouette
  • Return visualization payload dict with mask_raw, bbox, silhouette, track_id, label, confidence
  • Return payload in all paths (preprocess-only, not-ready-to-classify, and classified)
• Updated run() to:
  • Compute per-frame EMA FPS with alpha=0.1 smoothing
  • Call self._visualizer.update() with all required parameters
  • Break loop if visualizer returns False (user pressed 'q')
• Updated close() to close visualizer via self._visualizer.close() in finally path

Key Design Decisions

• Used object | None type for _visualizer to avoid circular import issues
• EMA FPS uses alpha=0.1 for reasonable smoothing while maintaining responsiveness
• Visualization payload returned in all paths ensures consistent behavior
• Visualizer cleanup happens in close() which is called in finally block

Verification Results

• uv run python -m opengait.demo --help - PASSED
• uv run python -m opengait.demo --source foo --checkpoint bar --config baz --device cpu --visualize - PASSED (reaches file-not-found as expected)
• Constructor accepts visualize parameter - PASSED

Task 3 Regression Fix - COMPLETED

Problem

LSP errors at lines 343-350 due to calling .get() / .update() on object typed values (viz_payload, _visualizer).

Solution

Used cast() to tell the type checker the actual types:

1. viz_payload is cast to dict[str, object] before calling .get()
2. self._visualizer is cast to object and methods are accessed via getattr()

Key Changes

• In run(): viz_dict = cast(dict[str, object], viz_payload) before .get() calls
• In run(): visualizer = cast(object, self._visualizer) then getattr(visualizer, "update", None)
• In close(): Same pattern for calling .close() on visualizer

Verification

• lsp_diagnostics(opengait/demo/pipeline.py) - ZERO ERRORS (only acceptable warnings)
• uv run python -m opengait.demo --help - PASSED
• uv run python -m opengait.demo --source foo --checkpoint bar --config baz --device cpu --visualize - PASSED (reaches file-not-found)

Task 4: CLI Validation Pattern

Lesson

When migrating from click to argparse, ensure ALL behavior is preserved:

• Input validation (both CLI-level and runtime)
• Exit code mapping for different error types
• User-friendly error messages

Pattern for CLI Entry Points

if __name__ == "__main__":
    args = parser.parse_args()
    
    # CLI-level validation
    if args.preprocess_only and not args.silhouette_export_path:
        print("Error: ...", file=sys.stderr)
        raise SystemExit(2)
    
    try:
        validate_runtime_inputs(...)
        pipeline = ScoliosisPipeline(...)
        raise SystemExit(pipeline.run())
    except ValueError as err:
        print(f"Error: {err}", file=sys.stderr)
        raise SystemExit(2) from err
    except RuntimeError as err:
        print(f"Runtime error: {err}", file=sys.stderr)
        raise SystemExit(1) from err

Key Takeaway

Exit code parity matters for test suites and shell scripting. ValueError (user input errors) -> 2, RuntimeError (system/runtime errors) -> 1.

Task 5: YOLO Model Path Relocation

Task Summary

Moved yolo11n-seg.pt from repo root to ckpt/yolo11n-seg.pt and updated all in-repo references.

Files Modified

1. opengait/demo/__main__.py - Updated --yolo-model default
2. opengait/demo/pipeline.py - Updated --yolo-model default
3. tests/demo/test_pipeline.py - Updated YOLO_MODEL_PATH

Verification

• All 3 references now point to ckpt/yolo11n-seg.pt
• Tests pass: 12 passed, 1 skipped in 37.35s

Oracle Caveat Fix #1: Simplified inspect-based conditional logic

Change Summary

• Removed import inspect from opengait/demo/__main__.py
• Removed inspect.signature(ScoliosisPipeline.__init__) call
• Removed conditional if "visualize" in sig.parameters: check
• Now passes visualize=args.visualize directly in pipeline_kwargs

Rationale

The inspect-based conditional was unnecessary complexity for an intra-package API contract. ScoliosisPipeline.init explicitly accepts the visualize parameter, so the runtime check provided no value and added maintenance burden.

Verification

• uv run python -m opengait.demo --help - PASSED (all CLI options preserved)
• uv run pytest tests/demo/test_pipeline.py -q - PASSED (12 passed, 1 skipped)
• lsp_diagnostics(opengait/demo/__main__.py) - No new errors (only pre-existing warnings)

Behavior Parity

CLI behavior unchanged. --visualize flag still supported and passed to pipeline. Default yolo path preserved: ckpt/yolo11n-seg.pt

Oracle Caveat Fix #2: Explicit Typing for Visualizer

Change Summary

• Replaced loose _visualizer: object | None with explicit OpenCVVisualizer | None using TYPE_CHECKING forward reference
• Removed cast(object, ...) and getattr(..., "update"/"close") indirection in favor of direct method calls
• Added explicit casts for values extracted from viz_payload dict to satisfy type checker

Rationale

The previous implementation used object typing and runtime introspection (getattr) to avoid circular imports and maintain optional dependency semantics. However, this sacrificed type safety and code clarity. Using TYPE_CHECKING allows us to:

1. Import the actual type for static analysis without runtime import
2. Call methods directly on the typed visualizer instance
3. Maintain lazy import behavior (OpenCVVisualizer still only imported when visualize=True)

Key Changes

1. Added TYPE_CHECKING block with forward import: from .visualizer import OpenCVVisualizer
2. Changed _visualizer type annotation from object | None to OpenCVVisualizer | None
3. In run(): Removed cast(object, self._visualizer) and getattr(visualizer, "update", None) pattern
   • Now calls self._visualizer.update(...) directly
   • Added explicit casts for dict values extracted from viz_payload
4. In close(): Removed cast(object, self._visualizer) and getattr(visualizer, "close", None) pattern
   • Now calls self._visualizer.close() directly

Verification

• lsp_diagnostics(opengait/demo/pipeline.py) - ZERO ERRORS (only pre-existing warnings unrelated to visualizer)
• uv run pytest tests/demo/test_pipeline.py -q - PASSED (12 passed, 1 skipped)
• Runtime behavior unchanged: lazy import preserved, EMA FPS calculation unchanged, quit handling unchanged

Type Safety Improvement

Before: Runtime introspection required, no static type checking on visualizer methods After: Full static type checking on visualizer.update() and visualizer.close() calls, proper type inference for all parameters

Oracle Non-blocking Improvement: _prepare_both_view Redundant Work Removal

Change Summary

• Modified _prepare_both_view in opengait/demo/visualizer.py to eliminate wasted text-rendering work
• Previously: Called _prepare_raw_view and _prepare_normalized_view which drew mode indicators, then converted to grayscale (destroying the text), then drew combined indicator
• Now: Inlines the view preparation logic without mode indicators, preserving only the final combined indicator

Rationale

The sub-view mode indicators ("Raw Mask" and "Normalized") were being drawn and immediately destroyed by grayscale conversion before stacking. This was pure overhead with no visual effect. The final combined indicator ("Both: Raw | Normalized") is the only one visible to users.

Behavior Preservation

• Visual output unchanged: Final combined mode indicator still displayed
• Mode toggle semantics untouched: mask_mode cycling (0->1->2->0) unchanged
• Placeholder handling preserved: None inputs still produce zero-filled arrays
• All existing tests pass: 12 passed, 1 skipped

Code Quality Impact

• Reduced unnecessary OpenCV text rendering operations
• Eliminated redundant BGR->Gray->BGR conversions on sub-views
• Improved maintainability by making the wasted work explicit (removed)

Verification

• lsp_diagnostics(opengait/demo/visualizer.py) - 0 errors (4 pre-existing Any warnings)
• uv run pytest tests/demo/test_pipeline.py -q - PASSED (12 passed, 1 skipped)

Oracle Non-blocking Cleanup: Duplicate Import Removal

Change Summary

• Removed duplicated import block at lines 17-28 in tests/demo/test_pipeline.py
• Duplicated imports: json, pickle, Path, subprocess, sys, time, Final, cast, pytest, torch, ScoNetDemo
• Kept first import block (lines 1-15) intact

Rationale

Identical import block appeared twice consecutively—likely from a merge conflict resolution or copy-paste error. No functional impact, but code hygiene improvement.

Verification

• uv run pytest tests/demo/test_pipeline.py -q - PASSED (12 passed, 1 skipped)
• lsp_diagnostics(tests/demo/test_pipeline.py) - No new errors (only pre-existing warnings)

Behavior Preservation

• No test logic modified
• All imports required by tests remain available
• Import order and formatting unchanged

Dependency Configuration: cvmmap-client Local Path Source

Task Summary

Added cvmmap-client as a dependency sourced from local path /home/crosstyan/Code/cvmmap-python-client.

Changes Made

1. pyproject.toml:

   • Added "cvmmap-client" to `[project] dependencies list
   • Added [tool.uv.sources] section with path mapping:

     [tool.uv.sources]
     cvmmap-client = { path = "/home/crosstyan/Code/cvmmap-python-client" }
     

2. uv.lock: Updated automatically via uv lock to include:

   • cvmmap-client v0.1.0 (from file:// path)
   • pyzmq v27.1.0 (transitive dependency)

Verification

• uv lock - PASSED (resolved 104 packages)
• uv run python -c "from cvmmap import CvMmapClient; print('ok')" - PASSED

Key Points

• Package name in provider repo: cvmmap-client (distribution name)
• Import path: from cvmmap import CvMmapClient (module name)
• uv path sources require absolute paths
• Lockfile captures the path dependency for reproducibility

Bugfix: _FrameMetadata NameError in cvmmap_source()

Root Cause

The _FrameMetadata Protocol class was defined inside a TYPE_CHECKING block, making it available only during type checking but not at runtime. However, cvmmap_source() used _FrameMetadata in a runtime function annotation:

async def _async_generator() -> AsyncIterator[tuple[np.ndarray, _FrameMetadata]]:

This caused NameError: name '_FrameMetadata' is not defined when the function was called.

Fix

Moved both _FrameMetadata and _CvMmapClient Protocol classes out of the TYPE_CHECKING block to module level. These protocols are lightweight (just method signatures) and don't require heavy imports, so there's minimal runtime cost.

Verification

• lsp_diagnostics: Only 1 warning (unnecessary type: ignore comment)
• uv run python -c "from opengait.demo.input import cvmmap_source": Import successful
• uv run pytest tests/demo/: 74 passed, 3 skipped
• Runtime test confirms generator creation works without NameError

Pattern to Remember

When using Protocol classes for type annotations:

• If the annotation is used in a nested function (defined inside another function), it needs to be available at runtime
• TYPE_CHECKING-guarded types can only be used in annotations that are evaluated at type-check time
• Module-level function annotations are evaluated at definition time (runtime)

Fix: Window Freeze on No-Detection Frames

Problem

When YOLO doesn't detect any person in a frame, process_frame() returns None. The visualizer update was conditional on viz_payload is not None, causing the OpenCV window to freeze when no detections were present for extended periods.

Solution

Modified ScoliosisPipeline.run() in opengait/demo/pipeline.py to always call self._visualizer.update() when visualization is enabled, even when viz_payload is None. When no detection data is available, the visualizer receives None values for bbox, mask_raw, silhouette, label, and confidence, which it handles gracefully by displaying placeholder views.

Changes Made

1. opengait/demo/pipeline.py (lines 344-393):

   • Changed condition from if self._visualizer is not None and viz_payload is not None: to if self._visualizer is not None:
   • Added else branch to set default values (all None/0) when viz_payload is None
   • Visualizer now receives frame and FPS data every frame, preventing window freeze

2. tests/demo/test_pipeline.py:

   • Added MockVisualizer class to track update calls
   • Added test_pipeline_visualizer_updates_on_no_detection() regression test
   • Test verifies visualizer is called for all frames even when YOLO returns no detections

Verification

• New regression test passes (RED -> GREEN TDD cycle)
• All 13 existing pipeline tests pass (1 skipped)
• No new LSP errors introduced
• Visualizer handles None values gracefully (existing behavior)

Task 1: Stabilize overlapped duplicate code blocks - COMPLETED

Changes Made to opengait/demo/pipeline.py

1. Fixed tuple unpacking in _select_silhouette (lines 181-187):

   • Changed mask_raw, bbox, track_id = selected to mask_raw, bbox_mask, bbox_frame, track_id = selected
   • Updated mask_to_silhouette() call to use bbox_mask (mask coordinate space)
   • Changed return statement to use bbox_frame (frame coordinate space)
   • This fixes the 4-tuple unpacking from select_person() which returns (mask, bbox_mask, bbox_frame, track_id)

2. Added _last_viz_payload attribute:

   • Added _last_viz_payload: dict[str, object] | None class field after _visualizer
   • Initialized to None in __init__ after visualizer initialization

3. Updated visualizer logic in run() for no-detection caching:

   • Changed condition from if self._visualizer is not None and viz_payload is not None: to just if self._visualizer is not None:
   • Added caching logic: when viz_payload is not None, cache a copy with v.copy() if hasattr(v, 'copy') else v for each value
   • Use cached payload when current frame has no detection: viz_data = viz_payload if viz_payload is not None else self._last_viz_payload
   • Handle case when no cache exists: use default None/0 values
   • Visualizer now updates every frame when enabled, preventing window freeze

Verification Results

• LSP diagnostics: No new critical errors (only pre-existing warnings)
• Tests: 14 passed, 1 skipped
• Import test: OK

Key Implementation Detail

The caching logic uses a dict comprehension with .copy() for mutable values (numpy arrays) to prevent mutation issues, satisfying the test requirement that cached masks should be copies, not the same object reference.

Task: BBoxXYXY Type Alias Introduction - COMPLETED

Summary

Introduced and applied BBoxXYXY alias for bbox semantics across demo module files. The alias was already defined in preprocess.py; this task applied it consistently to pipeline.py which had raw tuple[int, int, int, int] annotations.

Changes Made

1. opengait/demo/pipeline.py:
   • Added import: from .preprocess import BBoxXYXY, frame_to_person_mask, mask_to_silhouette
   • Changed _select_silhouette return type annotation (line 176):
     • Before: tuple[int, int, int, int]
     • After: BBoxXYXY
   • Changed fallback cast annotation (line 192):
     • Before: tuple[UInt8[ndarray, "h w"], tuple[int, int, int, int]] | None
     • After: tuple[UInt8[ndarray, "h w"], BBoxXYXY] | None
   • Changed bbox cast in run() method (line 372):
     • Before: cast(tuple[int, int, int, int] | None, bbox_obj)
     • After: cast(BBoxXYXY | None, bbox_obj)

Files Already Using BBoxXYXY (No Changes Needed)

• opengait/demo/window.py: Already imports and uses BBoxXYXY for select_person() return type
• opengait/demo/visualizer.py: Already imports and uses BBoxXYXY for _draw_bbox() and update() parameters
• opengait/demo/preprocess.py: Defines BBoxXYXY = tuple[int, int, int, int] with docstring

Verification

• grep 'tuple\[int, int, int, int\]' in demo/ scope: Only matches alias definition (preprocess.py line 27)
• lsp_diagnostics: No new errors introduced from changes
• uv run pytest tests/demo/test_pipeline.py -q: 14 passed, 1 skipped

Semantic Preservation

• No runtime behavior changes - purely type annotation cleanup
• Mask-space vs frame-space bbox semantics unchanged
• All bbox coordinate calculations preserved exactly

2026-02-28: TypedDict Contract Applied for DemoResult

Changes Made

1. opengait/demo/output.py: create_result() already returns DemoResult TypedDict (confirmed)
2. opengait/demo/pipeline.py:
   • Added DemoResult to imports from .output
   • Changed _result_buffer type from list[dict[str, object]] to list[DemoResult]
   • Fixed pre-existing type error in visualization payload caching by extracting dict comprehension to explicit loop with Callable[[], object] cast

Key Patterns

• TypedDict provides structural typing for dictionary results
• DemoResult fields: frame, track_id, label, confidence, window, timestamp_ns
• Pipeline buffer now enforces consistent result typing throughout the pipeline

Type Safety

• output.py: No diagnostics (clean)
• pipeline.py: Only warnings (no errors) - warnings are pre-existing pyarrow/dynamic import related
• All 14 tests pass

2026-02-28: Visualization Regression Fixes - BBox Scale and Both-Mode Raw Mask

Issue A: BBox Scale Bug in Fallback Path

Problem: In _select_silhouette() fallback path (when select_person() returns None), the bbox from frame_to_person_mask() was in mask-space coordinates but was being drawn on the full-size frame, causing the overlay to appear much smaller than actual.

Root Cause:

• frame_to_person_mask() returns bbox in mask coordinate space (scaled down from original frame)
• Primary path through select_person() correctly returns both bbox_mask and bbox_frame
• Fallback path was returning mask-space bbox directly without conversion

Fix:

• Added frame-space bbox conversion in fallback path using result.orig_shape
• Calculates scale factors: scale_x = frame_w / mask_w, scale_y = frame_h / mask_h
• Applies scaling to bbox coordinates: bbox_frame = (int(bbox_mask[0] * scale_x), ...)
• Safely falls back to mask-space bbox if orig_shape unavailable or dimensions invalid

Code Location: opengait/demo/pipeline.py lines 206-226 in _select_silhouette()

Issue B: Both-Mode Raw Mask Appears Empty

Problem: In _prepare_both_view(), the raw mask pane appeared empty/black in "Both" display mode.

Root Cause:

• Primary path returns mask as float32 with values in [0, 1] range
• Fallback path returns mask as uint8 with values in [0, 255] range
• OpenCV display expects uint8 [0, 255] for proper visualization
• Float32 [0, 1] values displayed as nearly black

Fix:

• Added dtype normalization in _prepare_both_view() before resizing
• Check: if mask_gray.dtype == np.float32 or mask_gray.dtype == np.float64:
• Convert: mask_gray = (mask_gray * 255).astype(np.uint8)
• Ensures consistent uint8 [0, 255] display format regardless of input dtype

Code Location: opengait/demo/visualizer.py lines 330-332 in _prepare_both_view()

Verification

• All 14 pipeline tests pass (1 skipped)
• LSP diagnostics show only pre-existing warnings
• No new tests needed - existing test coverage sufficient

2026-02-28: Tuple-to-TypedDict Migration for select_person and _select_silhouette

Summary

Successfully migrated two function return contracts from anonymous tuples to explicit TypedDict structures:

1. window.select_person() → PersonSelection | None
2. pipeline._select_silhouette() → SilhouetteSelection | None

Key TypedDict Definitions

PersonSelection (in opengait/demo/window.py):

class PersonSelection(TypedDict):
    mask: "Float[ndarray, 'h w']"  # jaxtyping annotated
    bbox_mask: "tuple[int, int, int, int]"
    bbox_frame: "tuple[int, int, int, int]"
    track_id: int

SilhouetteSelection (in opengait/demo/pipeline.py):

class SilhouetteSelection(TypedDict):
    silhouette: "Float[ndarray, '64 44']"  # jaxtyping annotated
    mask_raw: "UInt8[ndarray, 'h w']"     # jaxtyping annotated
    bbox: "BBoxXYXY"
    track_id: int

Migration Pattern

• Replace tuple[X, Y, Z] | None return type with TypedDictName | None
• Replace tuple unpacking at call sites with dict key access
• Update all return statements to return dict literals
• Ensure jaxtyping annotations are preserved on ndarray fields

Files Modified

• opengait/demo/window.py - Added PersonSelection TypedDict, updated select_person
• opengait/demo/pipeline.py - Added SilhouetteSelection TypedDict, updated _select_silhouette
• tests/demo/test_window.py - Updated assertions to use dict keys
• tests/demo/test_pipeline.py - Updated mock return to use dict

Verification

• uv run pytest tests/demo/test_window.py -q → 22 passed, 1 skipped
• uv run pytest tests/demo/test_pipeline.py -q → 14 passed, 1 skipped
• No errors in lsp_diagnostics (only pre-existing warnings)

Behavior Confirmation

No behavior change - purely a typing/contract refactor. All visualization, caching, and bbox scaling logic remains unchanged.

2026-02-28: ProcessFrameResult TypedDict Added

Summary

Added ProcessFrameResult TypedDict for ScoliosisPipeline.process_frame() return contract.

ProcessFrameResult Definition (pipeline.py)

class ProcessFrameResult(TypedDict):
    """Structured return type for process_frame method.

    Contains visualization payload and optional classification result.
    Uses jaxtyping for array fields to preserve shape information.
    """

    mask_raw: "UInt8[ndarray, 'h w'] | None"
    bbox: "BBoxXYXY | None"
    silhouette: "Float[ndarray, '64 44'] | None"
    track_id: int
    label: "str | None"
    confidence: "float | None"
    result: "DemoResult | None"

Files Modified

• opengait/demo/pipeline.py - Added ProcessFrameResult TypedDict, updated process_frame return type

Changes Made

1. Added ProcessFrameResult TypedDict with jaxtyping annotations for array fields
2. Updated process_frame() -> ProcessFrameResult | None
3. Updated all return statements to include result field (None for non-classified paths)
4. Updated _last_viz_payload type from dict[str, object] to ProcessFrameResult
5. Updated cast in run() from dict[str, object] to ProcessFrameResult

Verification

• uv run pytest tests/demo/test_pipeline.py -q → 14 passed, 1 skipped
• No errors in lsp_diagnostics (only pre-existing warnings)

Behavior Confirmation

No behavior change - purely a typing/contract refactor.

2026-02-28: Type Quality Cleanup (Oracle Feedback)

Changes Made

1. window.py - PersonSelection bbox type consistency

   • Changed bbox_mask and bbox_frame types from "tuple[int, int, int, int]" to "BBoxXYXY"
   • Uses the existing type alias for semantic clarity and consistency

2. pipeline.py - run() visualization payload cleanup

   • Replaced generic cached: dict[str, object] with cached: ProcessFrameResult
   • Replaced .get() access with direct key access on ProcessFrameResult
   • Simplified extraction: removed intermediate *_obj variables and casting cascade
   • Maintained copy semantics for mutable numpy arrays (mask_raw, silhouette)

Benefits

• Type-safe caching with explicit ProcessFrameResult structure
• Direct key access eliminates unnecessary .get() calls and type casts
• Consistent use of BBoxXYXY alias across window and pipeline modules

Verification

• uv run pytest tests/demo/test_window.py -q → 22 passed, 1 skipped
• uv run pytest tests/demo/test_pipeline.py -q → 14 passed, 1 skipped
• No runtime behavior changes

2026-02-28: bbox_frame Documentation Added

Documentation Changes

Updated docstrings to clarify coordinate space semantics for bbox fields:

PersonSelection (window.py):

Coordinate spaces:
    - bbox_mask: Bounding box in mask coordinate space (XYXY: x1, y1, x2, y2).
      This is the downsampled mask resolution (e.g., 100x100), used for
      silhouette extraction and mask-space operations.
    - bbox_frame: Bounding box in original frame pixel coordinates (XYXY: x1, y1, x2, y2).
      This is the full-resolution frame size (e.g., 640x480), used for
      visualization overlay on the original video frame.

SilhouetteSelection (pipeline.py):

The bbox field is in original frame pixel coordinates (XYXY: x1, y1, x2, y2),
suitable for visualization overlay on the original video frame.

ProcessFrameResult (pipeline.py):

The bbox field is in original frame pixel coordinates (XYXY: x1, y1, x2, y2),
suitable for visualization overlay on the original video frame.

select_person function (window.py):

- bbox_mask: bounding box in mask coordinate space (XYXY format: x1, y1, x2, y2).
  This is the downsampled mask resolution, used for silhouette extraction.
- bbox_frame: bounding box in original frame pixel coordinates (XYXY format: x1, y1, x2, y2).
  This is the full-resolution frame size, used for visualization overlay.

Key Distinction

• bbox_mask: Mask-space coordinates (downsampled, for processing)
• bbox_frame: Frame-space coordinates (full resolution, for visualization)

Verification

• uv run pytest tests/demo/test_window.py tests/demo/test_pipeline.py -q → 36 passed, 1 skipped

2026-02-28: bbox_frame Documentation in select_person

Docstring Updated (window.py)

Updated select_person function docstring to explicitly document coordinate spaces:

Returns:
    PersonSelection dictionary with keys:
    - mask: the person's segmentation mask
    - bbox_mask: bounding box in mask coordinate space (XYXY format: x1, y1, x2, y2).
      This is the downsampled mask resolution (e.g., 100x100), used for
      silhouette extraction and mask-space operations.
    - bbox_frame: bounding box in original frame pixel coordinates (XYXY format: x1, y1, x2, y2).
      This is the full-resolution frame size (e.g., 640x480), used for
      visualization overlay on the original video frame.
    - track_id: the person's track ID

Key Distinction

• bbox_mask: Mask-space coordinates (downsampled resolution, for processing)
• bbox_frame: Frame-space coordinates (full resolution, for visualization)

Verification

• uv run pytest tests/demo/test_window.py -q → 22 passed

2026-02-28: pkl Alias for silhouette-export-format

Summary

Added pkl as an alias for pickle in --silhouette-export-format CLI option.

Implementation

In opengait/demo/pipeline.py __init__ method:

self._silhouette_export_format = silhouette_export_format
# Normalize format alias: pkl -> pickle
if self._silhouette_export_format == "pkl":
    self._silhouette_export_format = "pickle"

Behavior

• pkl → normalized to pickle → pickle export
• pickle → unchanged → pickle export
• parquet → unchanged → parquet export
• invalid → unchanged → ValueError on export

Verification

• All existing tests pass
• Custom validation test confirms:
  • pkl alias works
  • pickle still works
  • parquet still works
  • invalid formats still rejected