docs(sisyphus): record demo fixes and preprocess research
Capture validated debugging outcomes and ScoNet preprocessing findings in persistent notes so future sessions can resume with verified context instead of redoing the same investigation.
This commit is contained in:
@@ -110,4 +110,191 @@ Updated opengait/demo/__main__.py to restore behavior parity:
|
||||
### Verification
|
||||
- `uv run pytest tests/demo/test_pipeline.py -q` -> 12 passed, 1 skipped
|
||||
- `uv run python -m opengait.demo --help` -> --visualize flag present
|
||||
- lsp_diagnostics -> only acceptable warnings (reportAny, unused call results)
|
||||
- lsp_diagnostics -> only acceptable warnings (reportAny, unused call results)
|
||||
## Fixed: NameError for _FrameMetadata in input.py
|
||||
|
||||
**Status:** RESOLVED
|
||||
|
||||
**Error:**
|
||||
```
|
||||
NameError: name '_FrameMetadata' is not defined
|
||||
File: opengait/demo/input.py
|
||||
Location: cvmmap_source() nested async generator return annotation
|
||||
```
|
||||
|
||||
**Resolution:**
|
||||
Moved `_FrameMetadata` and `_CvMmapClient` Protocol classes from inside `if TYPE_CHECKING:` block to module level in `opengait/demo/input.py`.
|
||||
|
||||
**Verification:**
|
||||
- Import test passes
|
||||
- All 74 demo tests pass (3 skipped)
|
||||
- Runtime test confirms `cvmmap_source()` returns generator without NameError
|
||||
|
||||
## Fixed: Window Freeze on No-Detection Frames
|
||||
|
||||
**Status:** RESOLVED
|
||||
|
||||
**Issue:** When running demo with `--visualize` flag on video streams with intermittent or no person detections, the OpenCV window would freeze because `viz_payload` was `None` when no detection occurred, and the visualizer update was skipped.
|
||||
|
||||
**Root Cause:** In `ScoliosisPipeline.run()`, the visualizer update was conditional on both `self._visualizer is not None` AND `viz_payload is not None`. When YOLO returned no detections, `process_frame()` returned `None`, skipping the visualizer update entirely.
|
||||
|
||||
**Fix:** Modified the condition to update visualizer whenever it's enabled, passing `None` values for missing detection data. The `OpenCVVisualizer.update()` method already handles `None` values gracefully by displaying placeholder views.
|
||||
|
||||
**Files Changed:**
|
||||
- `opengait/demo/pipeline.py`: Lines 344-393
|
||||
- `tests/demo/test_pipeline.py`: Added `MockVisualizer` class and `test_pipeline_visualizer_updates_on_no_detection()`
|
||||
|
||||
**Testing:**
|
||||
- New regression test: `test_pipeline_visualizer_updates_on_no_detection()` - PASSED
|
||||
- All existing pipeline tests: 13 passed, 1 skipped
|
||||
- LSP diagnostics: No new errors
|
||||
|
||||
## Task 1: Stabilize overlapped duplicate code blocks - RESOLVED
|
||||
|
||||
### Issues Fixed
|
||||
|
||||
1. **Tuple unpacking mismatch in _select_silhouette**:
|
||||
- Issue: `select_person()` returns 4-tuple `(mask, bbox_mask, bbox_frame, track_id)`, but code was unpacking as 3-tuple
|
||||
- Resolution: Updated unpacking to handle all 4 values and use correct bbox for mask_to_silhouette vs return value
|
||||
|
||||
2. **Window freeze on no-detection frames**:
|
||||
- Issue: Visualizer only updated when `viz_payload is not None`, causing window freeze when no person detected
|
||||
- Resolution: Update visualizer every frame when enabled, using cached payload for no-detection frames
|
||||
|
||||
3. **Cached payload mutation risk**:
|
||||
- Issue: Storing direct reference to viz_payload could allow mutation of cached data
|
||||
- Resolution: Create copies of mutable values (numpy arrays) when caching using `v.copy() if hasattr(v, 'copy')`
|
||||
|
||||
### Pre-existing Issues (Not Fixed)
|
||||
- LSP type errors with `DemoResult` vs `dict[str, object]` in `_result_buffer` (pre-existing)
|
||||
- PyArrow type warnings (pre-existing, acceptable)
|
||||
|
||||
### Tools Used
|
||||
- sed for targeted line replacements
|
||||
- Python script for complex multi-line block replacement
|
||||
- Native Edit tool for simple single-line changes
|
||||
- pytest for verification
|
||||
|
||||
|
||||
## Task: BBoxXYXY Type Alias Introduction - COMPLETED
|
||||
|
||||
### Scope
|
||||
Type-semantics cleanup only. No new features, no behavior changes.
|
||||
|
||||
### Changes Summary
|
||||
- **opengait/demo/pipeline.py**: 3 occurrences of `tuple[int, int, int, int]` converted to `BBoxXYXY`
|
||||
- Import statement updated to include BBoxXYXY from .preprocess
|
||||
- Function return type annotation in `_select_silhouette()`
|
||||
- Cast expression type annotation in `run()` method
|
||||
|
||||
### No Issues Encountered
|
||||
- All other scoped files (window.py, visualizer.py, preprocess.py) already used BBoxXYXY correctly
|
||||
- Tests pass without modification
|
||||
- No semantic regressions detected
|
||||
|
||||
### Pre-existing Conditions Preserved
|
||||
- LSP warnings in pipeline.py (DemoResult type mismatch, PyArrow Any types) - unchanged
|
||||
- LSP warnings in window.py (torch/numpy Any types) - unchanged
|
||||
- LSP warnings in visualizer.py (cv2.Any types) - unchanged
|
||||
|
||||
All warnings existed before this task and are unrelated to bbox type annotations.
|
||||
|
||||
## 2026-02-28: TypedDict Contract Application
|
||||
|
||||
### Pre-existing Issues Encountered
|
||||
1. **Visualization payload type error** (line 353): Dictionary comprehension with `hasattr(v, 'copy')` check caused basedpyright error because type checker cannot verify attribute existence through runtime introspection
|
||||
- Fixed by extracting to explicit for-loop with `Callable[[], object]` cast pattern
|
||||
|
||||
2. **Type narrowing in comprehensions**: Type narrowing (`is not None` checks) doesn't propagate into dictionary comprehensions, requiring explicit cast before the comprehension
|
||||
|
||||
### Remaining Warnings (Pre-existing, Not Related to Task)
|
||||
- pyarrow types show as `Any` (dynamic import)
|
||||
- Some `list.append()` methods show as partially unknown
|
||||
- These are in export code paths unrelated to DemoResult typing
|
||||
|
||||
## Fixed: BBox Scale Bug in Fallback Path (2026-02-28)
|
||||
|
||||
**Status:** RESOLVED
|
||||
|
||||
**Issue:** In `_select_silhouette()` fallback path, bbox returned from `frame_to_person_mask()` was in mask-space coordinates, causing visualization overlay to appear at wrong scale.
|
||||
|
||||
**Resolution:**
|
||||
Added coordinate space conversion using `result.orig_shape` to scale bbox from mask-space to frame-space:
|
||||
- Extract frame dimensions from `result.orig_shape` with safe fallback
|
||||
- Calculate scale factors between mask and frame dimensions
|
||||
- Apply scaling to bbox coordinates for frame-space positioning
|
||||
- Graceful fallback to mask-space bbox if conversion not possible
|
||||
|
||||
**Files Changed:**
|
||||
- `opengait/demo/pipeline.py`: Lines 206-226 in `_select_silhouette()` method
|
||||
|
||||
**Verification:**
|
||||
- `uv run pytest tests/demo/test_pipeline.py -q` - PASSED (14 passed, 1 skipped)
|
||||
- LSP diagnostics: No new errors
|
||||
|
||||
## Fixed: Both-Mode Raw Mask Empty Display (2026-02-28)
|
||||
|
||||
**Status:** RESOLVED
|
||||
|
||||
**Issue:** In `_prepare_both_view()`, raw mask pane appeared empty because primary path returns float32 [0,1] mask while display expects uint8 [0,255].
|
||||
|
||||
**Resolution:**
|
||||
Added dtype normalization to ensure mask is always uint8 before display:
|
||||
- Check if mask is float32/float64 dtype
|
||||
- If so, scale by 255 and convert to uint8
|
||||
- Handles both primary path (float) and fallback path (uint8) consistently
|
||||
|
||||
**Files Changed:**
|
||||
- `opengait/demo/visualizer.py`: Lines 330-332 in `_prepare_both_view()` method
|
||||
|
||||
**Verification:**
|
||||
- `uv run pytest tests/demo/test_pipeline.py -q` - PASSED (14 passed, 1 skipped)
|
||||
- LSP diagnostics: No new errors
|
||||
|
||||
|
||||
|
||||
## 2026-02-28: Tuple-to-TypedDict Migration Issues
|
||||
|
||||
### Regressions Encountered
|
||||
|
||||
1. **Duplicate code blocks in pipeline.py**
|
||||
- Root cause: Incremental edits left stale tuple unpacking alongside new dict access
|
||||
- Fix: Used Python script for atomic replacement instead of multiple small edits
|
||||
|
||||
2. **Stale tuple assertions in test_window.py**
|
||||
- Root cause: Tests had both dict assertions AND tuple unpacking/assertions
|
||||
- Fix: Systematically removed all `assert mask...`, `assert bbox...`, `assert tid...` lines
|
||||
|
||||
3. **Mock return value overwrite in test_pipeline.py**
|
||||
- Root cause: Dict mock was immediately overwritten by tuple mock
|
||||
- Fix: Removed the second tuple mock assignment
|
||||
|
||||
### Lessons Learned
|
||||
|
||||
- Use Python scripts for complex multi-line replacements to avoid file corruption
|
||||
- Always grep for ALL tuple unpacking patterns (`a, b = result`, `a, b, c = result`, etc.)
|
||||
- Tests may have BOTH dict and tuple assertions - must remove both
|
||||
- Mock return values may be defined twice - check for overwrites
|
||||
|
||||
### Pre-existing Issues (not introduced)
|
||||
|
||||
- `BBoxXYXY` import unused in window.py (pre-existing)
|
||||
- Various pyright warnings about Any types (pre-existing)
|
||||
- Unused imports in pipeline.py (pre-existing)
|
||||
|
||||
|
||||
## 2026-02-28: bbox_frame Documentation (Minor)
|
||||
|
||||
### Issue
|
||||
User requested clarification of `bbox_frame` semantics in docstrings.
|
||||
|
||||
### Resolution
|
||||
Updated `select_person` docstring in `window.py` to explicitly document:
|
||||
- `bbox_mask`: mask-space XYXY (downsampled, for processing)
|
||||
- `bbox_frame`: frame-space XYXY (full resolution, for visualization)
|
||||
|
||||
### Files Changed
|
||||
- `opengait/demo/window.py` - Docstring only
|
||||
|
||||
### Note
|
||||
The git checkout restored the tuple return; updated to dict return to match TypedDict contract.
|
||||
|
||||
Reference in New Issue
Block a user