# Work Plan: Real-time OpenCV Visualizer for Demo

## Objective
Add a real-time OpenCV-based visualizer to the `opengait/demo` pipeline. It will display the live stream with bounding boxes, tracking IDs, FPS, label, and confidence, alongside a window for the segmentation result. The user can toggle between the raw high-res segmentation mask, the normalized 64x44 silhouette, or both side-by-side using keyboard controls.

## Scope
- **IN**: Create `visualizer.py`, update `pipeline.py` to calculate rolling FPS and extract visualization payloads, update `__main__.py` with a `--visualize` CLI flag.
- **OUT**: Modifying underlying ScoNet, data loaders, or publishers.

## Guardrails & Assumptions
- **Graceful degradation**: Handle frames where no person is detected (bbox/mask is None) without crashing.
- **Cleanup**: Ensure `cv2.destroyAllWindows()` is called when the pipeline stops or crashes.
- **Non-blocking**: `cv2.waitKey(1)` must be used to keep the pipeline moving in real-time.

## Tasks

### Task 1: Add CLI Flag in `__main__.py`
- Modify `opengait/demo/__main__.py`.
- Add `--visualize` boolean argument to the `argparse` configuration (`action="store_true"`).
- Pass `visualize=args.visualize` to the `ScoliosisPipeline` constructor.

### Task 2: Create `visualizer.py`
- Create `opengait/demo/visualizer.py` and implement `class OpenCVVisualizer`.
- Add an internal state variable `self.mask_mode = 0` (0: Both, 1: Raw, 2: Normalized).
- Add method `update(frame, bbox, track_id, mask_raw, silhouette, label, confidence, fps)`.
- **Main Window**: Draw `bbox` (if exists), put text for `track_id`, `fps`, `label`, and `confidence`. Use `cv2.imshow('OpenGait Stream', frame)`.
- **Segmented Window**: Display `mask_raw`, `silhouette` (upscaled), or a side-by-side combination based on `self.mask_mode`. Use `cv2.imshow('Segmented Result', segmented_display)`.
- Include `key = cv2.waitKey(1)`. 
  - If `key == ord('m')`, increment `self.mask_mode` (modulo 3) to toggle the mask view.
  - If `key == ord('q')`, return `False` to signal pipeline exit.
- Add a `close()` method calling `cv2.destroyAllWindows()`.

### Task 3: Update `pipeline.py` for Data Extraction & FPS
- Modify `ScoliosisPipeline.__init__` to accept `visualize: bool = False` and conditionally instantiate `self.visualizer = OpenCVVisualizer()`.
- Update FPS logic in `run()`: Calculate an EMA (Exponential Moving Average) FPS per frame for real-time overlay.
- Modify `_select_silhouette` to return `(silhouette, mask_raw, bbox)` so the raw mask and bbox are preserved for visualization.
- In `process_frame`, gather and return `mask_raw`, `bbox`, `silhouette`, `track_id`, `label`, `confidence` to the main loop (either in the result dict or as a separate return value).
- In the `run()` loop, after processing, extract these values and call `self.visualizer.update(...)`. Break loop if it returns `False`.
- Add a `try/finally` block in `run()` to ensure `self.visualizer.close()` is called.

## Final Verification Wave
- Run the demo with `--visualize` on a sample video.
- Verify both windows appear and update in real-time.
- Verify FPS, tracking ID, label, and confidence are rendered correctly on the main window.
- Verify pressing 'm' toggles the segmented window between Both, Raw, and Normalized views.
- Verify pressing 'q' closes the windows and exits the pipeline cleanly.