crosstyan/OpenGait
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(demo): add export and silhouette visualization outputs

Browse Source 
Add preprocess-only silhouette export and configurable result exporters so demo runs can be persisted for offline analysis and reproducible evaluation. Include optional parquet support and CLI visualization dumps while updating tests and tracking notes for the verified pipeline/debug workflow.
This commit is contained in:
crosstyan
2026-02-27 17:16:20 +08:00
parent 3496a1beb7
commit f501119d43
10 changed files with 1101 additions and 217 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.sisyphus/plans/sconet-pipeline.md
+62 -62
View File
@@ -80,10 +80,10 @@ Create a self-contained scoliosis screening pipeline that runs standalone (no DD
- `tests/demo/test_pipeline.py` — Integration / smoke tests
### Definition of Done
- [ ] `uv run python -m opengait.demo --source ./assets/sample.mp4 --checkpoint ./ckpt/ScoNet-20000.pt --max-frames 120` exits 0 and prints predictions (no NATS by default when `--nats-url` not provided)
- [ ] `uv run pytest tests/demo/ -q` passes all tests
- [ ] Pipeline processes ≥15 FPS on desktop GPU with 720p input
- [ ] JSON schema validated: `{"frame": int, "track_id": int, "label": str, "confidence": float, "window": int, "timestamp_ns": int}`
- [x] `uv run python -m opengait.demo --source ./assets/sample.mp4 --checkpoint ./ckpt/ScoNet-20000.pt --max-frames 120` exits 0 and prints predictions (no NATS by default when `--nats-url` not provided)
- [x] `uv run pytest tests/demo/ -q` passes all tests
- [x] Pipeline processes ≥15 FPS on desktop GPU with 720p input
- [x] JSON schema validated: `{"frame": int, "track_id": int, "label": str, "confidence": float, "window": int, "timestamp_ns": int}`
### Must Have
- Deterministic preprocessing matching ScoNet training data exactly (64×44, float32, [0,1])
@@ -245,11 +245,11 @@ Max Concurrent: 4 (Waves 1 & 2)
- `opengait/modeling/models/__init__.py`: Shows the repo's package init convention (dynamic imports vs empty)
**Acceptance Criteria**:
- [ ] `opengait/demo/__init__.py` exists
- [ ] `opengait/demo/__main__.py` exists with stub entry point
- [ ] `tests/demo/conftest.py` exists with at least one fixture
- [ ] `uv sync` succeeds without errors
- [ ] `uv run python -c "import ultralytics; import nats; import jaxtyping; import beartype; import click; print('OK')"` prints OK
- [x] `opengait/demo/__init__.py` exists
- [x] `opengait/demo/__main__.py` exists with stub entry point
- [x] `tests/demo/conftest.py` exists with at least one fixture
- [x] `uv sync` succeeds without errors
- [x] `uv run python -c "import ultralytics; import nats; import jaxtyping; import beartype; import click; print('OK')"` prints OK
**QA Scenarios:**
@@ -354,10 +354,10 @@ Max Concurrent: 4 (Waves 1 & 2)
- `sconet_scoliosis1k.yaml`: Contains the exact hyperparams (channels, num_parts, etc.) for building layers
**Acceptance Criteria**:
- [ ] `opengait/demo/sconet_demo.py` exists with `ScoNetDemo(nn.Module)` class
- [ ] No `torch.distributed` imports in the file
- [ ] `ScoNetDemo` does not inherit from `BaseModel`
- [ ] `uv run python -c "from opengait.demo.sconet_demo import ScoNetDemo; print('OK')"` works
- [x] `opengait/demo/sconet_demo.py` exists with `ScoNetDemo(nn.Module)` class
- [x] No `torch.distributed` imports in the file
- [x] `ScoNetDemo` does not inherit from `BaseModel`
- [x] `uv run python -c "from opengait.demo.sconet_demo import ScoNetDemo; print('OK')"` works
**QA Scenarios:**
@@ -455,9 +455,9 @@ Max Concurrent: 4 (Waves 1 & 2)
- Ultralytics masks: Need to know exact API to extract binary masks from YOLO output
**Acceptance Criteria**:
- [ ] `opengait/demo/preprocess.py` exists
- [ ] `mask_to_silhouette()` returns `np.ndarray` of shape `(64, 44)` dtype `float32` with values in `[0, 1]`
- [ ] Returns `None` for masks below MIN_MASK_AREA
- [x] `opengait/demo/preprocess.py` exists
- [x] `mask_to_silhouette()` returns `np.ndarray` of shape `(64, 44)` dtype `float32` with values in `[0, 1]`
- [x] Returns `None` for masks below MIN_MASK_AREA
**QA Scenarios:**
@@ -573,11 +573,11 @@ Max Concurrent: 4 (Waves 1 & 2)
- `test_cvmmap.py`: Shows the canonical consumer pattern we must wrap
**Acceptance Criteria**:
- [ ] `opengait/demo/input.py` exists with `opencv_source`, `cvmmap_source`, `create_source` as functions (not classes)
- [ ] `create_source('./some/video.mp4')` returns a generator/iterable
- [ ] `create_source('cvmmap://default')` returns a generator (or raises if cv-mmap not installed)
- [ ] `create_source('0')` returns a generator for camera index 0
- [ ] Any custom generator `def my_source(): yield (frame, meta)` can be used directly by the pipeline
- [x] `opengait/demo/input.py` exists with `opencv_source`, `cvmmap_source`, `create_source` as functions (not classes)
- [x] `create_source('./some/video.mp4')` returns a generator/iterable
- [x] `create_source('cvmmap://default')` returns a generator (or raises if cv-mmap not installed)
- [x] `create_source('0')` returns a generator for camera index 0
- [x] Any custom generator `def my_source(): yield (frame, meta)` can be used directly by the pipeline
**QA Scenarios:**
@@ -691,11 +691,11 @@ Max Concurrent: 4 (Waves 1 & 2)
- Ultralytics API: Need to handle `None` track IDs and extract correct tensors
**Acceptance Criteria**:
- [ ] `opengait/demo/window.py` exists with `SilhouetteWindow` class and `select_person` function
- [ ] Buffer is bounded (deque with maxlen)
- [ ] `get_tensor()` returns shape `[1, 1, 30, 64, 44]` when full
- [ ] Track ID change triggers reset
- [ ] Gap exceeding threshold triggers reset
- [x] `opengait/demo/window.py` exists with `SilhouetteWindow` class and `select_person` function
- [x] Buffer is bounded (deque with maxlen)
- [x] `get_tensor()` returns shape `[1, 1, 30, 64, 44]` when full
- [x] Track ID change triggers reset
- [x] Gap exceeding threshold triggers reset
**QA Scenarios:**
@@ -807,10 +807,10 @@ Max Concurrent: 4 (Waves 1 & 2)
- cv-mmap-gui: Confirms NATS is the right transport for this ecosystem
**Acceptance Criteria**:
- [ ] `opengait/demo/output.py` exists with `ConsolePublisher`, `NatsPublisher`, `create_publisher`
- [ ] ConsolePublisher prints valid JSON to stdout
- [ ] NatsPublisher connects and publishes without crashing (when NATS available)
- [ ] NatsPublisher logs warning and doesn't crash when NATS unavailable
- [x] `opengait/demo/output.py` exists with `ConsolePublisher`, `NatsPublisher`, `create_publisher`
- [x] ConsolePublisher prints valid JSON to stdout
- [x] NatsPublisher connects and publishes without crashing (when NATS available)
- [x] NatsPublisher logs warning and doesn't crash when NATS unavailable
**QA Scenarios:**
@@ -901,9 +901,9 @@ Max Concurrent: 4 (Waves 1 & 2)
- `BaseSilCuttingTransform`: Defines the 64→44 cut + /255 contract we must match
**Acceptance Criteria**:
- [ ] `tests/demo/test_preprocess.py` exists with ≥5 test cases
- [ ] `uv run pytest tests/demo/test_preprocess.py -q` passes
- [ ] Tests cover: valid mask, tiny mask, empty mask, determinism
- [x] `tests/demo/test_preprocess.py` exists with ≥5 test cases
- [x] `uv run pytest tests/demo/test_preprocess.py -q` passes
- [x] Tests cover: valid mask, tiny mask, empty mask, determinism
**QA Scenarios:**
@@ -995,9 +995,9 @@ Max Concurrent: 4 (Waves 1 & 2)
- `evaluator.py`: Defines expected prediction behavior (argmax of mean logits)
**Acceptance Criteria**:
- [ ] `tests/demo/test_sconet_demo.py` exists with ≥4 test cases
- [ ] `uv run pytest tests/demo/test_sconet_demo.py -q` passes
- [ ] Tests cover: construction, forward shape, predict output, no-DDP enforcement
- [x] `tests/demo/test_sconet_demo.py` exists with ≥4 test cases
- [x] `uv run pytest tests/demo/test_sconet_demo.py -q` passes
- [x] Tests cover: construction, forward shape, predict output, no-DDP enforcement
**QA Scenarios:**
@@ -1106,10 +1106,10 @@ Max Concurrent: 4 (Waves 1 & 2)
- Ultralytics: The YOLO `.track()` call is the only external API used directly in this file
**Acceptance Criteria**:
- [ ] `opengait/demo/pipeline.py` exists with `ScoliosisPipeline` class
- [ ] `opengait/demo/__main__.py` exists with click CLI
- [ ] `uv run python -m opengait.demo --help` prints usage without errors
- [ ] All public methods have jaxtyping annotations where tensor/array args are involved
- [x] `opengait/demo/pipeline.py` exists with `ScoliosisPipeline` class
- [x] `opengait/demo/__main__.py` exists with click CLI
- [x] `uv run python -m opengait.demo --help` prints usage without errors
- [x] All public methods have jaxtyping annotations where tensor/array args are involved
**QA Scenarios:**
@@ -1146,7 +1146,7 @@ Max Concurrent: 4 (Waves 1 & 2)
- Files: `opengait/demo/pipeline.py`, `opengait/demo/__main__.py`
- Pre-commit: `uv run python -m opengait.demo --help`
- [ ] 10. Unit Tests — Single-Person Policy + Window Reset
- [x] 10. Unit Tests — Single-Person Policy + Window Reset
**What to do**:
- Create `tests/demo/test_window.py`
@@ -1188,8 +1188,8 @@ Max Concurrent: 4 (Waves 1 & 2)
- Direct test target
**Acceptance Criteria**:
- [ ] `tests/demo/test_window.py` exists with ≥6 test cases
- [ ] `uv run pytest tests/demo/test_window.py -q` passes
- [x] `tests/demo/test_window.py` exists with ≥6 test cases
- [x] `uv run pytest tests/demo/test_window.py -q` passes
**QA Scenarios:**
@@ -1208,7 +1208,7 @@ Max Concurrent: 4 (Waves 1 & 2)
- Files: `tests/demo/test_window.py`
- Pre-commit: `uv run pytest tests/demo/test_window.py -q`
- [ ] 11. Sample Video for Smoke Testing
- [x] 11. Sample Video for Smoke Testing
**What to do**:
- Acquire or create a short sample video for pipeline smoke testing
@@ -1278,7 +1278,7 @@ Max Concurrent: 4 (Waves 1 & 2)
---
- [ ] 12. Integration Tests — End-to-End Smoke Test
- [x] 12. Integration Tests — End-to-End Smoke Test
**What to do**:
- Create `tests/demo/test_pipeline.py`
@@ -1320,9 +1320,9 @@ Max Concurrent: 4 (Waves 1 & 2)
- `output.py`: Need JSON schema to assert against
**Acceptance Criteria**:
- [ ] `tests/demo/test_pipeline.py` exists with ≥4 test cases
- [ ] `CUDA_VISIBLE_DEVICES=0 uv run pytest tests/demo/test_pipeline.py -q` passes
- [ ] Tests cover: happy path, max-frames, invalid source, invalid checkpoint
- [x] `tests/demo/test_pipeline.py` exists with ≥4 test cases
- [x] `CUDA_VISIBLE_DEVICES=0 uv run pytest tests/demo/test_pipeline.py -q` passes
- [x] Tests cover: happy path, max-frames, invalid source, invalid checkpoint
**QA Scenarios:**
@@ -1367,7 +1367,7 @@ Max Concurrent: 4 (Waves 1 & 2)
- Files: `tests/demo/test_pipeline.py`
- Pre-commit: `CUDA_VISIBLE_DEVICES=0 uv run pytest tests/demo/test_pipeline.py -q`
- [ ] 13. NATS Integration Test
- [x] 13. NATS Integration Test
**What to do**:
- Create `tests/demo/test_nats.py`
@@ -1418,9 +1418,9 @@ Max Concurrent: 4 (Waves 1 & 2)
- nats-py: Need subscriber API to consume and validate messages
**Acceptance Criteria**:
- [ ] `tests/demo/test_nats.py` exists with ≥2 test cases
- [ ] Tests are skippable when Docker/NATS not available
- [ ] `CUDA_VISIBLE_DEVICES=0 uv run pytest tests/demo/test_nats.py -q` passes (when Docker available)
- [x] `tests/demo/test_nats.py` exists with ≥2 test cases
- [x] Tests are skippable when Docker/NATS not available
- [x] `CUDA_VISIBLE_DEVICES=0 uv run pytest tests/demo/test_nats.py -q` passes (when Docker available)
**QA Scenarios:**
@@ -1457,19 +1457,19 @@ Max Concurrent: 4 (Waves 1 & 2)
> 4 review agents run in PARALLEL. ALL must APPROVE. Rejection → fix → re-run.
- [ ] F1. **Plan Compliance Audit** — `oracle`
- [x] F1. **Plan Compliance Audit** — `oracle`
Read the plan end-to-end. For each "Must Have": verify implementation exists (read file, run command). For each "Must NOT Have": search codebase for forbidden patterns (torch.distributed imports in demo/, BaseModel subclassing). Check evidence files exist in .sisyphus/evidence/. Compare deliverables against plan.
Output: `Must Have [N/N] | Must NOT Have [N/N] | Tasks [N/N] | VERDICT: APPROVE/REJECT`
- [ ] F2. **Code Quality Review** — `unspecified-high`
- [x] F2. **Code Quality Review** — `unspecified-high`
Run linter + `uv run pytest tests/demo/ -q`. Review all new files in `opengait/demo/` for: `as any`/type:ignore, empty catches, print statements used instead of logging, commented-out code, unused imports. Check AI slop: excessive comments, over-abstraction, generic variable names.
Output: `Tests [N pass/N fail] | Files [N clean/N issues] | VERDICT`
- [ ] F3. **Real Manual QA** — `unspecified-high`
- [x] F3. **Real Manual QA** — `unspecified-high`
Start from clean state. Run pipeline with sample video: `uv run python -m opengait.demo --source ./assets/sample.mp4 --checkpoint ./ckpt/ScoNet-20000.pt --max-frames 120`. Verify predictions are printed to console (no `--nats-url` = console output). Run with NATS: start container, run pipeline with `--nats-url nats://127.0.0.1:4222`, subscribe and validate JSON schema. Test edge cases: missing video file (graceful error), no checkpoint (graceful error), --help flag.
Output: `Scenarios [N/N pass] | Edge Cases [N tested] | VERDICT`
- [ ] F4. **Scope Fidelity Check** — `deep`
- [x] F4. **Scope Fidelity Check** — `deep`
For each task: read "What to do", read actual files created. Verify 1:1 — everything in spec was built (no missing), nothing beyond spec was built (no creep). Check "Must NOT do" compliance: no torch.distributed in demo/, no BaseModel subclass, no TensorRT code, no multi-person logic. Flag unaccounted changes.
Output: `Tasks [N/N compliant] | Scope [CLEAN/N issues] | VERDICT`
@@ -1506,9 +1506,9 @@ uv run python -m opengait.demo --help
```
### Final Checklist
- [ ] All "Must Have" present
- [ ] All "Must NOT Have" absent
- [ ] All tests pass
- [ ] Pipeline runs at ≥15 FPS on desktop GPU
- [ ] JSON schema matches spec
- [ ] No torch.distributed imports in opengait/demo/
- [x] All "Must Have" present
- [x] All "Must NOT Have" absent
- [x] All tests pass
- [x] Pipeline runs at ≥15 FPS on desktop GPU
- [x] JSON schema matches spec
- [x] No torch.distributed imports in opengait/demo/