Add resumable ScoNet skeleton training diagnostics
This commit is contained in:
@@ -75,6 +75,7 @@ The silhouette and skeleton-map pipelines are different experiments and should n
|
||||
|
||||
* `Scoliosis1K-sil-pkl` is the silhouette modality used by the standard ScoNet configs.
|
||||
* pose-derived heatmap roots such as `Scoliosis1K_sigma_8.0/pkl` or DRF exports are skeleton-map inputs and require `in_channel: 2`.
|
||||
* DRF does **not** use the silhouette stream as an input. It uses `0_heatmap.pkl` plus `1_pav.pkl`.
|
||||
|
||||
Naming note:
|
||||
|
||||
@@ -89,6 +90,18 @@ A strong silhouette checkpoint does not validate the skeleton-map path. In parti
|
||||
|
||||
So if you are debugging DRF or `ScoNet-MT-ske` reproduction, do not use `ScoNet-20000-better.pt` as evidence that the heatmap preprocessing is correct.
|
||||
|
||||
### Overlay caveat
|
||||
|
||||
Do not treat a direct overlay between `Scoliosis1K-sil-pkl` and pose-derived skeleton maps as a valid alignment test.
|
||||
|
||||
Reason:
|
||||
|
||||
* the released silhouette modality is an estimated segmentation output from `PP-HumanSeg v2`
|
||||
* the released pose modality is an estimated keypoint output from `ViTPose`
|
||||
* the two modalities are normalized by different preprocessing pipelines before they reach OpenGait
|
||||
|
||||
So a silhouette-vs-skeleton mismatch in a debug figure is usually a cross-modality frame-of-reference issue, not proof that the raw dataset is bad. The more important check for skeleton-map debugging is whether the **limb and joint channels align with each other** inside `0_heatmap.pkl`.
|
||||
|
||||
---
|
||||
|
||||
## Pose-to-Heatmap Conversion
|
||||
@@ -146,6 +159,21 @@ If you explicitly want train-only PAV min-max statistics, add:
|
||||
--stats_partition=./datasets/Scoliosis1K/Scoliosis1K_118.json
|
||||
```
|
||||
|
||||
### Heatmap debugging notes
|
||||
|
||||
Current confirmed findings from local debugging:
|
||||
|
||||
* the raw pose dataset itself looks healthy; poor `ScoNet-MT-ske` results are not explained by obvious missing-joint collapse
|
||||
* a larger heatmap sigma can materially blur away the articulated structure; `sigma=8` was much broader than the silhouette geometry, while smaller sigma values recovered more structure
|
||||
* an earlier bug aligned the limb and joint channels separately; that made the two channels of `0_heatmap.pkl` slightly misregistered
|
||||
* the heatmap path is now patched so limb and joint channels share one alignment crop
|
||||
|
||||
Remaining caution:
|
||||
|
||||
* the exported skeleton map is stored as `64x64`
|
||||
* if the runtime config uses `BaseSilCuttingTransform`, the network actually sees `64x44`
|
||||
* that symmetric left/right crop is not automatically wrong, but it is still a meaningful ablation point for skeleton-map experiments
|
||||
|
||||
The output layout is:
|
||||
|
||||
```text
|
||||
|
||||
Reference in New Issue
Block a user