Add resumable ScoNet skeleton training diagnostics

This commit is contained in:
2026-03-09 15:57:13 +08:00
parent 4e0b0a18dc
commit 36aef46a0d
15 changed files with 1226 additions and 44 deletions
+28
View File
@@ -75,6 +75,7 @@ The silhouette and skeleton-map pipelines are different experiments and should n
* `Scoliosis1K-sil-pkl` is the silhouette modality used by the standard ScoNet configs.
* pose-derived heatmap roots such as `Scoliosis1K_sigma_8.0/pkl` or DRF exports are skeleton-map inputs and require `in_channel: 2`.
* DRF does **not** use the silhouette stream as an input. It uses `0_heatmap.pkl` plus `1_pav.pkl`.
Naming note:
@@ -89,6 +90,18 @@ A strong silhouette checkpoint does not validate the skeleton-map path. In parti
So if you are debugging DRF or `ScoNet-MT-ske` reproduction, do not use `ScoNet-20000-better.pt` as evidence that the heatmap preprocessing is correct.
### Overlay caveat
Do not treat a direct overlay between `Scoliosis1K-sil-pkl` and pose-derived skeleton maps as a valid alignment test.
Reason:
* the released silhouette modality is an estimated segmentation output from `PP-HumanSeg v2`
* the released pose modality is an estimated keypoint output from `ViTPose`
* the two modalities are normalized by different preprocessing pipelines before they reach OpenGait
So a silhouette-vs-skeleton mismatch in a debug figure is usually a cross-modality frame-of-reference issue, not proof that the raw dataset is bad. The more important check for skeleton-map debugging is whether the **limb and joint channels align with each other** inside `0_heatmap.pkl`.
---
## Pose-to-Heatmap Conversion
@@ -146,6 +159,21 @@ If you explicitly want train-only PAV min-max statistics, add:
--stats_partition=./datasets/Scoliosis1K/Scoliosis1K_118.json
```
### Heatmap debugging notes
Current confirmed findings from local debugging:
* the raw pose dataset itself looks healthy; poor `ScoNet-MT-ske` results are not explained by obvious missing-joint collapse
* a larger heatmap sigma can materially blur away the articulated structure; `sigma=8` was much broader than the silhouette geometry, while smaller sigma values recovered more structure
* an earlier bug aligned the limb and joint channels separately; that made the two channels of `0_heatmap.pkl` slightly misregistered
* the heatmap path is now patched so limb and joint channels share one alignment crop
Remaining caution:
* the exported skeleton map is stored as `64x64`
* if the runtime config uses `BaseSilCuttingTransform`, the network actually sees `64x44`
* that symmetric left/right crop is not automatically wrong, but it is still a meaningful ablation point for skeleton-map experiments
The output layout is:
```text