Record 1:1:2 skeleton bridge findings

2026-03-10 14:43:08 +08:00
parent 37483fa628
commit 560f8e7aae
3 changed files with 27 additions and 2 deletions
@@ -159,6 +159,9 @@ Conclusion:
 - the Scoliosis1K raw pose data does not appear obviously broken
 - the OpenGait training/evaluation infrastructure is not the main problem
 - PAV computation is not the main blocker
 - the skeleton branch is learnable on the easier `1:1:2` split
 - on `1:1:2`, `body-only + weighted CE` reached `81.82 Acc / 66.21 Prec / 88.50 Rec / 65.96 F1` on the full test set
 - on the same split, `body-only + plain CE` improved that further to `83.16 Acc / 68.24 Prec / 80.02 Rec / 68.47 F1` at `7000`
 ### Not reproducible with current evidence
@@ -171,6 +174,8 @@ Conclusion:
 - the papers are probably directionally correct
 - but the skeleton-map and DRF pipelines are under-specified
 - the missing implementation details are important enough that a faithful independent reproduction is not currently achievable from the paper text and released materials alone
 - the `1:1:8` class ratio is not just a nuisance; it appears to be a major driver of the current skeleton/DRF failure mode
 - on the easier `1:1:2` split, weighted CE is not currently the winning recipe; the best local full-test result so far came from plain CE
 ## Recommended standard for future work in this repo
@@ -30,12 +30,23 @@ Use it for:
 | 2026-03-09 | `ScoNet_skeleton_118_sigma15_joint8_sharedalign_nocut_adamw_proxy_1gpu` | ScoNet-MT-ske proxy | `Scoliosis1K-drf-pkl-118-sigma15-joint8-sharedalign` | Fast proxy route: `no-cut`, `AdamW`, `8x8`, `total_iter=2000`, `eval_iter=500`, `test_seq_subset_size=128` | interrupted | superseded by geometry-fixed proxy before completion |
 | 2026-03-10 | `ScoNet_skeleton_118_sigma15_joint8_geomfix_proxy_1gpu` | ScoNet-MT-ske proxy | `Scoliosis1K-drf-pkl-118-sigma15-joint8-geomfix` | Geometry ablation: aspect-ratio-preserving crop+pad instead of square-warp resize; `AdamW`, `no-cut`, `8x8`, `total_iter=2000`, `eval_iter=500`, fixed test subset seed `118` | complete | proxy subset unstable: `500 24.22/8.07/33.33/13.00`, `1000 60.16/68.05/58.13/55.25`, `1500 26.56/58.33/35.64/17.68`, `2000 27.34/63.96/37.02/20.14` (Acc/Prec/Rec/F1) |
 | 2026-03-10 | `ScoNet_skeleton_118_sigma15_joint8_sharedalign_weightedce_proxy_1gpu` | ScoNet-MT-ske proxy | `Scoliosis1K-drf-pkl-118-sigma15-joint8-sharedalign` | Training-side imbalance ablation: kept the current best shared-align geometry, restored `SGD` baseline settings, and applied weighted CE with class weights `[1.0, 4.0, 4.0]`; `total_iter=2000`, `eval_iter=500`, fixed test subset seed `118` | complete | `500 24.22/8.07/33.33/13.00`, `1000 71.09/48.12/53.93/50.19`, `1500 46.09/52.26/52.34/43.72`, `2000 37.50/47.03/45.45/34.28` (Acc/Prec/Rec/F1) |
 | 2026-03-10 | `ScoNet_skeleton_118_sigma15_joint8_bodyonly_proxy_1gpu` | ScoNet-MT-ske proxy | `Scoliosis1K-drf-pkl-118-sigma15-joint8-bodyonly` | Representation ablation: dropped face keypoints and head limbs from the skeleton-map export while keeping the current shared-align `sigma_limb=1.5 / sigma_joint=8.0` setup; `SGD`, `total_iter=2000`, `eval_iter=500`, fixed test subset seed `118` | complete | `500 53.91/40.60/50.68/40.26`, `1000 65.62/42.86/56.30/47.36`, `1500 28.12/50.28/36.55/19.53`, `2000 26.56/74.93/35.64/17.69` (Acc/Prec/Rec/F1) |
 | 2026-03-10 | `ScoNet_skeleton_118_sigma15_joint8_bodyonly_weightedce_proxy_1gpu` | ScoNet-MT-ske proxy | `Scoliosis1K-drf-pkl-118-sigma15-joint8-bodyonly` | Combined the two strongest partial clues: body-only skeleton map plus weighted CE with class weights `[1.0, 4.0, 4.0]`; `SGD`, `total_iter=2000`, `eval_iter=500`, fixed test subset seed `118` | interrupted | superseded at `100` iterations by the 2-GPU proxy variant |
 | 2026-03-10 | `ScoNet_skeleton_118_sigma15_joint8_bodyonly_weightedce_proxy_2gpu` | ScoNet-MT-ske proxy | `Scoliosis1K-drf-pkl-118-sigma15-joint8-bodyonly` | Same body-only + weighted-CE proxy, relaunched on the `5070 Ti + 3090` pair for faster iteration while keeping the fixed subset seed `118` | complete | `500 68.75/44.46/58.12/49.59`, `1000 57.81/52.49/34.41/26.42`, `1500 41.41/53.84/51.46/40.77`, `2000 39.84/53.76/52.10/39.24` (Acc/Prec/Rec/F1) |
 | 2026-03-10 | `ScoNet_skeleton_112_sigma15_joint8_bodyonly_weightedce_bridge_2gpu_10k` | ScoNet-MT-ske bridge | `Scoliosis1K-drf-pkl-118-sigma15-joint8-bodyonly` + `Scoliosis1K_112.json` | Diagnostic bridge on the easier `1:1:2` split using the current best skeleton recipe (`body-only + weighted CE`), extended to `10000` iterations with proportional LR milestones and eval/save every `1000` | complete | best proxy subset at `7000`: `82.03/66.53/88.84/64.91`; full test at `7000`: `81.82/66.21/88.50/65.96` (Acc/Prec/Rec/F1) |
 | 2026-03-10 | `ScoNet_skeleton_112_sigma15_joint8_bodyonly_plaince_bridge_2gpu_10k` | ScoNet-MT-ske bridge | `Scoliosis1K-drf-pkl-118-sigma15-joint8-bodyonly` + `Scoliosis1K_112.json` | Same `1:1:2` body-only bridge as above, but removed weighted CE to test whether class weighting was suppressing precision on the easier split | interrupted | superseded before meaningful progress by the user-requested 1-GPU rerun on the `5070 Ti` |
 | 2026-03-10 | `ScoNet_skeleton_112_sigma15_joint8_bodyonly_plaince_bridge_1gpu_10k` | ScoNet-MT-ske bridge | `Scoliosis1K-drf-pkl-118-sigma15-joint8-bodyonly` + `Scoliosis1K_112.json` | Same plain-CE `1:1:2` bridge, relaunched on the `5070 Ti` only per user request | complete | best proxy subset at `7000`: `88.28/69.12/74.15/68.80`; full test at `7000`: `83.16/68.24/80.02/68.47`; final proxy at `10000`: `75.00/65.00/63.41/54.55` (Acc/Prec/Rec/F1) |
 | 2026-03-10 | `ScoNet_skeleton_112_sigma15_joint8_headlite_plaince_bridge_1gpu_10k` | ScoNet-MT-ske bridge | `Scoliosis1K-drf-pkl-112-sigma15-joint8-headlite` + `Scoliosis1K_112.json` | Added `head-lite` structure (nose plus shoulder links, no eyes/ears) on top of the plain-CE `1:1:2` bridge; first `3090` launch OOMed due unrelated occupancy, then relaunched on the UUID-pinned `5070 Ti` | running | pending |
 ## Current best skeleton baseline
 Current best `ScoNet-MT-ske`-style result:
- `ScoNet_skeleton_118_sigma15_joint8_sharedalign_2gpu_bs12x8`
+- practical best on the easier `1:1:2` split:
- `50.47 Acc / 69.31 Prec / 54.58 Rec / 48.63 F1`
+  - `ScoNet_skeleton_112_sigma15_joint8_bodyonly_plaince_bridge_1gpu_10k` at `7000`
  - `83.16 Acc / 68.24 Prec / 80.02 Rec / 68.47 F1`
 - best result retained on the harder `1:1:8` split:
  - `ScoNet_skeleton_118_sigma15_joint8_sharedalign_2gpu_bs12x8`
  - `50.47 Acc / 69.31 Prec / 54.58 Rec / 48.63 F1`
 ## Notes
@@ -44,3 +55,8 @@ Current best `ScoNet-MT-ske`-style result:
 - The long `ScoNet_skeleton_118_sigma15_joint8_sharedalign_nocut_adamw_1gpu_bs8x8` run was intentionally interrupted and superseded by the shorter proxy run once fast-iteration support was added.
 - The geometry-fixed proxy run fit the train split quickly but did not produce a stable proxy validation curve, so it should not be promoted to a full 20k run.
 - The weighted-CE proxy briefly improved the proxy peak at `1000` iterations, but it also collapsed afterward, so class weighting alone is not a sufficient fix for the skeleton branch.
 - The body-only proxy improved the early `500`-iteration proxy result relative to some earlier runs, but it still collapsed badly after `1000`, so removing face/head structure alone is also not a sufficient fix.
 - Combining `body-only` with `weighted CE` gave the best `500`-iteration proxy seen so far (`68.75 Acc / 49.59 F1` on the fixed 128-sequence subset), but it still degraded substantially by `1000+`, which points more toward schedule/imbalance dynamics than a single missing representation tweak.
 - Full-test check on retained checkpoints from the combined `body-only + weighted CE` run did not confirm a simple early-stop win: the retained `1000` checkpoint scored `56.61 Acc / 52.11 Prec / 34.15 Rec / 25.61 F1` on the full test split, while `2000` scored `41.52 Acc / 56.75 Prec / 54.75 Rec / 40.09 F1`. That means the small proxy subset is useful for screening but not reliable enough to choose the final stopping point by itself.
 - The `1:1:2` bridge run is the strongest evidence so far that the skeleton branch is learnable: with the current best skeleton recipe (`body-only + weighted CE`), the full test score at `7000` reached `81.82 Acc / 66.21 Prec / 88.50 Rec / 65.96 F1`. That still trails the ScoNet paper’s stronger `1:1:2` result, but it is dramatically better than the `1:1:8` skeleton runs and makes class distribution a first-order factor in the reproduction gap.
 - Removing weighted CE on the `1:1:2` bridge improved the current best full-test result further: `body-only + plain CE` reached `83.16 Acc / 68.24 Prec / 80.02 Rec / 68.47 F1` at `7000`, so weighted CE does not currently look beneficial on the easier split.
@@ -66,12 +66,16 @@ The main findings so far are:
 - The `1:1:8` skeleton-map control trained with healthy optimization metrics but evaluated very poorly.
 - A recent `ScoNet-MT-ske`-style control on `Scoliosis1K_sigma_8.0/pkl` finished with `36.45%` accuracy and `32.78%` macro-F1.
 - That result is far below the paper's `1:1:8` ScoNet-MT range and far below the silhouette baseline behavior.
 - On the easier `1:1:2` split, the skeleton branch is clearly learnable:
  - `body-only + weighted CE` reached `81.82%` accuracy and `65.96%` macro-F1 on the full test set at `7000`
  - `body-only + plain CE` improved that to `83.16%` accuracy and `68.47%` macro-F1 at `7000`
 The current working conclusion is:
 - the core ScoNet trainer is not the problem
 - the strong silhouette checkpoint is not evidence that the skeleton-map path works
 - the main remaining suspect is the skeleton-map representation and preprocessing path
 - for practical model development, `1:1:2` is currently the better working split than `1:1:8`
 For readability in this repo's docs, `ScoNet-MT-ske` refers to the skeleton-map variant that the DRF paper writes as `ScoNet-MT^{ske}`.