docs: track paper sources and scoliosis experiment updates
This commit is contained in:
@@ -10,10 +10,10 @@ Ground truth policy for this audit:
|
||||
|
||||
## Papers and local references
|
||||
|
||||
- ScoNet paper: [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2407.05726v3/main.tex)
|
||||
- DRF paper: [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex)
|
||||
- local run history: [scoliosis_training_change_log.md](/home/crosstyan/Code/OpenGait/docs/scoliosis_training_change_log.md)
|
||||
- current status note: [sconet-drf-status-and-training.md](/home/crosstyan/Code/OpenGait/docs/sconet-drf-status-and-training.md)
|
||||
- ScoNet paper: [arXiv-2407.05726v3-main.tex](papers/arXiv-2407.05726v3-main.tex)
|
||||
- DRF paper: [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex)
|
||||
- local run history: [scoliosis_training_change_log.md](scoliosis_training_change_log.md)
|
||||
- current status note: [sconet-drf-status-and-training.md](sconet-drf-status-and-training.md)
|
||||
|
||||
## What is reproducible
|
||||
|
||||
@@ -21,8 +21,8 @@ Ground truth policy for this audit:
|
||||
|
||||
Evidence:
|
||||
|
||||
- The ScoNet paper states the standard `1:1:8` evaluation protocol and the SGD schedule clearly in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2407.05726v3/main.tex#L201) and [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2407.05726v3/main.tex#L205).
|
||||
- The same paper reports that multi-task ScoNet-MT is much stronger than single-task ScoNet, including the class-ratio study in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2407.05726v3/main.tex#L277).
|
||||
- The ScoNet paper states the standard `1:1:8` evaluation protocol and the SGD schedule clearly in [arXiv-2407.05726v3-main.tex](papers/arXiv-2407.05726v3-main.tex) and the same tracked TeX source documents the class-ratio study.
|
||||
- The same paper reports that multi-task ScoNet-MT is much stronger than single-task ScoNet, including the class-ratio study in [arXiv-2407.05726v3-main.tex](papers/arXiv-2407.05726v3-main.tex).
|
||||
- In this repo, the standard silhouette ScoNet path is stable:
|
||||
- the model/trainer/evaluator path is intact
|
||||
- a strong silhouette checkpoint reproduces cleanly on the correct split family
|
||||
@@ -37,12 +37,12 @@ Conclusion:
|
||||
|
||||
Evidence:
|
||||
|
||||
- The DRF paper defines the method as `skeleton map + PAV + PGA` in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L130).
|
||||
- The DRF paper defines the method as `skeleton map + PAV + PGA` in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex).
|
||||
- It defines:
|
||||
- pelvis-centering and height normalization in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L135)
|
||||
- two-channel skeleton maps in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L139)
|
||||
- PAV metrics in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L164)
|
||||
- PGA channel/spatial attention in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L196)
|
||||
- pelvis-centering and height normalization in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex)
|
||||
- two-channel skeleton maps in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex)
|
||||
- PAV metrics in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex)
|
||||
- PGA channel/spatial attention in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex)
|
||||
- This repo now has a functioning DRF model and DRF-specific preprocessing path implementing those ideas.
|
||||
|
||||
Conclusion:
|
||||
@@ -55,7 +55,7 @@ Conclusion:
|
||||
|
||||
Evidence:
|
||||
|
||||
- The DRF paper defines 8 symmetric joint pairs and 3 asymmetry metrics in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L160).
|
||||
- The DRF paper defines 8 symmetric joint pairs and 3 asymmetry metrics in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex).
|
||||
- The local preprocessing implements those metrics and produces stable sequence-level PAVs.
|
||||
- Local dataset analysis showed the PAV still carries useful signal, even with a simple probe.
|
||||
|
||||
@@ -70,7 +70,7 @@ Conclusion:
|
||||
|
||||
Evidence:
|
||||
|
||||
- The DRF paper describes the skeleton map as a dense, silhouette-like two-channel representation in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L139).
|
||||
- The DRF paper describes the skeleton map as a dense, silhouette-like two-channel representation in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex).
|
||||
- It does not specify crucial rasterization details such as:
|
||||
- numeric `sigma`
|
||||
- joint-vs-limb relative weighting
|
||||
@@ -81,7 +81,7 @@ Evidence:
|
||||
- Local runs show these details matter a lot:
|
||||
- `sigma=8` skeleton runs were very poor
|
||||
- smaller sigma and fixed limb/joint alignment improved results materially
|
||||
- the best local skeleton baseline is still only `50.47 Acc / 48.63 Macro-F1`, far below the paper's `82.5 / 76.6` for ScoNet-MT-ske in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L252)
|
||||
- the best local skeleton baseline is still only `50.47 Acc / 48.63 Macro-F1`, far below the paper's `82.5 / 76.6` for ScoNet-MT-ske in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex)
|
||||
|
||||
Conclusion:
|
||||
|
||||
@@ -92,8 +92,8 @@ Conclusion:
|
||||
|
||||
Evidence:
|
||||
|
||||
- The ScoNet paper cites the attention-transfer visualization family in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2407.05726v3/main.tex#L246).
|
||||
- The DRF paper cites Zhou et al. CAM in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L295).
|
||||
- The ScoNet paper cites the attention-transfer visualization family in [arXiv-2407.05726v3-main.tex](papers/arXiv-2407.05726v3-main.tex).
|
||||
- The DRF paper cites Zhou et al. CAM in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex).
|
||||
- Neither paper states:
|
||||
- which layer is visualized
|
||||
- whether visualization is before or after temporal pooling
|
||||
@@ -114,8 +114,8 @@ Evidence:
|
||||
- The DRF paper reports:
|
||||
- ScoNet-MT-ske: `82.5 Acc / 81.4 Prec / 74.3 Rec / 76.6 F1`
|
||||
- DRF: `86.0 Acc / 84.1 Prec / 79.2 Rec / 80.8 F1`
|
||||
in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L252)
|
||||
- The best local skeleton-map baseline so far is recorded in [scoliosis_training_change_log.md](/home/crosstyan/Code/OpenGait/docs/scoliosis_training_change_log.md):
|
||||
in [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex)
|
||||
- The best local skeleton-map baseline so far is recorded in [scoliosis_training_change_log.md](scoliosis_training_change_log.md):
|
||||
- `50.47 Acc / 69.31 Prec / 54.58 Rec / 48.63 F1`
|
||||
- Local DRF runs are also well below the paper:
|
||||
- `58.08 / 78.80 / 60.22 / 56.99`
|
||||
@@ -143,7 +143,7 @@ Conclusion:
|
||||
|
||||
Evidence:
|
||||
|
||||
- The ScoNet and DRF papers describe the standard split as `745 train / 748 test` in [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2407.05726v3/main.tex#L201) and [main.tex](/home/crosstyan/Code/OpenGait/research/arXiv-2509.00872v1/main.tex#L280).
|
||||
- The ScoNet and DRF papers describe the standard split as `745 train / 748 test` in [arXiv-2407.05726v3-main.tex](papers/arXiv-2407.05726v3-main.tex) and [arXiv-2509.00872v1-main.tex](papers/arXiv-2509.00872v1-main.tex).
|
||||
- The released partition file matching the `1:1:8` class ratio in this repo is effectively `744 / 749`.
|
||||
|
||||
Conclusion:
|
||||
@@ -162,6 +162,8 @@ Conclusion:
|
||||
- the skeleton branch is learnable on the easier `1:1:2` split
|
||||
- on `1:1:2`, `body-only + weighted CE` reached `81.82 Acc / 66.21 Prec / 88.50 Rec / 65.96 F1` on the full test set
|
||||
- on the same split, `body-only + plain CE` improved that further to `83.16 Acc / 68.24 Prec / 80.02 Rec / 68.47 F1` at `7000`
|
||||
- a later explicit rerun of the `body-only + plain CE` `7000` full-test eval reproduced that same `83.16 / 68.24 / 80.02 / 68.47` result
|
||||
- adding back limited head context via `head-lite` did not improve the full-test score; its `7000` checkpoint reached only `78.07 Acc / 65.42 Prec / 80.50 Rec / 62.08 F1`
|
||||
|
||||
### Not reproducible with current evidence
|
||||
|
||||
@@ -176,6 +178,7 @@ Conclusion:
|
||||
- the missing implementation details are important enough that a faithful independent reproduction is not currently achievable from the paper text and released materials alone
|
||||
- the `1:1:8` class ratio is not just a nuisance; it appears to be a major driver of the current skeleton/DRF failure mode
|
||||
- on the easier `1:1:2` split, weighted CE is not currently the winning recipe; the best local full-test result so far came from plain CE
|
||||
- `head-lite` may help the small fixed proxy subset, but that gain did not transfer to the full `TEST_SET`, so `body-only + plain CE` remains the best practical skeleton recipe
|
||||
|
||||
## Recommended standard for future work in this repo
|
||||
|
||||
|
||||
Reference in New Issue
Block a user