# DRF Author Checkpoint Compatibility Note

This note records what happened when evaluating the author-provided DRF bundle in this repo:

- checkpoint: `artifact/scoliosis_drf_author_118_compat/DRF_118_unordered_iter2w_lr0.001_8830-08000.pt`
- config: `ckpt/drf_author/drf_scoliosis1k_20000.yaml`

The short version:
- the weight file is real and structurally usable
- the provided YAML is not a reliable source of truth
- the main problem was integration-contract mismatch, not a broken checkpoint

## What Was Wrong

The author bundle was internally inconsistent in several ways.

### 1. Split mismatch

The DRF paper says the main experiment uses `1:1:8`, i.e. the `118` split.

But the provided YAML pointed to:
- `./datasets/Scoliosis1K/Scoliosis1K_112.json`

while the checkpoint filename itself says:
- `DRF_118_...`

So the bundle already disagreed with itself.

### 2. Class-order mismatch

The biggest hidden bug was class ordering.

The current repo evaluator assumes:
- `negative = 0`
- `neutral = 1`
- `positive = 2`

But the author stub in `research/drf.py` uses:
- `negative = 0`
- `positive = 1`
- `neutral = 2`

That means an otherwise good checkpoint can look very bad if logits are interpreted in the wrong class order.

### 3. Legacy module-name mismatch

The author checkpoint stores PGA weights under:
- `attention_layer.*`

The current repo uses:
- `PGA.*`

This is a small compatibility issue, but it must be remapped before loading.

### 4. Preprocessing/runtime-contract mismatch

The author checkpoint does not line up with the stale YAML’s full runtime contract.

Most importantly, it did **not** work well with the more paper-literal local export:
- `Scoliosis1K-drf-pkl-118-paper`

It worked much better with the more OpenGait-like aligned export:
- `Scoliosis1K-drf-pkl-118-aligned`

That strongly suggests the checkpoint was trained against a preprocessing/runtime path closer to the aligned OpenGait integration than to the later local “paper-literal” summed-heatmap ablation.

## What Was Added In-Tree

The current repo now has a small compatibility layer in:
- `opengait/modeling/models/drf.py`

It does two things:
- remaps legacy keys `attention_layer.* -> PGA.*`
- supports configurable `model_cfg.label_order`

The model also canonicalizes inference logits back into the repo’s evaluator order, so author checkpoints can be evaluated without modifying the evaluator itself.

## Tested Compatibility Results

### Best usable author-checkpoint path

Config:
- `configs/drf/drf_author_eval_118_aligned_1gpu.yaml`

Dataset/runtime:
- dataset root: `Scoliosis1K-drf-pkl-118-aligned`
- partition: `Scoliosis1K_118.json`
- transform: `BaseSilCuttingTransform`
- label order:
  - `negative`
  - `positive`
  - `neutral`

Result:
- `80.24 Acc / 76.73 Prec / 76.40 Rec / 76.56 F1`

This is the strongest recovered path so far.

### Verified provenance of `Scoliosis1K-drf-pkl-118-aligned`

The `118-aligned` root is no longer just an informed guess. It was verified
directly against the raw pose source:
- `/mnt/public/data/Scoliosis1K/Scoliosis1K-pose-pkl`

The matching preprocessing path is:
- `datasets/pretreatment_scoliosis_drf.py`
- default heatmap config:
  - `configs/drf/pretreatment_heatmap_drf.yaml`
- archived equivalent config:
  - `configs/drf/pretreatment_heatmap_drf_118_aligned.yaml`

That means the aligned root was produced with:
- shared `sigma: 8.0`
- `align: True`
- `final_img_size: 64`
- default `heatmap_reduction=upstream`
- no `--stats_partition`, i.e. dataset-level PAV min-max stats

Equivalent command:

```bash
uv run python datasets/pretreatment_scoliosis_drf.py \
  --pose_data_path /mnt/public/data/Scoliosis1K/Scoliosis1K-pose-pkl \
  --output_path /mnt/public/data/Scoliosis1K/Scoliosis1K-drf-pkl-118-aligned
```

Verification evidence:
- a regenerated `0_heatmap.pkl` sample from the raw pose input matched the stored
  `Scoliosis1K-drf-pkl-118-aligned` sample exactly (`array_equal == True`)
- a full recomputation of `pav_stats.pkl` from the raw pose input matched the
  stored `pav_min`, `pav_max`, and `stats_partition=None` exactly

So `118-aligned` is the old default OpenGait-style DRF export, not the later:
- `118-paper` paper-literal summed-heatmap export
- `118` train-only-stats splitroot export
- `sigma15` / `sigma15_joint8` exports

### Targeted preprocessing ablations around the recovered path

After verifying the aligned root provenance, a few focused runtime/data ablations
were tested against the author checkpoint to see which part of the contract still
mattered most.

Baseline:
- `118-aligned`
- `BaseSilCuttingTransform`
- result:
  - `80.24 Acc / 76.73 Prec / 76.40 Rec / 76.56 F1`

Hybrid 1:
- aligned heatmap + splitroot PAV
- result:
  - `77.30 Acc / 73.70 Prec / 73.04 Rec / 73.28 F1`

Hybrid 2:
- splitroot heatmap + aligned PAV
- result:
  - `80.37 Acc / 77.16 Prec / 76.48 Rec / 76.80 F1`

Runtime ablation:
- `118-aligned` + `BaseSilTransform` (`no-cut`)
- result:
  - `49.93 Acc / 50.49 Prec / 51.58 Rec / 47.75 F1`

What these ablations suggest:
- `BaseSilCuttingTransform` is necessary; `no-cut` breaks the checkpoint badly
- dataset-level PAV stats (`stats_partition=None`) matter more than the exact
  aligned-vs-splitroot heatmap writer
- the heatmap export is still part of the contract, but it is no longer the
  dominant remaining mismatch

### Other tested paths

`configs/drf/drf_author_eval_118_splitroot_1gpu.yaml`
- dataset root: `Scoliosis1K-drf-pkl-118`
- result:
  - `77.17 Acc / 73.61 Prec / 72.59 Rec / 72.98 F1`

`configs/drf/drf_author_eval_112_1gpu.yaml`
- dataset root: `Scoliosis1K-drf-pkl`
- partition: `Scoliosis1K_112.json`
- result:
  - `85.19 Acc / 57.98 Prec / 56.65 Rec / 57.30 F1`

`configs/drf/drf_author_eval_118_paper_1gpu.yaml`
- dataset root: `Scoliosis1K-drf-pkl-118-paper`
- transform: `BaseSilTransform`
- result:
  - `27.24 Acc / 9.08 Prec / 33.33 Rec / 14.27 F1`

## Interpretation

What these results mean:

- the checkpoint is not garbage
- the original “very bad” local eval was mostly a compatibility failure
- the largest single hidden bug was the class-order mismatch
- the author checkpoint is also sensitive to which local DRF dataset root is used
- the recovered runtime is now good enough to make the checkpoint believable, but
  preprocessing alone did not recover the paper DRF headline row

What they do **not** mean:

- we have perfectly reconstructed the author’s original training path
- the provided YAML is trustworthy as-is
- the paper’s full DRF claim is fully reproduced here

One practical caveat on `1:1:2` vs `1:1:8` comparisons in this repo:
- local `Scoliosis1K_112.json` and `Scoliosis1K_118.json` are not the same train/test
  split with only a different class ratio
- they differ substantially in membership
- so local `112` vs `118` results should not be overinterpreted as a pure
  class-balance ablation unless the train/test pool is explicitly held fixed

The strongest recovered result:
- `80.24 / 76.73 / 76.40 / 76.56`

This is close to the paper’s reported `ScoNet-MT^ske` F1 and much better than our earlier broken compat evals, but it is still below the paper’s DRF headline result:
- paper DRF: `86.0 Acc / 84.1 Prec / 79.2 Rec / 80.8 F1`

## Practical Recommendation

If someone wants to use the author checkpoint in this repo today, the recommended path is:

1. use `configs/drf/drf_author_eval_118_aligned_1gpu.yaml`
2. keep the author label order:
   - `negative, positive, neutral`
3. keep the legacy `attention_layer -> PGA` remap in the model
4. do **not** assume the stale `112` YAML is the correct training/eval contract

If someone wants to push this further, the highest-value next step is:
- finetune from the author checkpoint on the aligned `118` path instead of starting DRF from scratch

## How To Run

Recommended eval:

```bash
CUDA_VISIBLE_DEVICES=GPU-9cc7b26e-90d4-0c49-4d4c-060e528ffba6 \
uv run torchrun --nproc_per_node=1 --master_port=29693 \
  opengait/main.py \
  --cfgs ./configs/drf/drf_author_eval_118_aligned_1gpu.yaml \
  --phase test
```

Other compatibility checks:

```bash
CUDA_VISIBLE_DEVICES=GPU-9cc7b26e-90d4-0c49-4d4c-060e528ffba6 \
uv run torchrun --nproc_per_node=1 --master_port=29695 \
  opengait/main.py \
  --cfgs ./configs/drf/drf_author_eval_112_1gpu.yaml \
  --phase test

CUDA_VISIBLE_DEVICES=GPU-9cc7b26e-90d4-0c49-4d4c-060e528ffba6 \
uv run torchrun --nproc_per_node=1 --master_port=29696 \
  opengait/main.py \
  --cfgs ./configs/drf/drf_author_eval_118_splitroot_1gpu.yaml \
  --phase test

CUDA_VISIBLE_DEVICES=GPU-9cc7b26e-90d4-0c49-4d4c-060e528ffba6 \
uv run torchrun --nproc_per_node=1 --master_port=29697 \
  opengait/main.py \
  --cfgs ./configs/drf/drf_author_eval_118_paper_1gpu.yaml \
  --phase test
```

If someone wants to reproduce this on another machine, the usual paths to change are:
- `data_cfg.dataset_root`
- `data_cfg.dataset_partition`
- `evaluator_cfg.restore_hint`

The archived artifact bundle is:
- `artifact/scoliosis_drf_author_118_compat`