crosstyan/OpenGait
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

1.0.0 official release (#18)

Browse Source 
* fix bug in fix_BN

* gaitgl OUMVLP support.

* update ./doc/3.advance_usage.md Cross-Dataset Evalution & Data Agumentation

* update config

* update docs.3

* update docs.3

* add loss doc and gather input decorator

* refine the create model doc

* support rearrange directory of unzipped OUMVLP

* fix some bugs in loss_aggregator.py

* refine docs and little fix

* add oumvlp pretreatment description

* pretreatment dataset fix oumvlp description

* add gaitgl oumvlp result

* assert gaitgl input size

* add pipeline

* update the readme.

* update pipeline and readme

* Corrigendum.

* add logo and remove path

* update new logo

* Update README.md

* modify logo size

Co-authored-by: 12131100 <12131100@mail.sustech.edu.cn>
Co-authored-by: noahshen98 <77523610+noahshen98@users.noreply.github.com>
Co-authored-by: Noah <595311942@qq.com>
This commit is contained in:
Junhao Liang
2021-12-08 20:05:28 +08:00
committed by GitHub
parent 6e71c7ac34
commit bb6cd5149a
39 changed files with 11401 additions and 230 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/0.prepare_dataset.md
+101
View File
@@ -0,0 +1,101 @@
# Prepare dataset
Suppose you have downloaded the original dataset, we need to preprocess the data and save it as pickle file. Remember to set your path to the root of processed dataset in [config/*.yaml](config/).
## Preprocess
**CASIA-B**
Download URL: http://www.cbsr.ia.ac.cn/GaitDatasetB-silh.zip
- Original
```
CASIA-B
001 (subject)
bg-01 (type)
000 (view)
001-bg-01-000-001.png (frame)
001-bg-01-000-002.png (frame)
......
......
......
......
```
- Run `python misc/pretreatment.py --input_path CASIA-B --output_path CASIA-B-pkl`
- Processed
```
CASIA-B-pkl
001 (subject)
bg-01 (type)
000 (view)
000.pkl (contains all frames)
......
......
......
```
**OUMVLP**
Step1: Download URL: http://www.am.sanken.osaka-u.ac.jp/BiometricDB/GaitMVLP.html
Step2: Unzip the dataset, you will get a structure directory like:
- Original
```
OUMVLP-raw
Silhouette_000-00 (view-sequence)
00001 (subject)
0001.png (frame)
0002.png (frame)
......
00002
0001.png (frame)
0002.png (frame)
......
......
Silhouette_000-01
00001
0001.png (frame)
0002.png (frame)
......
00002
0001.png (frame)
0002.png (frame)
......
......
Silhouette_015-00
......
Silhouette_015-01
......
......
```
Step3 : To rearrange directory of OUMVLP dataset, turning to id-type-view structure, Run
```
python misc/rearrange_OUMVLP.py --input_path OUMVLP-raw --output_path OUMVLP-rearrange
```
Step4: Transforming images to pickle file, run
```
python misc/pretreatment.py --input_path OUMVLP-rearrange --output_path OUMVLP-pkl
```
- Processed
```
OUMVLP-pkl
00001 (subject)
00 (sequence)
000 (view)
000.pkl (contains all frames)
015 (view)
015.pkl (contains all frames)
...
01 (sequence)
000 (view)
000.pkl (contains all frames)
015 (view)
015.pkl (contains all frames)
......
00002 (subject)
......
......
```
## Split dataset
You can use the partition file in [misc/partitions](misc/partitions/) directly, or you can create yours. Remember to set your path to the partition file in [config/*.yaml](config/).
docs/1.detailed_config.md
+188
View File
@@ -0,0 +1,188 @@
# Configuration item
### data_cfg
* Data configuration
>
> * Args
> * dataset_name: Only support `CASIA-B` and `OUMVLP` now.
> * dataset_root: The path of storing your dataset.
> * num_workers: The number of workers to collect data.
> * dataset_partition: The path of storing your dataset partition file. It splits the dataset to two parts, including train set and test set.
> * cache: If `True`, load all data to memory during buiding dataset.
> * test_dataset_name: The name of test dataset.
----
### loss_cfg
* Loss function
> * Args
> * type: Loss function type, support `TripletLoss` and `CrossEntropyLoss`.
> * loss_term_weights: loss weight.
> * log_prefix: the prefix of loss log.
----
### optimizer_cfg
* Optimizer
> * Args
> * solver: Optimizer type, example: `SGD`, `Adam`.
> * **others**: Please refer to `torch.optim`.
### scheduler_cfg
* Learning rate scheduler
> * Args
> * scheduler : Learning rate scheduler, example: `MultiStepLR`.
> * **others** : Please refer to `torch.optim.lr_scheduler`.
----
### model_cfg
* Model to be trained
> * Args
> * model : Model type, please refer to [Model Library](../lib/modeling/models) for the supported values.
> * **others** : Please refer to the [Training Configuration File of Corresponding Model](../config).
----
### evaluator_cfg
* Evaluator configuration
> * Args
> * enable_float16: If `True`, enable the auto mixed precision mode.
> * restore_ckpt_strict: If `True`, check whether the checkpoint is the same as the defined model.
> * restore_hint: `int` value indicates the iteration number of restored checkpoint; `str` value indicates the path to restored checkpoint.
> * save_name: The name of the experiment.
> * eval_func: The function name of evaluation. For `CASIA-B`, choose `identification`.
> * sampler:
> - type: The name of sampler. Choose `InferenceSampler`.
> - sample_type: In general, we use `all_ordered` to input all frames by its natural order, which makes sure the tests are consistent.
> - batch_size: `int` values.
> - **others**: Please refer to [data.sampler](../lib/data/sampler.py) and [data.collate_fn](../lib/data/collate_fn.py)
> * transform: Support `BaseSilCuttingTransform`, `BaseSilTransform`. The difference between them is `BaseSilCuttingTransform` cut out the black pixels on both sides horizontally.
> * metric: `euc` or `cos`, generally, `euc` performs better.
----
### trainer_cfg
* Trainer configuration
> * Args
> * fix_BN: If `True`, we fix the weight of all `BatchNorm` layers.
> * log_iter: Log the information per `log_iter` iterations.
> * save_iter: Save the checkpoint per `save_iter` iterations.
> * with_test: If `True`, we test the model every `save_iter` iterations. A bit of performance impact.(*Disable in Default*)
> * optimizer_reset: If `True` and `restore_hint!=0`, reset the optimizer while restoring the model.
> * scheduler_reset: If `True` and `restore_hint!=0`, reset the scheduler while restoring the model.
> * sync_BN: If `True`, applies Batch Normalization synchronously.
> * total_iter: The total training iterations, `int` values.
> * sampler:
> - type: The name of sampler. Choose `TripletSampler`.
> - sample_type: `[all, fixed, unfixed]` indicates the number of frames used to test, while `[unordered, ordered]` means whether input sequence by its natural order. Example: `fixed_unordered` means selecting fixed number of frames randomly.
> - batch_size: *[P,K]* where `P` denotes the subjects in training batch while the `K` represents the sequences every subject owns. **Example**:
> - 8
> - 16
> - **others**: Please refer to [data.sampler](../lib/data/sampler.py) and [data.collate_fn](../lib/data/collate_fn.py).
> * **others**: Please refer to `evaluator_cfg`.
---
**Note**:
- All the config items will be merged into [default.yaml](../config/default.yaml), and the current config is preferable.
- The output directory, which includes the log, checkpoint and summary files, is depended on the defined `dataset_name`, `model` and `save_name` settings, like `output/${dataset_name}/${model}/${save_name}`.
# Example
```yaml
data_cfg:
dataset_name: CASIA-B
dataset_root: your_path
dataset_partition: ./misc/partitions/CASIA-B_include_005.json
num_workers: 1
remove_no_gallery: false # Remove probe if no gallery for it
test_dataset_name: CASIA-B
evaluator_cfg:
enable_float16: true
restore_ckpt_strict: true
restore_hint: 60000
save_name: Baseline
eval_func: identification
sampler:
batch_shuffle: false
batch_size: 16
sample_type: all_ordered # all indicates whole sequence used to test, while ordered means input sequence by its natural order; Other options: fixed_unordered
frames_all_limit: 720 # limit the number of sampled frames to prevent out of memory
metric: euc # cos
transform:
- type: BaseSilCuttingTransform
img_w: 64
loss_cfg:
- loss_term_weights: 1.0
margin: 0.2
type: TripletLoss
log_prefix: triplet
- loss_term_weights: 0.1
scale: 16
type: CrossEntropyLoss
log_prefix: softmax
log_accuracy: true
model_cfg:
model: Baseline
backbone_cfg:
in_channels: 1
layers_cfg: # Layers configuration for automatically model construction
- BC-64
- BC-64
- M
- BC-128
- BC-128
- M
- BC-256
- BC-256
# - M
# - BC-512
# - BC-512
type: Plain
SeparateFCs:
in_channels: 256
out_channels: 256
parts_num: 31
SeparateBNNecks:
class_num: 74
in_channels: 256
parts_num: 31
bin_num:
- 16
- 8
- 4
- 2
- 1
optimizer_cfg:
lr: 0.1
momentum: 0.9
solver: SGD
weight_decay: 0.0005
scheduler_cfg:
gamma: 0.1
milestones: # Learning Rate Reduction at each milestones
- 20000
- 40000
scheduler: MultiStepLR
trainer_cfg:
enable_float16: true # half_percesion float for memory reduction and speedup
fix_layers: false
log_iter: 100
restore_ckpt_strict: true
restore_hint: 0
save_iter: 10000
save_name: Baseline
sync_BN: true
total_iter: 60000
sampler:
batch_shuffle: true
batch_size:
- 8 # TripletSampler, batch_size[0] indicates Number of Identity
- 16 # batch_size[1] indicates Samples sequqnce for each Identity
frames_num_fixed: 30 # fixed frames number for training
frames_num_max: 50 # max frames number for unfixed training
frames_num_min: 25 # min frames number for unfixed traing
sample_type: fixed_unordered # fixed control input frames number, unordered for controlling order of input tensor; Other options: unfixed_ordered or all_ordered
type: TripletSampler
transform:
- type: BaseSilCuttingTransform
img_w: 64
```
docs/2.how_to_create_your_model.md
+86
View File
@@ -0,0 +1,86 @@
# How to Create Your Own Model
## Pipeline
![Pipeline](../assets/pipeline.png)
## A new model
If you want to design a new model, you need to write a class inherited from `BaseModel`, e.g, NewModel in newmodel.py:
```python
from ..base_model import BaseModel
class NewModel(BaseModel):
def __init__(self, cfgs, is_training):
super().__init__(cfgs, is_training)
def build_network(self, model_cfg):
self.encoder = ...
def forward(self, inputs):
ipts, labs, typs, viws, seqL = inputs
sils = ipts[0]
if len(sils.size()) == 4:
sils = sils.unsqueeze(2)
del ipts
n, s, c, h, w = sils.size()
embed_1, logits, embed = self.encoder(sils)
return {
'training_feat': {
'triplet': {'embeddings': embed_1, 'labels': labs},
'softmax': {'logits': logits, 'labels': labs}
},
'visual_summary': {
'image/sils': sils.view(n*s, 1, h, w)
},
'inference_feat': {
'embeddings': embed
}
}
```
In your model class, at least you need to implement `build_network()` and `forward()` functions. The first is used to build the netwroks, and it does not need `return value`. Another is used to calculate the features, the `return value` is fixed in dictionary format
> `training_feat` is for the loss computing, and it must be a `dict` object.
>
> `visual_summary` is for visualization, and it must be a `dict` object.
>
> `inference_feat` is for the inference, and it must be a `dict` object.
>
> `triplet` and `softmax` are the prefixes (or names) of the loss function.
>
> `embeddings`, `logits` and `labels` are the input arguments of the loss function.
More information should be seen in [base_model.py](../lib/modeling/base_model.py) and [loss_aggregator.py](../lib/modeling/loss_aggregator.py).
After finishing the model file, you have two steps left to do:
**Step 1**: Put your newmodel.py under `lib/modeling/models`.
**Step 2**: Specify the model name in a yaml file:
```yaml
model_cfg:
model: NewModel
param1: ...
param2: ...
param3: ...
```
## A new loss
If you want to write a new loss, you need to write a class inherited from `lib/modeling/losses`, like this
```python
from .base import BaseLoss
class NewLoss(BaseLoss):
def __init__(self, *args, **kwargs):
super(NewLoss, self).__init__(*args, **kargs)
@gather_and_scale_wrapper
def forward(self, embeddings, labels):
pass
```
Remember to use `gather_and_scale_wrapper` to wrap your forward function if your loss is computed by pairs like `triplet`. By this, we gather all features to one GPU card and scale the loss by the number of GPUs.
Then, put your loss in `lib/modeling/losses` so that you can use it in config file.
Moreover, refer to [loss_aggregator.py](../lib/modeling/loss_aggregator.py) to explore how does your defined loss work in the model.
docs/3.advanced_usages.md
+88
View File
@@ -0,0 +1,88 @@
# Advanced Usages
### Cross-Dataset Evalution
> You can conduct cross-dataset evalution by just modifying several arguments in your [data_cfg](../config/baseline.yaml#L1).
>
> Take [baseline.yaml](../config/baseline.yaml) as an example:
> ```yaml
> data_cfg:
> dataset_name: CASIA-B
> dataset_root: your_path
> dataset_partition: ./misc/partitions/CASIA-B_include_005.json
> num_workers: 1
> remove_no_gallery: false # Remove probe if no gallery for it
> test_dataset_name: CASIA-B
> ```
> Now, suppose we get the model trained on [CASIA-B](http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp), and then we want to test it on [OUMVLP](http://www.am.sanken.osaka-u.ac.jp/BiometricDB/GaitMVLP.html).
>
> We should alter the `dataset_root`, `dataset_partition` and `test_dataset_name`, just like:
> ```yaml
> data_cfg:
> dataset_name: CASIA-B
> dataset_root: your_OUMVLP_path
> dataset_partition: ./misc/partitions/OUMVLP.json
> num_workers: 1
> remove_no_gallery: false # Remove probe if no gallery for it
> test_dataset_name: OUMVLP
> ```
---
>
<!-- ### Identification Function
> Sometime, your test dataset may be neither the popular [CASIA-B](http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp) nor the largest [OUMVLP](http://www.am.sanken.osaka-u.ac.jp/BiometricDB/GaitMVLP.html). Meanwhile, you need to customize a special identification function to fit your dataset.
>
> * If your path structure is similar to [CASIA-B](http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp) (the 3-flod style: `id-type-view`), we recommand you to -->
### Data Augmentation
> In OpenGait, there is a basic transform class almost called by all the models, this is [BaseSilCuttingTransform](../lib/data/transform.py#L20), which is used to cut the input silhouettes.
>
> Accordingly, by referring to this implementation, you can easily customize the data agumentation in just two steps:
> * *Step1*: Define the transform function or class in [transform.py](../lib/data/transform.py), and make sure it callable. The style of [torchvision.transforms](https://pytorch.org/vision/stable/_modules/torchvision/transforms/transforms.html) is recommanded, and following shows a demo;
>> ```python
>> import torchvision.transforms as T
>> class demo1():
>> def __init__(self, args):
>> pass
>>
>> def __call__(self, seqs):
>> '''
>> seqs: with dimension of [sequence, height, width]
>> '''
>> pass
>> return seqs
>>
>> class demo2():
>> def __init__(self, args):
>> pass
>>
>> def __call__(self, seqs):
>> pass
>> return seqs
>>
>> def TransformDemo(base_args, demo1_args, demo2_args):
>> transform = T.Compose([
>> BaseSilCuttingTransform(**base_args),
>> demo1(args=demo1_args),
>> demo2(args=demo2_args)
>> ])
>> return transform
>> ```
> * *Step2*: Reset the [`transform`](../config/baseline.yaml#L100) arguments in your config file:
>> ```yaml
>> transform:
>> - type: TransformDemo
>> base_args: {'img_w': 64}
>> demo1_args: false
>> demo2_args: false
>> ```
### Visualization
> To learn how does the model work, sometimes, you need to visualize the intermediate result.
>
> For this purpose, we have defined a built-in instantiation of [`torch.utils.tensorboard.SummaryWriter`](https://pytorch.org/docs/stable/tensorboard.html), that is [`self.msg_mgr.writer`](../lib/utils/msg_manager.py#L24), to make sure you can log the middle information everywhere you want.
>
> Demo: if we want to visualize the output feature of [baseline's backbone](../lib/modeling/models/baseline.py#L27), we could just insert the following codes at [baseline.py#L28](../lib/modeling/models/baseline.py#L28):
>> ```python
>> summary_writer = self.msg_mgr.writer
>> if torch.distributed.get_rank() == 0 and self.training and self.iteration % 100==0:
>> summary_writer.add_video('outs', outs.mean(2).unsqueeze(2), self.iteration)
>> ```
> Note that this example requires the [`moviepy`](https://github.com/Zulko/moviepy) package, and hence you should run `pip install moviepy` first.