Disentangled-Person-Image-Generation

Tensorflow implementation of CVPR 2018 paper Disentangled Person Image Generation

Two stages framework

Stage-I: reconstruction

Sampling phase

Dependencies

Resources

TF-record data preparation steps

You can skip this data preparation procedure if directly using the tf-record data files.

  1. cd datasets
  2. ./run_convert_market.sh to download and convert the original images, poses, attributes, segmentations
  3. ./run_convert_DF.sh to download and convert the original images, poses
  4. [Optional] ./run_convert_RCV.sh to convert the original images and pose coordinates, i.e. (row, column, visibility) (e.g. from OpenPose or MaskRCNN), which can be useful for other datasets. Note: we also provide the convert code for Market-1501 Attribute and Market-1501 Segmentation results from PSPNet. These extra info. are provided for further research. In our experiments, pose mask are ontained from pose key-points (see _getPoseMask function in convert .py files).

Training steps

  1. Download the tf-record training data.
  2. Modify the log_dir and log_dir_pretrain in the run_market_train.sh/run_DF_train.sh scripts.

  3. run run_market_train.sh/run_DF_train.sh

    Note: we use a triplet instead of pair real/fake for adversarial training to keep training more stable.

Testing steps

  1. Download the pretrained models and tf-record testing data.
  2. Modify the log_dir and log_dir_pretrain in the run_market_test.sh/run_DF_test.sh scripts.
  3. run run_market_test.sh/run_DF_test.sh

Fg/Bg/Pose sampling on Market-1501

Appearance sampling on DeepFashion dataset

Pose sampling on DeepFashion dataset

Pose interpolation between real images

Pose guided person image generation

Citation

@inproceedings{ma2018disentangled,
  title={Disentangled Person Image Generation},
  author={Ma, Liqian and Sun, Qianru and Georgoulis, Stamatios and Van Gool, Luc and Schiele, Bernt and Fritz, Mario},
  booktitle={{IEEE} Conference on Computer Vision and Pattern Recognition},
  year={2018}
}

Related projects