When I first approached semantic manipulation problem there was no solution like CycleGAN, or later findings. And even now all of them, including new methods produce artifacts when changing image content.
I am using CelebA dataset to train the model. There are two files you would need to reproduce results: img_align_celeba.zip and list_attr_celeba.txt
You can download them from here http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html and put into {PROJECT_DIR}/data
After that initialize data and train the model by running
sh init_data.sh
python train.py
Consider following advices if you want to build this kind of a model:
The code is inspired by pytorch-CycleGAN-and-pix2pix . This paper GANimation: Anatomically-aware Facial Animation from a Single Image arXiv:1807.09251 describes similar training scheme.