SRZoo

Introduction

SRZoo is a collection of toolkits and models for deep learning-based image super-resolution. It provides various pre-trained state-of-the-art super-resolution models that are ready for use.

Here are the key features of SRZoo:

SRZoo provides official pre-trained models of various super-resolution methods.
With SRZoo, you can easily obtain the super-resolved images from the supported super-resolution methods.
It is possible to employ the super-resolution models in various environments such as GPUs supporting CUDA and web browsers via TensorFlow.js.
It is possible to compare the performance of the super-resolution methods with the same evaluation metrics and the same environment.

You can find our motivation and some detailed description of SRZoo such as performance comparison in the following paper.

J.-H. Choi, J.-H. Kim, J.-S. Lee. SRZoo: an integrated repository for super-resolution using deep learning. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), May 2020 [Paper] [arXiv]

Requirements

Python 3.6 or newer
TensorFlow 1.12 or newer

Pre-trained super-resolution models

We currently provide the following pre-trained super-resolution models, where the model parameters are provided by the original authors. Please cite the paper of the original authors when you use the models.

Name	Config	Upscaling factor	Model	Source
EDSR-baseline	edsr_baseline.json	2	edsr_baseline_x2.pb	EDSR-PyTorch
EDSR-baseline	edsr_baseline.json	3	edsr_baseline_x3.pb	EDSR-PyTorch
EDSR-baseline	edsr_baseline.json	4	edsr_baseline_x4.pb	EDSR-PyTorch
EDSR	edsr.json	2	edsr_x2.pb	EDSR-PyTorch
EDSR	edsr.json	3	edsr_x3.pb	EDSR-PyTorch
EDSR	edsr.json	4	edsr_x4.pb	EDSR-PyTorch
EUSR	eusr.json	2	eusr_x2.pb	EUSR-TensorFlow
EUSR	eusr.json	4	eusr_x4.pb	EUSR-TensorFlow
EUSR	eusr.json	8	eusr_x8.pb	EUSR-TensorFlow
DBPN	dbpn.json	2	dbpn_x2.pb	DBPN-Pytorch
DBPN	dbpn.json	4	dbpn_x4.pb	DBPN-Pytorch
DBPN	dbpn.json	8	dbpn_x8.pb	DBPN-Pytorch
RCAN	rcan.json	2	rcan_x2.pb	RCAN
RCAN	rcan.json	3	rcan_x3.pb	RCAN
RCAN	rcan.json	4	rcan_x4.pb	RCAN
RCAN	rcan.json	8	rcan_x8.pb	RCAN
MSRN	msrn.json	2	msrn_x2.pb	MSRN-PyTorch
MSRN	msrn.json	3	msrn_x3.pb	MSRN-PyTorch
MSRN	msrn.json	4	msrn_x4.pb	MSRN-PyTorch
4PP-EUSR	4pp_eusr.json	4	4pp_eusr_pirm_x4.pb	tf-perceptual-eusr
ESRGAN	esrgan.json	4	esrgan_x4.pb	ESRGAN
RRDB	rrdb.json	4	rrdb_x4.pb	ESRGAN
CARN	carn.json	2	carn_x2.pb	CARN-pytorch
CARN	carn.json	3	carn_x3.pb	CARN-pytorch
CARN	carn.json	4	carn_x4.pb	CARN-pytorch
FRSR	frsr_x2.json	2	frsr_x2.pb	NatSR
FRSR	natsr.json	3	frsr_x3.pb	NatSR
FRSR	natsr.json	4	frsr_x4.pb	NatSR
NatSR	natsr.json	4	natsr_x4.pb	NatSR

Super-resolved image retrieval

SRZoo offers a simple image retrieval via get_sr.py, e.g.,

python get_sr.py --config_path=configs/edsr.json --model_path=edsr_x4.pb --input_path=LR --output_path=SR --scale=4

Arguments:

config_path: Path of the model config file.
model_path: Path of the pre-trained model file.
input_path: Path of the input low-resolution images.
output_path: Path of the output super-resolved images will be saved.
scale: Upscaling factor.
self_ensemble: Specify this to employ geometric self-ensemble.
cuda_device: CUDA device index to be used (will be set to the environment variable 'CUDA_VISIBLE_DEVICES').

※ Some models can be run only on GPUs due to the different ordering of the dimensions.

Performance evaluation

With the obtained super-resolved images, it is possible to evaluate the performance via evaluate_sr.py, e.g.,

python evaluate_sr.py --sr_path=SR --truth_path=HR

Arguments:

sr_path: Path of the super-resolved images.
truth_path: Path of the ground-truth images.
shave_borders: The amount of shaving pixles on borders of the images. It is usually set to the upscaling factor.
color_mode: Color conversion mode. ycbcry: Y channel of the YCbCr color space. rgb: RGB channels of the RGB color space.
evaluators: Comma-separated evaluation methods. The evaluators in the evaluators/ folder will be used.
ouptut_name: Filename of the output CSV file.

You can also write your own evaluation metric by implementing an evaluator class that inherits BaseEvaluator in the evaluators/ folder.

Model conversion

It is possible to convert the other pre-trained super-resolution models. Please refer to the converter/ folder for more information. In addition, please refer to the config/ folder to write your own model config file along with the converted model.

Miscellaneous

Image downscaling utilities

We also provide the downscaling utilities for evaluating the super-resolution models, which are in the utils/downscale/ folder.

Employing other image processing models

Since SRZoo is developed to deal with models considering images as both inputs and outputs, our repository can be used to employ the other image processing algorithms with only a few modifications. As a proof-of-concept, we provide a pre-trained deep learning-based image compression model in SRZoo.

Name	Config	Model	Source
fab-jul/imgcomp-cvpr	fabjul_imgcomp.json	fabjul_imgcomp_a.pb	imgcomp-cvpr
fab-jul/imgcomp-cvpr	fabjul_imgcomp.json	fabjul_imgcomp_b.pb	imgcomp-cvpr
fab-jul/imgcomp-cvpr	fabjul_imgcomp.json	fabjul_imgcomp_c.pb	imgcomp-cvpr

To use these models, simply set the upscaling factor (e.g., --scale option of get_sr.py) to 1. Note that only the GPU mode is currently supported for these models.