This repository includes:

<br/>UNet with SeResNext101 backbone, trained on a synthetic dataset using this repository
<br/>UNet with SeResNext50 backbone, trained on only 500 images + augmentation using this repository
<br/>(left: input image, middle: ground truth, right: prediction)
<br/>FPN with ResNet18 backbone, trained on only 180 images using this repository
<br/>MaskRCNN with ResNet101 backbone, trained on COCO dataset, weights file ported from [matterport/Mask_RCNN](
See [Custom Backbone]( for more details.

# Installation

**i. How to set up a virtual environment and install on it**<br/>
  sudo apt-get install virtualenv
  virtualenv -p python3 venv
  git clone
  cd image-segmentation
  source activate 
  cat requirements.txt | while read p; do pip install $p; done

You should run the following commands every time you open a new terminal in order to run any of python files

  cd /path/to/image-segmentation
  source activate
  # the second line is equivalent to: 
  # source ../venv/bin/activate && export PYTHONPATH=`pwd`/image-segmentation
  # i.e. activating the virtual environment and add the image-segmentation/image-segmentation folder to the PYTHONPATH

ii. How to install without a virtual environment
Note that working on a virtual environment is highly recommended. But if you insist on not using it, you can still do so:

  git clone
  cd image-segmentation
  cat requirements.txt | while read p; do pip install --user $p; done


1. Python 3.5+
2. segmentation-models==0.2.0
3. keras>=2.2.0
4. keras-applications>=1.0.7 
5. tensorflow(-gpu)=>1.8.0 (tested on 1.10.0)

How to run examples

Please read the instruction written in files in each example folder

  1. Custom Backbone
    This example illustrates how to build MaskRCNN with your custom backbone architecture. In particular, I adopted matterport's implementation of ResNet, which is slightly different from qubvel's. Moreover, you can run the inference using the pretrained MaskRCNN_coco.h5. (I slightly modified the 'mask_rcnn_coco.h5' in matterport/Mask_RCNN/releases to make this example work: the only differences are layer names)

  2. Imagenet Classification
    This example shows the imagenet classification results for various backbone architectures.

  3. Create KITTI Label
    This example is a code that I used to simplify some of the object class labels in KITTI dataset. (For instance, I merged the 5 separate classes 'car', 'truck', 'bus', 'caravan' and 'trailer' into a single class called 'vehicle')

  4. Configurations
    Some example cfg files that describes the segmentation models and training processes

How to train your own FPN / LinkNet / PSPNet / UNet model on KITTI dataset

i. Download the modified KITTI dataset from release page (or make your own dataset into the same format) and place it under datasets folder.

How to train your own MaskRCNN model on COCO dataset

i. Download the COCO dataset. To do this, simply run:

  cd /path/to/image-segmentation/datasets

How to visualize inference

You can visualize your model's inference in a pop-up window:

python -c=/path/to/infer.cfg -w=/path/to/best_model.h5 -l=/path/to/class_names.json \

or save the results as image files [This will create a directory named 'results' under the directory you provided in -i option, and write the viusalized inference images in it]:

python -c=/path/to/infer.cfg -w=/path/to/best_model.h5 -l=/path/to/class_names.json \


  title={Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow},
  author={Waleed Abdulla},
  journal={GitHub repository},