deepy: A highly extensible deep learning framework based on Theano

Build Quality PyPI version Requirements Status Documentation Status MIT

deepy is a deep learning framework for designing models with complex architectures.

Many important components such as LSTM and Batch Normalization are implemented inside.

Although highly flexible, deepy maintains a clean high-level interface.

From deepy 0.2.0, you can easily design very complex computational graphs such as Neural Turing Machines.

Example codes will be added shortly.

Recent updates

deepy now supports training on multiple GPUs, see the following example for training neural machine translation models.

https://github.com/zomux/neuralmt

Dependencies

Tutorials (Work in progress)

http://deepy.readthedocs.org/en/latest/

Clean interface

# A multi-layer model with dropout for MNIST task.
from deepy import *

model = NeuralClassifier(input_dim=28*28)
model.stack(Dense(256, 'relu'),
            Dropout(0.2),
            Dense(256, 'relu'),
            Dropout(0.2),
            Dense(10, 'linear'),
            Softmax())

trainer = MomentumTrainer(model)

annealer = LearningRateAnnealer(trainer)

mnist = MiniBatches(MnistDataset(), batch_size=20)

trainer.run(mnist, controllers=[annealer])

Examples

Enviroment setting

MNIST Handwriting task

Variational auto-encoders

Language model

Penn Treebank benchmark

Char-based language models

Deep Q learning

Auto encoders

Train with CG and L-BFGS

DRAW

See https://github.com/uaca/deepy-draw

# Train the model
python mnist_training.py
# Create animation
python animation.py experiments/draw/mnist1.gz

Highway networks

Effect of different initialization schemes

python experiments/initialization_schemes/gaussian.py
python experiments/initialization_schemes/uniform.py
python experiments/initialization_schemes/xavier_glorot.py
python experiments/initialization_schemes/kaiming_he.py

Sorry for that deepy is not well documented currently, but the framework is designed in the spirit of simplicity and readability. This will be improved if someone requires.

Raphael Shu, 2016