• Search by Module
• Search by Words
• Search Projects

• Java
• Python
• JavaScript
• TypeScript
• C++
• Scala
• Blog

Project: vgg-benchmarks (GitHub Link)

• vgg-benchmarks-master
  • results
    • v100
      • benchmark_mxnet.output
      • benchmark_mxnet.dmon
      • benchmark_keras_tensorflow.output
      • benchmark_tensorflow_slim.output
      • INFO.md
      • benchmark_tensorflow.output
    • k520
      • benchmark_mxnet.output
      • benchmark_mxnet.dmon
      • benchmark_keras_tensorflow.output
      • benchmark_keras_theano.dmon
      • benchmark_tensorflow_slim.output
      • INFO.md
      • benchmark_tensorflow_slim.dmon
      • benchmark_caffe.dmon
      • benchmark_tensorflow.output
      • benchmark_keras_theano.output
      • benchmark_tensorflow.dmon
      • benchmark_caffe.output
      • benchmark_keras_tensorflow.dmon
    • maxwell_titan_x
      • benchmark_mxnet.output
      • benchmark_keras_tensorflow.output
      • benchmark_tensorflow_slim.output
      • INFO.md
      • benchmark_tensorflow.output
      • benchmark_keras_theano.output
      • benchmark_caffe.output
      • benchmark_neon.output
    • k80
      • benchmark_mxnet.output
      • benchmark_keras_tensorflow.output
      • benchmark_tensorflow_slim.output
      • INFO.md
      • benchmark_tensorflow.output
      • benchmark_keras_theano.output
      • benchmark_caffe.output
      • benchmark_neon.output
    • gtx_1080
      • benchmark_keras_tensorflow.output
      • benchmark_keras_theano.dmon
      • INFO.md
      • benchmark_caffe.dmon
      • benchmark_keras_theano.output
      • benchmark_caffe.output
      • benchmark_neon.output
      • benchmark_keras_tensorflow.dmon
  • images
    • list.txt
    • img0000.tiff
  • vgg16_solver.prototxt
  • vgg16_train_val.prototxt
  • benchmark_lasagne.py
  • benchmark_tensorflow.py
  • run.sh
  • benchmark_neon.py
  • vgg16_deploy.prototxt
  • LICENSE.md
  • parse_results.py
  • README.md
  • benchmark_mxnet.py
  • benchmark_caffe.py
  • benchmark_keras.py

Simple Deep Learning Benchmark (VGG16)

This mini-benchmark compares the speed of several deep learning frameworks on the VGG-16 network architecture.

Contributions welcome!

Results

The table contains training times in milliseconds per minibatch on the same VGG16 network in different frameworks. (Less is faster). Minibatch size is 16. The time is for a complete SGD step including parameter updates, not just the forward+backward time.

Framework	V100	GTX 1080	Maxwell Titan X	K80	K520
MXNet	81.22	N/A	324.63	1247.47	OOM
TensorFlow	88.03	N/A	332.15	1057.28	2290.58
TensorFlow (slim)	176.14	N/A	370.89	1126.70	2488.51
Keras (TensorFlow)	101.82	287.85	359.89	1020.97	OOM
Keras (Theano)	N/A	409.95	317.30	1141.79	2445.22
Neon	N/A	164.53	207.41	N/A	N/A
Caffe	N/A	244.44	311.06	787.81	OOM
Torch (1)	N/A	232.55	273.54	N/A	N/A

N/A - test not ran

OOM - test ran but failed due to running out of memory (on the K520 with only 4GB memory)

(1) The Torch benchmark is from https://github.com/jcjohnson/cnn-benchmarks (it is not included in this repo).

Running

bash run.sh

Note: this will back up and then restore your ~/.keras/keras.json

Caffe should be built inside caffe/ in the current directory (or a symlink).

Neon should be built anywhere and (just for the Neon test) the built Neon virtualenv should be activated.