Building a Pong playing AI

This repository contains the resources needed for the tutorial, Building a Pong playing AI in just 1 hour(Plus 4 days training time). The full video for the tutorial is on youtube here.

Installation Guide for OS X

Tested on a Macbook Pro (late 2013) with El Capitan, unsure if GPU-support works.

Requirements:

Install some image libraries and a X framework for MacOS:

brew install sdl_image
brew install Caskroom/cask/xquartz

Clone the repo:

git clone git@github.com:DanielSlater/PyDataLondon2016.git
cd PyDataLondon2016/

Create a virtual environment for Python 2:

conda create --name pong-ai-27 python=2
source activate pong-ai-27

Install listed dependencies plus opencv:

conda install matplotlib numpy opencv

Install tensorflow and pygame:

conda install -c https://conda.anaconda.org/jjhelmus tensorflow
conda install -c https://conda.binstar.org/quasiben pygame

Initialize submodules:

git submodule init
git submodule update

Symlink resources and common in folder examples:

cd examples/
ln -s ../resources/ 
ln -s ../common/

Run an example:

python 1_random_half_pong_player.py

Linux Nvidea GPU installation Guide

Tensorflow requires an NVidia GPU and only runs on Linux/Mac so if you don't have these Theano is an option (see below). The examples are all in Tensorflow, but that translates very easily to Theano and we have an example Q-learning Theano implementation that can be extended to work with Pong.

Windows/non nvidia gpu

Python

Either 2 or 3 is fine.

PyGame

Download which ever version matches the version of Python you plan on using.

Matplotlib

Match version

Theano Installation Guide for Windows

Download anaconda and install packages:

conda install mingw libpython numpy

Clone Theano repo:

git clone https://github.com/Theano/Theano.git

Install theano package:

cd Theano
python setup.py develop

Docker environment alternative

Docker build

Have a look at the Makefile, essentially this helps you setup an xquartz environment exposed to a docker container along with the required dependencies.
'make all' should in theory launch you into an environment capable of running th examples straight away.

Resources

PyGame Player

Used for running reinforcement learning agents against PyGame

PyGame Pong

PyGame implementation of pong

PyGame Half Pong

Even pong can be hard if you're just a machine. Half pong is a simplified version of pong, if you can believe it. The score and other bits of noise are removed from the game. There is only 1 bar and it is only 80x80 pixels which speeds up training and removes the need to downsize the screen

Examples