Deep AutoEncoders for Collaborative Filtering

This is not an official NVIDIA product. It is a research project described in: "Training Deep AutoEncoders for Collaborative Filtering"(https://arxiv.org/abs/1708.01715)

The model

The model is based on deep AutoEncoders.

AutEncoderPic

Requirements

Python 3.6
Pytorch: pipenv install
CUDA (recommended version >= 8.0)

Training using mixed precision with Tensor Cores

You would need NVIDIA Volta-based GPU
Checkout mixed precision branch
For theory on mixed precision training see Mixed Precision Training paper

Getting Started

Run unittests first

The code is intended to run on GPU. Last test can take a minute or two.

$ python -m unittest test/data_layer_tests.py
$ python -m unittest test/test_model.py

Tutorial

Checkout this tutorial by miguelgfierro.

Get the data

Note: Run all these commands within your DeepRecommender folder

Netflix prize

Download from here into your DeepRecommender folder

$ tar -xvf nf_prize_dataset.tar.gz
$ tar -xf download/training_set.tar
$ python ./data_utils/netflix_data_convert.py training_set Netflix

Data stats

Dataset	Netflix 3 months	Netflix 6 months	Netflix 1 year	Netflix full
Ratings train	13,675,402	29,179,009	41,451,832	98,074,901
Users train	311,315	390,795	345,855	477,412
Items train	17,736	17,757	16,907	17,768
Time range train	2005-09-01 to 2005-11-31	2005-06-01 to 2005-11-31	2004-06-01 to 2005-05-31	1999-12-01 to 2005-11-31
--------	----------------	-----------	------------
Ratings test	2,082,559	2,175,535	3,888,684	2,250,481
Users test	160,906	169,541	197,951	173,482
Items test	17,261	17,290	16,506	17,305
Time range test	2005-12-01 to 2005-12-31	2005-12-01 to 2005-12-31	2005-06-01 to 2005-06-31	2005-12-01 to 2005-12-31

Train the model

In this example, the model will be trained for 12 epochs. In paper we train for 102.

python run.py --gpu_ids 0 \
--path_to_train_data Netflix/NF_TRAIN \
--path_to_eval_data Netflix/NF_VALID \
--hidden_layers 512,512,1024 \
--non_linearity_type selu \
--batch_size 128 \
--logdir model_save \
--drop_prob 0.8 \
--optimizer momentum \
--lr 0.005 \
--weight_decay 0 \
--aug_step 1 \
--noise_prob 0 \
--num_epochs 12 \
--summary_frequency 1000

Note that you can run Tensorboard in parallel

$ tensorboard --logdir=model_save

Run inference on the Test set

python infer.py \
--path_to_train_data Netflix/NF_TRAIN \
--path_to_eval_data Netflix/NF_TEST \
--hidden_layers 512,512,1024 \
--non_linearity_type selu \
--save_path model_save/model.epoch_11 \
--drop_prob 0.8 \
--predictions_path preds.txt

Compute Test RMSE

python compute_RMSE.py --path_to_predictions=preds.txt

After 12 epochs you should get RMSE around 0.927. Train longer to get below 0.92

Results

It should be possible to achieve the following results. Iterative output re-feeding should be applied once during each iteration.

(exact numbers will vary due to randomization)

DataSet	RMSE	Model Architecture
Netflix 3 months	0.9373	n,128,256,256,dp(0.65),256,128,n
Netflix 6 months	0.9207	n,256,256,512,dp(0.8),256,256,n
Netflix 1 year	0.9225	n,256,256,512,dp(0.8),256,256,n
Netflix full	0.9099	n,512,512,1024,dp(0.8),512,512,n