Lanczos Network

This is the PyTorch implementation of Lanczos Network as described in the following ICLR 2019 paper:

@inproceedings{liao2019lanczos,
  title={LanczosNet: Multi-Scale Deep Graph Convolutional Networks},
  author={Liao, Renjie and Zhao, Zhizhen and Urtasun, Raquel and Zemel, Richard},
  booktitle={ICLR},
  year={2019}
}

Visualization

Benchmark

We also provide our own implementation of 9 recent graph neural networks on the QM8 benchmark:

graph convolution networks for fingerprint (GCN-FP)
gated graph neural networks (GGNN)
diffusion convolutional neural networks (DCNN)
Chebyshev networks (ChebyNet)
graph convolutional networks (GCN)
message passing neural networks (MPNN)
graph sample and aggregate (GraphSAGE)
graph partition neural networks (GPNN)
graph attention networks (GAT)

You should be able to reproduce the following results of weighted mean absolute error (MAE x 1.0e-3):

Methods	Validation MAE	Test MAE
GCN-FP	15.06 +- 0.04	14.80 +- 0.09
GGNN	12.94 +- 0.05	12.67 +- 0.22
DCNN	10.14 +- 0.05	9.97 +- 0.09
ChebyNet	10.24 +- 0.06	10.07 +- 0.09
GCN	11.68 +- 0.09	11.41 +- 0.10
MPNN	11.16 +- 0.13	11.08 +- 0.11
GraphSAGE	13.19 +- 0.04	12.95 +- 0.11
GPNN	12.81 +- 0.80	12.39 +- 0.77
GAT	11.39 +- 0.09	11.02 +- 0.06
LanczosNet	9.65 +- 0.19	9.58 +- 0.14
AdaLanczosNet	10.10 +- 0.22	9.97 +- 0.20

Note:

Above results are averaged over 3 runs with random seeds {1234, 5678, 9012}

Setup

To set up experiments, we need to download the preprocessed QM8 data and build our customized operators by running the following scripts:

./setup.sh

Note:

We also provide the script dataset/get_qm8_data.py to preprocess the raw QM8 data which requires the installation of DeepChem. It will produce a different train/dev/test split than what we used in the paper due to the randomness of DeepChem. Therefore, we suggest using our preprocessed data for a fair comparison.

Dependencies

Python 3, PyTorch(1.0)

Other dependencies can be installed via

pip install -r requirements.txt

Run Demos

Train

To run the training of experiment X where X is one of {qm8_lanczos_net, qm8_ada_lanczos_net, ...}:

python run_exp.py -c config/X.yaml

Note:

Please check the folder config for a full list of configuration yaml files.
Most hyperparameters in the configuration yaml file are self-explanatory.

Test

After training, you can specify the test_model field of the configuration yaml file with the path of your best model snapshot, e.g.,

test_model: exp/qm8_lanczos_net/LanczosNet_chemistry_2018-Oct-02-11-55-54_25460/model_snapshot_best.pth
To run the test of experiments X:

python run_exp.py -c config/X.yaml -t

Run on General Graph Datasets

I provide an example code for a synthetic graph regression problem, i.e., given multiple graphs, each of which is accompanied with node embeddings, it is required to predict a real-valued graph embedding vector per graph.

To generate the synthetic dataset:

cd dataset

PYTHONPATH=../ python get_graph_data.py
To run the training:

python run_exp.py -c config/graph_lanczos_net.yaml

Note:

Please read dataset/get_graph_data.py for more information on how to adapt it to your own graph datasets.
I only add support to LanczosNet by slightly modifying the learnable node embedding to the input node embedding in model/lanczos_net_general.py. It should be straightforward for you to add support to other models if you are interested.

Cite

Please cite our paper if you use this code in your research work.

Questions/Bugs

Please submit a Github issue or contact rjliao@cs.toronto.edu if you have any questions or find any bugs.