Probing Graph Representation

by Mohammad Sadegh Akhondzadeh, Vijay Lingam, Aleksandar Bojchevski

Published at the Artificial Intelligence and Statistics (AISTATS) 2023.

Abstract

Today we have a good theoretical understanding of the representational power of Graph Neural Networks (GNNs). For example, their limitations have been characterized in relation to a hierarchy of Weisfeiler-Lehman (WL) isomorphism tests. However, we do not know what is encoded in the learned representations. This is our main question. We answer it using a probing framework to quantify the amount of meaningful information captured in graph representations. Our findings on molecular datasets show the potential of probing for understanding the inductive biases of graph-based models. We compare different families of models, and show that transformer-based models capture more chemically relevant information compared to models based on message passing. We also study the effect of different design choices such as skip connections and virtual nodes. We advocate for probing as a useful diagnostic tool for evaluating and developing graph-based models.

Links

Paper | Code (Github) | Video | Poster | Short Version

Keywords

Probing, Graph Representation Learning, Probing Molecule Representation, Graph Transformer

Cite

Please cite our paper if you use the method in your own work:

--- Citation
@inproceedings{probing_graph_representation,
title = {Probing Graph Representation},
author = {Akhondzadeh, Mohammad Sadegh and Lingam, Vijay and Bojchevski, Aleksandar},
booktitle={Artificial Intelligence and Statistics, {AISTATS}},
year = {2023}
}
---