[D] Graph neural networks

Submitted by No_Captain_856 t3_ykajrg in MachineLearning

I have never used GNNs before. I was wondering if I need to give in input to the neural network also info about connections among data (an adjacency matrix) in addition to the data themselves. Thanks!

62

Comments

You must log in or register to comment.

gtancev t1_ius5iem wrote

The original paper explains the main ideas.

26

No_Captain_856 OP t1_ius6rkt wrote

Thank you! 😁😁

2

girvanabhasarasasvad t1_iussp4q wrote

You can also have a look at these later surveys that give an idea of the different types of GNNs. Also if you prefer Tensorflow you can use the Graph Nets library.

6

new_name_who_dis_ t1_iushhel wrote

Yes. Idk what libraries are popular now but I've used Pytorch Geometric, and it takes as input V which is NxF where N is number of nodes in the graph and F features of the nodes. And E which is 2xnum_edges, where each 2d element is a directed edge between the index of node in V to another index of node in V. E is basically the sparse representation of the adjacency matrix.

18

JoPeGame t1_iut00to wrote

I like Deep Graph Lib for GNN. Documentation is good and their Github is full of examples

11

seraschka t1_iuxf2c3 wrote

> I need to give in input to the neural network also info about connections among data (an adjacency matrix) in addition to the data themselves.

Yup :). In a nutshell, you can think of the forward pass as

  def forward(self, X, A):
    potential_msgs = torch.mm(X, self.W2)
    propagated_msgs = torch.mm(A, potential_msgs)
    root_update = torch.mm(X, self.W1)
    output = propagated_msgs + root_update + self.bias
    return output

where A is the adjacency matrix.

PS: I have a code notebook on coding a simple graph neural net from scratch if useful: https://github.com/rasbt/machine-learning-book/blob/main/ch18/ch18_part1.ipynb

5

nbviewerbot t1_iuxf3o0 wrote

I see you've posted a GitHub link to a Jupyter Notebook! GitHub doesn't render large Jupyter Notebooks, so just in case, here is an nbviewer link to the notebook:

https://nbviewer.jupyter.org/url/github.com/rasbt/machine-learning-book/blob/main/ch18/ch18%5C_part1.ipynb

Want to run the code yourself? Here is a binder link to start your own Jupyter server and try it out!

https://mybinder.org/v2/gh/rasbt/machine-learning-book/main?filepath=ch18%2Fch18%5C_part1.ipynb

^(I am a bot.) ^(Feedback) ^(|) ^(GitHub) ^(|) ^(Author)

3

miguelventura t1_iusrlt1 wrote

Yes, though connections may carry data themselves, in which case an adjacency matrix isn't enough. Some problem domains can be modeled in graphs where nodes are simply connected, but others may require a connection type to be defined (eg: "friend of" / "worked with" / "parent of") and even more data besides just a type (eg: if representing physical objects as a graph, you could encode distances as edge attributes).

So there's multiple ways to represent the graph structure, adj matrix being just one of them but that will have to be part of the input to the network.

4

DaltonSC2 t1_iusyvz6 wrote

>though connections may carry data themselves, in which case an adjacency matrix isn't enough

does using pointwise correlation as the adj matrix not give enough information to learn the different categories?

0

master3243 t1_iutotkj wrote

It's not just about learning different categories.

Imagine you're trying to study a social network of people, take twitter users for example, the individual nodes will probably be the users and the data associated with them (past tweets, bio, etc) while the edges would be the connection between users that you care about (e.g. A follows B, or A tweeted at B, or A retweeted post by B, etc.) and you can see how each of those connections carries information other than just a binary yes or no (e.g. When did A follow B? How many previous tweets did A see of B? How many followers did B have at the time? How many tweets did B have at that time?)

You can see how an individual edge can carry an extremely rich feature vector between nodes A and B where those features are separate from the features belonging to either node A and B themselves. Thus, it's possible that a binary adjacency matrix would not be enough to capture the intrinsic properties of that system.

2

After-Advertising-61 t1_iuu5h9b wrote

Have you ever encountered data or a scenario where the underlying process might be well represented by two types of edges? e.g. "Twitter replies to agree/support" vs "Twitter replies to disagree/condemn." Two phase networks have like this are great for representing all kinds of resonance modes (like plasmon), but I haven't quite found a good data science or applied stat application. My intuition is that two party political discourse might be well represented in this style.

1

telimektar t1_iuv5hi9 wrote

What you are describing is often referred to as heterogeneous graphs or knowledge graph, plenty of examples out there. Model wise there is a full literature dedicated to the topic.

1

Gnabenmeister t1_iuvcf70 wrote

I loved petar velickovic's introduction to them. They are on YouTube.

3

slashdave t1_iutzrjf wrote

Without edges, you don't have a graph

1