Graph Neural Networks (GNNs) are a new and increasingly popular family of deep neural network architectures to perform learning on graphs. Training them efficiently is challenging due the irregular nature graph data. The problem becomes even more when scaling large graphs that exceed capacity single devices. Standard approaches distributed DNN training, such as data model parallelism, do not di...