Distributed deep learning is an important and indispensable direction in the field of research. Earlier research has proposed many algorithms or techniques on accelerating distributed neural network training. This study discusses a new training scenario, namely data isolated learning. Specifically, each node its own local cannot be shared for some reasons. However, order to ensure generalizatio...