Constructing infinite deep neural networks with flexible expressiveness while training

The depth of the deep neural network (DNN) refers to number hidden layers between input and output an artificial network. It usually indicates a certain degree complexity computational cost (parameters floating point operations per second) expressiveness once structure is settled. In this study, we experimentally investigate effectiveness using ordinary differential equations (NODEs) as component provide further in continuous way relatively shallower networks rather than stacking more (discrete depth), which achieved improvement with fewer parameters. Experiments are conducted on classic DNNs, residual networks. Moreover, construct infinite flexible based NODEs, enabling system adjust its during training. On better hidden-space provided by adaptive step ResNet NODE (ResODE) managed achieve performances terms convergence accuracy standard networks, improvements widely observed popular benchmarks.