PartitionTuner: An operator scheduler for deep‐learning compilers supporting multiple heterogeneous processing units

Recently, embedded systems, such as mobile platforms, have multiple processing units that can operate in parallel, centralized (CPUs) and neural (NPUs). We use deep-learning compilers to generate machine code optimized for these systems from a deep network (DNN). However, the proposed so far codes sequentially execute DNN operators on single unit or parallel graphic (GPUs). In this study, we propose PartitionTuner, an operator scheduler supports heterogeneous PUs including CPUs NPUs. PartitionTuner operator-scheduling plan uses all available simultaneously minimize overall inference time. Operator scheduling is based analysis of architecture performance profiles individual group measured units. By experiments seven DNNs, generates plans perform 5.03% better than static type-based technique SqueezeNet. addition, outperforms recent profiling-based techniques ResNet50, ResNet18, SqueezeNet by 7.18%, 5.36%, 2.73%, respectively.