Fast Energy-Optimal Multi-Kernel DNN-like Application Allocation on Multi-FPGA Platforms

Platforms with multiple Field Programmable Gate Arrays (FPGAs), such as Amazon Web Services (AWS) F1 instances, can efficiently accelerate multi-kernel pipelined applications, e.g., Convolutional Neural Networks for machine vision tasks or transformer networks Natural Language Processing tasks. To reduce energy consumption when the FPGAs are underutilized, we propose a model to (1) find off-line minimum-power solution given throughput constraints, and (2) dynamically reprogram FPGA at runtime (which is complementary dynamic voltage frequency scaling) match best workloads they change. The optimization be solved using Mixed-Integer Non-Linear Programming (MINLP) solver, but it very slow. Hence, provide two heuristic methods that improve result quality within bounded time. We use several large designs demonstrate both heuristics obtain comparable results MINLP, solution, much better than cannot optimum amount of also thousands times faster MINLP solver.