Although deep neural networks have enjoyed remarkable success across a wide variety of tasks, their ever-increasing size also imposes significant overhead on deployment. To compress these models, knowledge distillation was proposed to transfer from cumbersome (teacher) network into lightweight (student) network. However, guidance teacher does not always improve the generalization students, espe...

Although language modeling has been trending upwards steadily, models available for low-resourced languages are limited to large multilingual such as mBERT and XLM-RoBERTa, which come with significant overheads deployment vis-à-vis their model size, inference speeds, etc. We attempt tackle this problem by proposing a novel methodology apply knowledge distillation techniques filter language-spec...

We propose a family of error-detecting stabilizer codes with an encoding rate of 1/3 that permit a transversal implementation of the gate T = exp (−iπZ/8) on all logical qubits. These codes are used to construct protocols for distilling high-quality “magic” states T |+〉 by Clifford group gates and Pauli measurements. The distillation overhead scales asO( log (1/ )), where is the output accuracy...