Bitpruning Learning Bitlengths For Aggressive And Accurate Quantization | Awesome Learning to Hash Add your paper to Learning2Hash

Awesome Learning to Hash

A Webpage dedicated to the latest research on Hash Function Learning. Maintained by Sean Moran.

Contact Sean Moran about this survey or website. Made with Jekyll and Hyde.

Bitpruning Learning Bitlengths For Aggressive And Accurate Quantization

Nikolić Miloš, Hacene Ghouthi Boukli, Bannon Ciaran, Lascorz Alberto Delmas, Courbariaux Matthieu, Bengio Yoshua, Gripon Vincent, Moshovos Andreas. Arxiv 2020

[Paper]    
ARXIV Quantisation Supervised

Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time and energy benefits on existing hardware designs that support short bitlengths. However, the question of finding the minimum bitlength for a desired accuracy remains open. We introduce a training method for minimizing inference bitlength at any granularity while maintaining accuracy. Namely, we propose a regularizer that penalizes large bitlength representations throughout the architecture and show how it can be modified to minimize other quantifiable criteria, such as number of operations or memory footprint. We demonstrate that our method learns thrifty representations while maintaining accuracy. With ImageNet, the method produces an average per layer bitlength of 4.13, 3.76 and 4.36 bits on AlexNet, ResNet18 and MobileNet V2 respectively, remaining within 2.0%, 0.5% and 0.5% of the base TOP-1 accuracy.

Similar Work