Elektronika ir Elektrotechnika
(Kaunas University of Technology)
•23 Aug 2021
Binary Quantization Analysis of Neural Networks Weights on MNIST Dataset
Perić, Zoran • Denić, Bojan • Savić, Milan • Vučić, Nikola • Simić, Nikola
Abstract
This paper considers the design of a binary scalar quantizer of Laplacian source and its application in compressed neural networks. The quantizer performance is investigated in a wide dynamic range of data variances, and for that purpose, we derive novel closed-form expressions. Moreover, we propose two selection criteria for the variance range of interest. Binary quantizers are further implemented for compressing neural network weights and its performance is analysed for a simple classification task. Good matching between theory and experiment is observed and a great possibility for implementation is indicated.
Funding
III44006
Author: Perić, Zoran • Denić, Bojan • Savić, Milan • Vučić, Nikola • Simić, Nikola
Abstract
This paper considers the design of a binary scalar quantizer of Laplacian source and its application in compressed neural networks. The quantizer performance is investigated in a wide dynamic range of data variances, and for that purpose, we derive novel closed-form expressions. Moreover, we propose two selection criteria for the variance range of interest. Binary quantizers are further implemented for compressing neural network weights and its performance is analysed for a simple classification task. Good matching between theory and experiment is observed and a great possibility for implementation is indicated.