Accuracy, Training Time and Hardware Efficiency Trade-Offs for Quantized Neural Networks on FPGAs

Pascal Bacchus; Robert Stewart; Ekaterina Komendantskaya

doi:10.1007/978-3-030-44534-8_10

Accuracy, Training Time and Hardware Efficiency Trade-Offs for Quantized Neural Networks on FPGAs

Pascal Bacchus, Robert Stewart, Ekaterina Komendantskaya

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

10 Citations (Scopus)

29 Downloads (Pure)

Abstract

Neural networks have proven a successful AI approach in many application areas. Some neural network deployments require low inference latency and lower power requirements to be useful e.g. autonomous vehicles and smart drones. Whilst FPGAs meet these requirements, hardware needs of neural networks to execute often exceed FPGA resources.

Emerging industry led frameworks aim to solve this problem by compressing the topology and precision of neural networks, eliminating computations that require memory for execution. Compressing neural networks inevitably comes at the cost of reduced inference accuracy.

This paper uses Xilinx's FINN framework to systematically evaluate the trade-off between precision, inference accuracy, training time and hardware resources of 64 quantized neural networks that perform MNIST character recognition.

We identify sweet spots around 3 bit precision in the quantization design space after training with 40 epochs, minimising both hardware resources and accuracy loss. With enough training, using 2 bit weights achieves almost the same inference accuracy as 3-8 bit weights.

Original language	English
Title of host publication	Applied Reconfigurable Computing. Architectures, Tools, and Applications
Subtitle of host publication	ARC 2020
Publisher	Springer
Pages	121-135
Number of pages	15
ISBN (Electronic)	9783030445348
ISBN (Print)	9783030445331
DOIs	https://doi.org/10.1007/978-3-030-44534-8_10
Publication status	Published - 2020
Event	16th International Symposium on Applied Reconfigurable Computing 2020 - University of Castilla-La Mancha, Toledo, Spain Duration: 1 Apr 2020 → 3 Apr 2020 https://arcoresearch.com/arc2020/

Publication series

Name	Lecture Notes in Computer Science
Volume	12083
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th International Symposium on Applied Reconfigurable Computing 2020
Abbreviated title	ARC2020
Country/Territory	Spain
City	Toledo
Period	1/04/20 → 3/04/20
Internet address	https://arcoresearch.com/arc2020/

Keywords

Deep learning
FPGA
Neural networks
Quantization

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-44534-8_10Licence: Copyright Publisher

Download pdf
© Springer Nature Switzerland AG 2020
Accepted author manuscript, 614 KB

Cite this

@inproceedings{6c1fdad5191245a8937c2803ad40b74e,

title = "Accuracy, Training Time and Hardware Efficiency Trade-Offs for Quantized Neural Networks on FPGAs",

abstract = "Neural networks have proven a successful AI approach in many application areas. Some neural network deployments require low inference latency and lower power requirements to be useful e.g. autonomous vehicles and smart drones. Whilst FPGAs meet these requirements, hardware needs of neural networks to execute often exceed FPGA resources.Emerging industry led frameworks aim to solve this problem by compressing the topology and precision of neural networks, eliminating computations that require memory for execution. Compressing neural networks inevitably comes at the cost of reduced inference accuracy.This paper uses Xilinx's FINN framework to systematically evaluate the trade-off between precision, inference accuracy, training time and hardware resources of 64 quantized neural networks that perform MNIST character recognition.We identify sweet spots around 3 bit precision in the quantization design space after training with 40 epochs, minimising both hardware resources and accuracy loss. With enough training, using 2 bit weights achieves almost the same inference accuracy as 3-8 bit weights.",

keywords = "Deep learning, FPGA, Neural networks, Quantization",

author = "Pascal Bacchus and Robert Stewart and Ekaterina Komendantskaya",

year = "2020",

doi = "10.1007/978-3-030-44534-8_10",

language = "English",

isbn = "9783030445331",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "121--135",

booktitle = "Applied Reconfigurable Computing. Architectures, Tools, and Applications",

address = "Switzerland",

note = "16th International Symposium on Applied Reconfigurable Computing 2020, ARC2020 ; Conference date: 01-04-2020 Through 03-04-2020",

url = "https://arcoresearch.com/arc2020/",

}

Bacchus, P, Stewart, R & Komendantskaya, E 2020, Accuracy, Training Time and Hardware Efficiency Trade-Offs for Quantized Neural Networks on FPGAs. in Applied Reconfigurable Computing. Architectures, Tools, and Applications: ARC 2020. Lecture Notes in Computer Science, vol. 12083, Springer, pp. 121-135, 16th International Symposium on Applied Reconfigurable Computing 2020, Toledo, Spain, 1/04/20. https://doi.org/10.1007/978-3-030-44534-8_10

Accuracy, Training Time and Hardware Efficiency Trade-Offs for Quantized Neural Networks on FPGAs. / Bacchus, Pascal; Stewart, Robert ; Komendantskaya, Ekaterina.

Applied Reconfigurable Computing. Architectures, Tools, and Applications: ARC 2020. Springer, 2020. p. 121-135 (Lecture Notes in Computer Science; Vol. 12083).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Accuracy, Training Time and Hardware Efficiency Trade-Offs for Quantized Neural Networks on FPGAs

AU - Bacchus, Pascal

AU - Stewart, Robert

AU - Komendantskaya, Ekaterina

PY - 2020

Y1 - 2020

N2 - Neural networks have proven a successful AI approach in many application areas. Some neural network deployments require low inference latency and lower power requirements to be useful e.g. autonomous vehicles and smart drones. Whilst FPGAs meet these requirements, hardware needs of neural networks to execute often exceed FPGA resources.Emerging industry led frameworks aim to solve this problem by compressing the topology and precision of neural networks, eliminating computations that require memory for execution. Compressing neural networks inevitably comes at the cost of reduced inference accuracy.This paper uses Xilinx's FINN framework to systematically evaluate the trade-off between precision, inference accuracy, training time and hardware resources of 64 quantized neural networks that perform MNIST character recognition.We identify sweet spots around 3 bit precision in the quantization design space after training with 40 epochs, minimising both hardware resources and accuracy loss. With enough training, using 2 bit weights achieves almost the same inference accuracy as 3-8 bit weights.

AB - Neural networks have proven a successful AI approach in many application areas. Some neural network deployments require low inference latency and lower power requirements to be useful e.g. autonomous vehicles and smart drones. Whilst FPGAs meet these requirements, hardware needs of neural networks to execute often exceed FPGA resources.Emerging industry led frameworks aim to solve this problem by compressing the topology and precision of neural networks, eliminating computations that require memory for execution. Compressing neural networks inevitably comes at the cost of reduced inference accuracy.This paper uses Xilinx's FINN framework to systematically evaluate the trade-off between precision, inference accuracy, training time and hardware resources of 64 quantized neural networks that perform MNIST character recognition.We identify sweet spots around 3 bit precision in the quantization design space after training with 40 epochs, minimising both hardware resources and accuracy loss. With enough training, using 2 bit weights achieves almost the same inference accuracy as 3-8 bit weights.

KW - Deep learning

KW - FPGA

KW - Neural networks

KW - Quantization

UR - http://www.scopus.com/inward/record.url?scp=85083029022&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-44534-8_10

DO - 10.1007/978-3-030-44534-8_10

M3 - Conference contribution

SN - 9783030445331

T3 - Lecture Notes in Computer Science

SP - 121

EP - 135

BT - Applied Reconfigurable Computing. Architectures, Tools, and Applications

PB - Springer

T2 - 16th International Symposium on Applied Reconfigurable Computing 2020

Y2 - 1 April 2020 through 3 April 2020

ER -

Accuracy, Training Time and Hardware Efficiency Trade-Offs for Quantized Neural Networks on FPGAs

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this