Array languages make neural networks fast

Artjoms Šinkarovs; Hans-Nikolai Vießmann; Sven-Bodo Scholz

doi:10.1145/3460944.3464312

Array languages make neural networks fast

Artjoms Šinkarovs, Hans-Nikolai Vießmann, Sven-Bodo Scholz

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

85 Downloads (Pure)

Abstract

Most implementations of machine learning algorithms are based on special-purpose frameworks such as TensorFlow or PyTorch. While these frameworks are convenient to use, they introduce multi-million lines of code dependency that one has to trust, understand and potentially modify. As an alternative, this paper investigates a direct implementation of a state of the art Convolutional Neural Network (CNN) in an array language. While our implementation requires 150 lines of code to define the special-purpose operators needed for CNNs, which are readily provided through frameworks such as TensorFlow and PyTorch, our implementation outperforms these frameworks by factors 2 and 3 on a fixed set of hardware - a 64-core GPU-accelerated machine; for a simple example network. The resulting specification is written in a rank-polymorphic data-parallel style, and it can be immediately leveraged by optimising compilers. Indeed, array languages make neural networks fast.

Original language	English
Title of host publication	ARRAY 2021: Proceedings of the 7th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming
Editors	Tze Meng Low, Jeremy Gibbons
Publisher	Association for Computing Machinery
Pages	39-50
Number of pages	12
ISBN (Electronic)	9781450384667
DOIs	https://doi.org/10.1145/3460944.3464312
Publication status	Published - 17 Jun 2021
Event	7th ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming 2021 - Virtual, Online, Canada Duration: 21 Jun 2021 → …

Conference

Conference	7th ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming 2021
Abbreviated title	ARRAY 2021
Country/Territory	Canada
City	Virtual, Online
Period	21/06/21 → …

Keywords

array language
machine learning

ASJC Scopus subject areas

Theoretical Computer Science
Computational Theory and Mathematics

Access to Document

10.1145/3460944.3464312Licence: CC BY

Download pdf
© 2021 Copyright held by the owner/author(s).
Final published version, 425 KBLicence: CC BY

Cite this

@inproceedings{22dc339953ba402c8984153d3712f794,

title = "Array languages make neural networks fast",

abstract = "Most implementations of machine learning algorithms are based on special-purpose frameworks such as TensorFlow or PyTorch. While these frameworks are convenient to use, they introduce multi-million lines of code dependency that one has to trust, understand and potentially modify. As an alternative, this paper investigates a direct implementation of a state of the art Convolutional Neural Network (CNN) in an array language. While our implementation requires 150 lines of code to define the special-purpose operators needed for CNNs, which are readily provided through frameworks such as TensorFlow and PyTorch, our implementation outperforms these frameworks by factors 2 and 3 on a fixed set of hardware - a 64-core GPU-accelerated machine; for a simple example network. The resulting specification is written in a rank-polymorphic data-parallel style, and it can be immediately leveraged by optimising compilers. Indeed, array languages make neural networks fast. ",

keywords = "array language, machine learning",

author = "Artjoms {\v S}inkarovs and Hans-Nikolai Vie{\ss}mann and Sven-Bodo Scholz",

note = "Funding Information: We would like to thank our reviewers for their constructive suggestions. This work is supported by the Engineering and Physical Sciences Research Council through grants EP/N028201/1 and EP/L016834/1. Publisher Copyright: {\textcopyright} 2021 Owner/Author.; 7th ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming 2021, ARRAY 2021 ; Conference date: 21-06-2021",

year = "2021",

month = jun,

day = "17",

doi = "10.1145/3460944.3464312",

language = "English",

pages = "39--50",

editor = "Low, {Tze Meng} and Jeremy Gibbons",

booktitle = "ARRAY 2021: Proceedings of the 7th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming",

publisher = "Association for Computing Machinery",

address = "United States",

}

Šinkarovs, A, Vießmann, H-N & Scholz, S-B 2021, Array languages make neural networks fast. in TM Low & J Gibbons (eds), ARRAY 2021: Proceedings of the 7th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming. Association for Computing Machinery, pp. 39-50, 7th ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming 2021, Virtual, Online, Canada, 21/06/21. https://doi.org/10.1145/3460944.3464312

Array languages make neural networks fast. / Šinkarovs, Artjoms; Vießmann, Hans-Nikolai; Scholz, Sven-Bodo.

ARRAY 2021: Proceedings of the 7th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming. ed. / Tze Meng Low; Jeremy Gibbons. Association for Computing Machinery, 2021. p. 39-50.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Array languages make neural networks fast

AU - Šinkarovs, Artjoms

AU - Vießmann, Hans-Nikolai

AU - Scholz, Sven-Bodo

N1 - Funding Information: We would like to thank our reviewers for their constructive suggestions. This work is supported by the Engineering and Physical Sciences Research Council through grants EP/N028201/1 and EP/L016834/1. Publisher Copyright: © 2021 Owner/Author.

PY - 2021/6/17

Y1 - 2021/6/17

N2 - Most implementations of machine learning algorithms are based on special-purpose frameworks such as TensorFlow or PyTorch. While these frameworks are convenient to use, they introduce multi-million lines of code dependency that one has to trust, understand and potentially modify. As an alternative, this paper investigates a direct implementation of a state of the art Convolutional Neural Network (CNN) in an array language. While our implementation requires 150 lines of code to define the special-purpose operators needed for CNNs, which are readily provided through frameworks such as TensorFlow and PyTorch, our implementation outperforms these frameworks by factors 2 and 3 on a fixed set of hardware - a 64-core GPU-accelerated machine; for a simple example network. The resulting specification is written in a rank-polymorphic data-parallel style, and it can be immediately leveraged by optimising compilers. Indeed, array languages make neural networks fast.

AB - Most implementations of machine learning algorithms are based on special-purpose frameworks such as TensorFlow or PyTorch. While these frameworks are convenient to use, they introduce multi-million lines of code dependency that one has to trust, understand and potentially modify. As an alternative, this paper investigates a direct implementation of a state of the art Convolutional Neural Network (CNN) in an array language. While our implementation requires 150 lines of code to define the special-purpose operators needed for CNNs, which are readily provided through frameworks such as TensorFlow and PyTorch, our implementation outperforms these frameworks by factors 2 and 3 on a fixed set of hardware - a 64-core GPU-accelerated machine; for a simple example network. The resulting specification is written in a rank-polymorphic data-parallel style, and it can be immediately leveraged by optimising compilers. Indeed, array languages make neural networks fast.

KW - array language

KW - machine learning

UR - http://www.scopus.com/inward/record.url?scp=85109026704&partnerID=8YFLogxK

U2 - 10.1145/3460944.3464312

DO - 10.1145/3460944.3464312

M3 - Conference contribution

AN - SCOPUS:85109026704

SP - 39

EP - 50

BT - ARRAY 2021: Proceedings of the 7th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming

A2 - Low, Tze Meng

A2 - Gibbons, Jeremy

PB - Association for Computing Machinery

T2 - 7th ACM SIGPLAN International Workshop on Libraries, Languages, and Compilers for Array Programming 2021

Y2 - 21 June 2021

ER -

Array languages make neural networks fast

Abstract

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this