Neural network problems (also referred to as “deep learning networks” or “models”) are encoded in a format that is not directly executable; they need to be translated into an executable form. This translation is the job of the neural network toolchain. Such translation is a highly complex task, involving multiple iterations over the problem description, applying sophisticated mathematical transformations at multiple levels of abstraction.
While traditional programming language toolchains are mature technologies with established solutions, deep learning toolchains are being developed at a rapid pace. Thus, it is more difficult to judge which deep learning toolchain is best, not only in terms of existing capability, but also future potential. Reservoir Labs has extensive experience with traditional compiler toolchains and applies that background to deep learning toolchains.
While building a simple neural network compiler is straightforward, to get good performance and high utilization of hardware requires considering a range of different design choices that can have significant effects on performance scalability and the ability to perform advanced optimizations. Effects such as processor utilization, memory latency, and concurrency create a complex web of dependencies requiring investigation of a large optimization space.
Reservoir is working with the leading neural network toolchains, such as TVM and Glow, to develop solutions for neural network hardware architectures. Our engineers adapt the general frameworks to specific architectures, accounting for specific memory hierarchies and instruction sets, and perform high level optimizations such as operator fusion. The neural network toolchain also achieves greatest leverage when it incorporates robust low-level compiler support, such as provided by LLVM, which is a complementary capability of Reservoir.