Flexible acceleration of convolutions on FPGAs: Planning NEURAghe 2.0

Convolutional Neural Networks are commonly employed in applications involving Computer Vision tasks like image/video classification/recognition/segmentation. The increasing focus of the community on this topic, has generated a wide scope of approaches that use different kernel shapes and techniques for executing convolutions with respect to the classic one, such as for example separable convolutions, deformable convolutions or deconvolutions ([4, 5]), frequently used in semantic segmentation tasks ([23, 13]). While it is common knowledge that FPGAs can be used to accelerate classic Convolutional layers in CNNs, there is limited literature about FPGA-based accelerators supporting less regular and common processing kernels ([20]). In our research, starting from the previous experience acquired developing NEURAghe, we plan to improve flexibility of CNN accelerators and to study new methodologies to improve efficiency on the previously mentioned use-cases. As a first experiment we focus on layered approaches based on 1D convolutions, that, as indicated by several recent research results, can be effectively used to classify and segment time series and sequences, as well as in tasks involving sequence modeling. In multiple scenarios a convolution approach applied on the time dimension, hereafter called Temporal Convolution Network (TCN) can outperform classic strategies relying on recurrent networks in terms of accuracy and training time. We modified NEURAghe to support TCN and validate results on an ECG-classification benchmark, achieving up to 95% efficiency in terms of GOPS/s with respect to the accelerator peak performance.