ProgressiveSpinalNet architecture for FC layers

21 Mar 2021  ·  Praveen Chopra ·

In deeplearning models the FC (fully connected) layer has biggest important role for classification of the input based on the learned features from previous layers. The FC layers has highest numbers of parameters and fine-tuning these large numbers of parameters, consumes most of the computational resources, so in this paper it is aimed to reduce these large numbers of parameters significantly with improved performance. The motivation is inspired from SpinalNet and other biological architecture. The proposed architecture has a gradient highway between input to output layers and this solves the problem of diminishing gradient in deep networks. In this all the layers receives the input from previous layers as well as the CNN layer output and this way all layers contribute in decision making with last layer. This approach has improved classification performance over the SpinalNet architecture and has SOTA performance on many datasets such as Caltech101, KMNIST, QMNIST and EMNIST. The source code is available at https://github.com/praveenchopra/ProgressiveSpinalNet.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Fine-Grained Image Classification Bird-225 Pre trained wide-resnet-101 Accuracy 99.55 # 2
Fine-Grained Image Classification Caltech-101 Pre trained wide-resnet-101 Accuracy 97.76 # 7
Fine-Grained Image Classification EMNIST-Digits VGG-5 Accuracy 99.82 # 1
Fine-Grained Image Classification EMNIST-Letters VGG-5 Accuracy 95.86 # 1
Fine-Grained Image Classification Fruits-360 Pre trained wide-resnet-101 Accuracy 99.97 # 1
Fine-Grained Image Classification Kuzushiji-MNIST VGG-5 Accuracy 98.98 # 1
Fine-Grained Image Classification MNIST Vanilla FC layer only Accuracy 98.19 # 1
Fine-Grained Image Classification QMNIST VGG-5 Accuracy 99.6867 # 1
Fine-Grained Image Classification STL-10 Pre trained wide-resnet-101 Accuracy 98.18 # 1

Methods