Receding Neuron Importances for Structured Pruning

no code implementations • 13 Apr 2022 • Mihai Suteu, Yike Guo

To tackle this issue, we introduce a simple BatchNorm variation with bounded scaling parameters, based on which we design a novel regularisation term that suppresses only neurons with low importance.

Regularizing Deep Multi-Task Networks using Orthogonal Gradients

1 code implementation • 14 Dec 2019 • Mihai Suteu, Yike Guo

Deep neural networks are a promising approach towards multi-task learning because of their capability to leverage knowledge across domains and learn general purpose representations.

Multi-Task Learning