1 code implementation • 2 Mar 2024 • Kaituo Feng, Changsheng Li, Dongchun Ren, Ye Yuan, Guoren Wang
However, the oversized neural networks render them impractical for deployment on resource-constrained systems, which unavoidably requires more computational time and resources during reference. To handle this, knowledge distillation offers a promising approach that compresses models by enabling a smaller student model to learn from a larger teacher model.
no code implementations • 18 Oct 2023 • Shiye Wang, Kaituo Feng, Changsheng Li, Ye Yuan, Guoren Wang
Typical Convolutional Neural Networks (ConvNets) depend heavily on large amounts of image data and resort to an iterative optimization algorithm (e. g., SGD or Adam) to learn network parameters, which makes training very time- and resource-intensive.
no code implementations • 2 Jul 2023 • Kaituo Feng, Yikun Miao, Changsheng Li, Ye Yuan, Guoren Wang
Knowledge distillation (KD) has shown to be effective to boost the performance of graph neural networks (GNNs), where the typical objective is to distill knowledge from a deeper teacher GNN into a shallower student GNN.
1 code implementation • 27 Mar 2023 • Kaituo Feng, Changsheng Li, Xiaolu Zhang, Jun Zhou
This will bring two big challenges to the existing dynamic GNN methods: (i) How to dynamically propagate appropriate information in an open temporal graph, where new class nodes are often linked to old class nodes.
1 code implementation • 22 Jul 2022 • Hanjie Li, Changsheng Li, Kaituo Feng, Ye Yuan, Guoren Wang, Hongyuan Zha
By this means, we can adaptively propagate knowledge to other nodes for learning robust node embedding representations.
no code implementations • 14 Jun 2022 • Kaituo Feng, Changsheng Li, Ye Yuan, Guoren Wang
Knowledge distillation (KD) has demonstrated its effectiveness to boost the performance of graph neural networks (GNNs), where its goal is to distill knowledge from a deeper teacher GNN into a shallower student GNN.