no code implementations • 19 May 2024 • Youbang Sun, Shixiang Chen, Alfredo Garcia, Shahin Shahrampour
Many classical and modern machine learning algorithms require solving optimization tasks under orthogonal constraints.
no code implementations • 18 Sep 2023 • Hao Sun, Li Shen, Shixiang Chen, Jingwei Sun, Jing Li, Guangzhong Sun, DaCheng Tao
Federated learning is an emerging distributed machine learning method, enables a large number of clients to train a model without exchanging their local data.
1 code implementation • 19 May 2023 • Yan Sun, Li Shen, Shixiang Chen, Liang Ding, DaCheng Tao
In federated learning (FL), a cluster of local clients are chaired under the coordination of the global server and cooperatively train one model with privacy protection.
no code implementations • 31 Mar 2023 • Jinxin Wang, Jiang Hu, Shixiang Chen, Zengde Deng, Anthony Man-Cho So
We focus on a class of non-smooth optimization problems over the Stiefel manifold in the decentralized setting, where a connected network of $n$ agents cooperatively minimize a finite-sum objective function with each component being weakly convex in the ambient Euclidean space.
no code implementations • 1 Mar 2023 • Hao Sun, Li Shen, Qihuang Zhong, Liang Ding, Shixiang Chen, Jingwei Sun, Jing Li, Guangzhong Sun, DaCheng Tao
Integrating SAM with adaptive learning rate and momentum acceleration, dubbed AdaSAM, has already been explored empirically to train large-scale deep neural networks without theoretical guarantee due to the triple difficulties in analyzing the coupled perturbation step, adaptive learning rate and momentum step.
no code implementations • 1 Mar 2023 • Chao Xue, Wei Liu, Shuai Xie, Zhenfang Wang, Jiaxing Li, Xuyang Peng, Liang Ding, Shanshan Zhao, Qiong Cao, Yibo Yang, Fengxiang He, Bohua Cai, Rongcheng Bian, Yiyan Zhao, Heliang Zheng, Xiangyang Liu, Dongkai Liu, Daqing Liu, Li Shen, Chang Li, Shijin Zhang, Yukang Zhang, Guanpu Chen, Shixiang Chen, Yibing Zhan, Jing Zhang, Chaoyue Wang, DaCheng Tao
Automated machine learning (AutoML) seeks to build ML models with minimal human effort.
no code implementations • 24 May 2022 • Linrui Zhang, Li Shen, Long Yang, Shixiang Chen, Bo Yuan, Xueqian Wang, DaCheng Tao
Safe reinforcement learning aims to learn the optimal policy while satisfying safety constraints, which is essential in real-world applications.
1 code implementation • 17 Mar 2022 • Yibo Yang, Shixiang Chen, Xiangtai Li, Liang Xie, Zhouchen Lin, DaCheng Tao
Modern deep neural networks for classification usually jointly learn a backbone for representation and a linear classifier to output the logit of each class.
Ranked #26 on Long-tail Learning on CIFAR-10-LT (ρ=100)
1 code implementation • 14 Feb 2021 • Shixiang Chen, Alfredo Garcia, Mingyi Hong, Shahin Shahrampour
The global function is represented as a finite sum of smooth local functions, where each local function is associated with one agent and agents communicate with each other over an undirected connected graph.
no code implementations • 22 Jan 2021 • Shixiang Chen, Alfredo Garcia, Mingyi Hong, Shahin Shahrampour
We study the convergence properties of Riemannian gradient method for solving the consensus problem (for an undirected connected graph) over the Stiefel manifold.
no code implementations • 18 Jul 2020 • Zhongruo Wang, Bingyuan Liu, Shixiang Chen, Shiqian Ma, Lingzhou Xue, Hongyu Zhao
This paper considers a widely adopted model for SSC, which can be formulated as an optimization problem over the Stiefel manifold with nonsmooth and nonconvex objective.
no code implementations • 5 May 2020 • Shixiang Chen, Zengde Deng, Shiqian Ma, Anthony Man-Cho So
Second, we propose a stochastic variant of ManPPA called StManPPA, which is well suited for large-scale computation, and establish its sublinear convergence rate.
no code implementations • 28 Apr 2020 • Shixiang Chen, Alfredo Garcia, Shahin Shahrampour
In this paper, we propose a distributed implementation of the stochastic subgradient method with a theoretical guarantee.
1 code implementation • 12 Nov 2019 • Xiao Li, Shixiang Chen, Zengde Deng, Qing Qu, Zhihui Zhu, Anthony Man Cho So
To the best of our knowledge, these are the first convergence guarantees for using Riemannian subgradient-type methods to optimize a class of nonconvex nonsmooth functions over the Stiefel manifold.
no code implementations • 27 Mar 2019 • Shixiang Chen, Shiqian Ma, Lingzhou Xue, Hui Zou
Sparse principal component analysis (PCA) and sparse canonical correlation analysis (CCA) are two essential techniques from high-dimensional statistics and machine learning for analyzing large-scale data.
no code implementations • 2 Nov 2018 • Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, Tong Zhang
We prove that the proposed method globally converges to a stationary point.
no code implementations • NeurIPS 2017 • Shixiang Chen, Shiqian Ma, Wei Liu
In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems.