2 code implementations • 12 Nov 2019 • Xinyan Dai, Xiao Yan, Kelvin K. W. Ng, Jie Liu, James Cheng
In this paper, we present a new angle to analyze the quantization error, which decomposes the quantization error into norm error and direction error.
1 code implementation • 12 Nov 2019 • Xinyan Dai, Xiao Yan, Kaiwen Zhou, Han Yang, Kelvin K. W. Ng, James Cheng, Yu Fan
In particular, at the high compression ratio end, HSQ provides a low per-iteration communication cost of $O(\log d)$, which is favorable for federated learning.
no code implementations • 26 Feb 2018 • Fanhua Shang, Yuanyuan Liu, Kaiwen Zhou, James Cheng, Kelvin K. W. Ng, Yuichi Yoshida
In order to make sufficient decrease for stochastic optimization, we design a new sufficient decrease criterion, which yields sufficient decrease versions of stochastic variance reduction algorithms such as SVRG-SD and SAGA-SD as a byproduct.