1 code implementation • 8 Oct 2023 • Haowei Lin, Yuntian Gu
Backed by theoretical analysis, this paper advocates for the measurement of the "OOD-ness" of a test case $\boldsymbol{x}$ through the likelihood ratio between out-distribution $\mathcal P_{\textit{out}}$ and in-distribution $\mathcal P_{\textit{in}}$.
no code implementations • NeurIPS 2023 • Guhao Feng, Bohang Zhang, Yuntian Gu, Haotian Ye, Di He, LiWei Wang
By using circuit complexity theory, we first give impossibility results showing that bounded-depth Transformers are unable to directly produce correct answers for basic arithmetic/equation tasks unless the model size grows super-polynomially with respect to the input length.
1 code implementation • 10 Oct 2022 • Quanlin Wu, Hang Ye, Yuntian Gu, Huishuai Zhang, LiWei Wang, Di He
In this paper, we propose a new self-supervised method, which is called Denoising Masked AutoEncoders (DMAE), for learning certified robust classifiers of images.
no code implementations • 22 Jun 2022 • Quanlin Wu, Hang Ye, Yuntian Gu
In this paper, we propose a novel guided diffusion purification approach to provide a strong defense against adversarial attacks.