no code implementations • 28 May 2024 • Youngwan Lee, Jeffrey Ryan Willette, Jonghee Kim, Sung Ju Hwang
To further investigate the reason for better generalization of the self-supervised ViT when trained by MAE (MAE-ViT) and the effect of the gradient correction of RC-MAE from the perspective of optimization, we visualize the loss landscapes of the self-supervised vision transformer by both MAE and RC-MAE and compare them with the supervised ViT (Sup-ViT).
no code implementations • ICLR 2022 • Jeffrey Ryan Willette, Hae Beom Lee, Juho Lee, Sung Ju Hwang
Numerous recent works utilize bi-Lipschitz regularization of neural network layers to preserve relative distances between data instances in the feature spaces of each layer.
no code implementations • 1 Jan 2021 • Jeffrey Ryan Willette, Juho Lee, Sung Ju Hwang
We demonstrate the effectiveness of our method and validate its performance on both classification and regression problems by applying it to the training of recent state-of-the-art neural network models.