Self-Supervised Learning

Edit

General • 46 methods

Self-Supervised Learning refers to a category of methods where we learn representations in a self-supervised way (i.e without labels). These methods generally involve a pretext task that is solved to learn a good representation and a loss function to learn with. Below you can find a continuously updating list of self-supervised methods.

Methods

Add a Method

Method	Year	Papers
Inpainting Context Encoders: Feature Learning by Inpainting	2016	634
MAE Masked Autoencoders Are Scalable Vision Learners	2021	412
SimCLR A Simple Framework for Contrastive Learning of Visual Representations	2020	202
Colorization Colorful Image Colorization	2016	181
MoCo Momentum Contrast for Unsupervised Visual Representation Learning	2019	128
Jigsaw Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles	2016	123
BYOL Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning	2020	108
DINO Emerging Properties in Self-Supervised Vision Transformers	2021	100
Contrastive Predictive Coding Representation Learning with Contrastive Predictive Coding	2018	95
Barlow Twins Barlow Twins: Self-Supervised Learning via Redundancy Reduction	2021	52
SwAV Unsupervised Learning of Visual Features by Contrasting Cluster Assignments	2020	46
MoCo v2 Improved Baselines with Momentum Contrastive Learning	2020	28
COLA Contrastive Learning of General-Purpose Audio Representations	2020	25
BiGAN Adversarial Feature Learning	2016	14
SEER Self-supervised Pretraining of Visual Features in the Wild	2021	13
CMCL UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning	2020	10
DeepCluster Deep Clustering for Unsupervised Learning of Visual Features	2018	9
NCL Neighborhood Contrastive Learning Applied to Online Patient Monitoring	2021	9
Dense Contrastive Learning Dense Contrastive Learning for Self-Supervised Visual Pre-Training	2020	8
PIRL Self-Supervised Learning of Pretext-Invariant Representations	2019	7
Mirror-BERT Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders	2021	6
Contrastive Multiview Coding Contrastive Multiview Coding	2019	5
ReLIC Representation Learning via Invariant Causal Mechanisms	2020	5
BigBiGAN Large Scale Adversarial Representation Learning	2019	4
RotNet RotNet: Fast and Scalable Estimation of Stellar Rotation Periods Using Convolutional Neural Networks	2020	4
NPID Unsupervised Feature Learning via Non-Parametric Instance Discrimination	2018	4
SSDS OCTAve: 2D en face Optical Coherence Tomography Angiography Vessel Segmentation in Weakly-Supervised Learning with Locality Augmentation	2022	4
MARLIN MARLIN: Masked Autoencoder for facial video Representation LearnINg	2022	4
NNCLR With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations	2021	4
Graph Contrastive Coding GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training	2020	3
DeCLUTR DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations	2020	3
CVRL Spatiotemporal Contrastive Video Representation Learning	2020	3
CRISS Cross-lingual Retrieval for Iterative Self-Supervised Training	2020	3
IMGEP Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning	2017	2
PASE+ Multi-task self-supervised learning for Robust Speech Recognition	2020	2
MoBY Self-Supervised Learning with Swin Transformers	2021	2
CPC v2 Data-Efficient Image Recognition with Contrastive Predictive Coding	2019	1
ALP-GMM Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments	2019	1
ClusterFit ClusterFit: Improving Generalization of Visual Representations	2019	1
NPID++ Self-Supervised Learning of Pretext-Invariant Representations	2019	1
VC R-CNN Visual Commonsense R-CNN	2020	1
ParamCrop ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning	2021	1
Trans-Encoder Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations	2021	1
Magnification Prior Contrastive Similarity Magnification Prior: A Self-Supervised Method for Learning Representations on Breast Cancer Histopathological Images	2022	1
Internet Explorer Internet Explorer: Targeted Representation Learning on the Open Web	2023	1

Self-Supervised Learning Edit

Methods Add a Method

Self-Supervised Learning

Edit

Methods

Add a Method