Vision Transformers
Compact Convolutional Transformers
Introduced by Hassani et al. in Escaping the Big Data Paradigm with Compact TransformersCompact Convolutional Transformers utilize sequence pooling and replace the patch embedding with a convolutional embedding, allowing for better inductive bias and making positional embeddings optional. CCT achieves better accuracy than ViT-Lite (smaller ViTs) and increases the flexibility of the input parameters.
Source: Escaping the Big Data Paradigm with Compact Transformers
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Image Classification | 5 | 12.82% |
| Retrieval | 2 | 5.13% |
| Text Generation | 2 | 5.13% |
| Decision Making | 2 | 5.13% |
| Semantic Segmentation | 2 | 5.13% |
| Facial Expression Recognition (FER) | 2 | 5.13% |
| Memorization | 2 | 5.13% |
| Image Captioning | 1 | 2.56% |
| Medical Image Classification | 1 | 2.56% |
Convolution
Transformer
