Search Results for author:
Found 4 papers, 2 papers with code
When does CLIP generalize better than unimodal models? When judging human-centric concepts
We compare CLIP’s visual stream against two visually trained networks and CLIP’s textual stream against two linguistically trained networks, as well as multimodal combinations of these networks.
Zero-shot cross-modal transfer of Reinforcement Learning policies through a Global Workspace
First, we train a 'Global Workspace' to exploit information collected about the environment via two input modalities (a visual input, or an attribute vector representing the state of the agent and/or its environment).
Semi-supervised Multimodal Representation Learning through a Global Workspace
Recent deep learning models can efficiently combine inputs from different modalities (e. g., images and text) and learn to align their latent representations, or to translate signals from one domain to another (as in image captioning, or text-to-image generation).
Does language help generalization in vision models?
Vision models trained on multimodal datasets can benefit from the wide availability of large image-caption datasets.
