2 code implementations • 10 Apr 2024 • Jinghong Chen, Weizhe Lin, Jingbiao Mei, Bill Byrne
The Directed Acyclic Transformer is a fast non-autoregressive (NAR) model that performs well in Neural Machine Translation.
1 code implementation • 13 Feb 2024 • Weizhe Lin, Jingbiao Mei, Jinghong Chen, Bill Byrne
Large Multimodal Models (LMMs) excel in natural language and visual understanding but are challenged by exacting tasks such as Knowledge-based Visual Question Answering (KB-VQA) which involve the retrieval of relevant information from document collections to use in shaping answers to questions.
Ranked #1 on Retrieval on InfoSeek (using extra training data)
no code implementations • 14 Nov 2023 • Jingbiao Mei, Jinghong Chen, Weizhe Lin, Bill Byrne, Marcus Tomalin
Finally, we demonstrate a retrieval-based hateful memes detection system, which is capable of making hatefulness classification based on data unseen in training from a database.
Ranked #2 on Meme Classification on Hateful Memes
1 code implementation • NeurIPS 2023 • Weizhe Lin, Jinghong Chen, Jingbiao Mei, Alexandru Coca, Bill Byrne
FLMR addresses two major limitations in RA-VQA's retriever: (1) the image representations obtained via image-to-text transforms can be incomplete and inaccurate and (2) relevance scores between queries and documents are computed with one-dimensional embeddings, which can be insensitive to finer-grained relevance.
Ranked #1 on Retrieval on OK-VQA
no code implementations • 30 Sep 2022 • Nanyang Ye, Jingbiao Mei, Zhicheng Fang, Yuwen Zhang, Ziqing Zhang, Huaying Wu, Xiaoyao Liang
For neural architecture search space design, instead of conducting neural architecture search on the whole feasible neural architecture search space, we first systematically explore the weight drifting tolerance of different neural network components, such as dropout, normalization, number of layers, and activation functions in which dropout is found to be able to improve the neural network robustness to weight drifting.