Search Results for author:
Found 406 papers, 149 papers with code
GUTS at SemEval-2022 Task 4: Adversarial Training and Balancing Methods for Patronizing and Condescending Language Detection
For Subtask B, framed as a multi-label classification problem, we utilize various improved multi-label cross-entropy loss functions and analyze the performance of our method.
Friendly Topic Assistant for Transformer Based Abstractive Summarization
Abstractive document summarization is a comprehensive task including document understanding and summary generation, in which area Transformer-based models have achieved the state-of-the-art performance.
Incorporating Instructional Prompts into a Unified Generative Framework for Joint Multiple Intent Detection and Slot Filling
Specifically, we design 5-type templates as instructional prompts, and each template includes a question that acts as the driver to teach UGEN to grasp the paradigm, options that list the candidate intents or slots to reduce the answer search space, and the context denotes original utterance.
BIRNAT: Bidirectional Recurrent Neural Networks with Adversarial Training for Video Snapshot Compressive Imaging
This measurement and the modulation masks are fed into our Recurrent Neural Network (RNN) to reconstruct the desired high-speed frames.
WordNet Troponymy and Extraction of “Manner-Result” Relations
The procedure of extraction includes three steps and the results are based on the analysis of the whole set of verbs in WordNet.
Translate-Train Embracing Translationese Artifacts
Translate-train is a general training approach to multilingual tasks.
Multi-view Content-aware Indexing for Long Document Retrieval
As they do not consider content structures, the resultant chunks can exclude vital information or include irrelevant content.
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
The quadratic complexity and weak length extrapolation of Transformers limits their ability to scale to long sequences, and while sub-quadratic solutions like linear attention and state space models exist, they empirically underperform Transformers in pretraining efficiency and downstream task accuracy.
SSwsrNet: A Semi-Supervised Few-Shot Learning Framework for Wireless Signal Recognition
Moreover, a modular semi-supervised learning method that combines labeled and unlabeled data using MixMatch is exploited to further improve the classification performance under few-sample conditions.
Toward Inference-optimal Mixture-of-Expert Large Language Models
Like dense models, training MoEs requires answering the same question: given a training budget, what is the optimal allocation on the model size and number of tokens?
DPA-Net: Structured 3D Abstraction from Sparse Views via Differentiable Primitive Assembly
We present a differentiable rendering framework to learn structured 3D abstractions in the form of primitive assemblies from sparse RGB images capturing a 3D object.
Multi-Task Dense Prediction via Mixture of Low-Rank Experts
Furthermore, to control the parameters and computational cost brought by the increase in the number of experts, we take inspiration from LoRA and propose to leverage the low-rank format of a vanilla convolution in the expert network.
Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion
Through the text semantic encoder and semantic interaction fusion decoder, Text-IF is accessible to the all-in-one infrared and visible image degradation-aware processing and the interactive flexible fusion outcomes.
Empowering Segmentation Ability to Multi-modal Large Language Models
Multi-modal large language models (MLLMs) can understand image-language prompts and demonstrate impressive reasoning ability.
TAPTR: Tracking Any Point with Transformers as Detection
Based on the observation that point tracking bears a great resemblance to object detection and tracking, we borrow designs from DETR-like algorithms to address the task of TAP.
Learning Transferable Time Series Classifier with Cross-Domain Pre-training from Language Model
To address this challenge, we propose CrossTimeNet, a novel cross-domain SSL learning framework to learn transferable knowledge from various domains to largely benefit the target downstream task.
AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Adversarial Visual-Instructions
To bridge this gap, we introduce AVIBench, a framework designed to analyze the robustness of LVLMs when facing various adversarial visual-instructions (AVIs), including four types of image-based AVIs, ten types of text-based AVIs, and nine types of content bias AVIs (such as gender, violence, cultural, and racial biases, among others).
Towards Personalized Evaluation of Large Language Models with An Anonymous Crowd-Sourcing Platform
Through this gateway, users have the opportunity to submit their questions, testing the models on a personalized and potentially broader range of capabilities.
Empowering Sequential Recommendation from Collaborative Signals and Semantic Relatedness
It is also significant to model the \textit{semantic relatedness} reflected in content features, e. g., images and text.
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
To address this issue, we introduce Chatbot Arena, an open platform for evaluating LLMs based on human preferences.
MeaCap: Memory-Augmented Zero-shot Image Captioning
The framework of MeaCap achieves the state-of-the-art performance on a series of zero-shot IC settings.
Improving Adversarial Energy-Based Model via Diffusion Process
Generative models have shown strong generation ability while efficient likelihood estimation is less explored.
CLLMs: Consistency Large Language Models
Parallel decoding methods such as Jacobi decoding show promise for more efficient LLM inference as it breaks the sequential nature of the LLM decoding process and transforms it into parallelizable computation.
AnaMoDiff: 2D Analogical Motion Diffusion via Disentangled Denoising
We present AnaMoDiff, a novel diffusion-based method for 2D motion analogies that is applied to raw, unannotated videos of articulated characters.
CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization
First, we design the coupled neural shape (CNS) representation for supporting 3D shape editing.
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Autoregressive decoding of large language models (LLMs) is memory bandwidth bounded, resulting in high latency and significant wastes of the parallel processing power of modern accelerators.
APIServe: Efficient API Support for Large-Language Model Inferencing
Large language models are increasingly integrated with external tools and APIs like ChatGPT plugins to extend their capability beyond language-centric tasks.
Activity Detection for Massive Connectivity in Cell-free Networks with Unknown Large-scale Fading, Channel Statistics, Noise Variance, and Activity Probability: A Bayesian Approach
This problem is even more severe in cell-free networks as there are many of these parameters to be acquired.
Overview of Sensing Attacks on Autonomous Vehicle Technologies and Impact on Traffic Flow
While perception systems in Connected and Autonomous Vehicles (CAVs), which encompass both communication technologies and advanced sensors, promise to significantly reduce human driving errors, they also expose CAVs to various cyberattacks.
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM).
Parameter-Efficient Conversational Recommender System as a Language Processing Task
Conversational recommender systems (CRS) aim to recommend relevant items to users by eliciting user preference through natural language conversation.
Focaler-IoU: More Focused Intersection over Union Loss
Existing researchs improve regression performance by utilizing the geometric relationship between bounding boxes, while ignoring the impact of difficult and easy sample distribution on bounding box regression.
Learning Implicit Representation for Reconstructing Articulated Objects
Our method simultaneously estimates the visible (explicit) representation (3D shapes, colors, camera parameters) and the implicit skeletal representation, from motion cues in the object video without 3D supervision.
Empirical Evidence for the Fragment level Understanding on Drug Molecular Structure of LLMs
AI for drug discovery has been a research hotspot in recent years, and SMILES-based language models has been increasingly applied in drug molecular design.
Crafter: Facial Feature Crafting against Inversion-based Identity Theft on Deep Models
However, such methods can not defend against adaptive attacks, in which an attacker takes a countermove against a known defence strategy.
SnapCap: Efficient Snapshot Compressive Video Captioning
To address these problems, in this paper, we propose a novel VC pipeline to generate captions directly from the compressed measurement, which can be captured by a snapshot compressive sensing camera and we dub our model SnapCap.
Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding
We propose the Chain-of-Table framework, where tabular data is explicitly used in the reasoning chain as a proxy for intermediate thoughts.
Ranked #3 on
Table-based Fact Verification
on TabFact
MPN: Leveraging Multilingual Patch Neuron for Cross-lingual Model Editing
Large language models are known for encoding a vast amount of factual knowledge, but they often becomes outdated due to the ever-changing nature of external information.
FED-NeRF: Achieve High 3D Consistency and Temporal Coherence for Face Video Editing on Dynamic NeRF
However, achieving simultaneously multi-view consistency and temporal coherence while editing video sequences remains a formidable challenge.
Shape-IoU: More Accurate Metric considering Bounding Box Shape and Scale
As an important component of the detector localization branch, bounding box regression loss plays a significant role in object detection tasks.
Deep Unfolding Network with Spatial Alignment for multi-modal MRI reconstruction
In this paper, we construct a novel Deep Unfolding Network with Spatial Alignment, termed DUN-SA, to appropriately embed the spatial alignment task into the reconstruction process.
Unlocking the Potential of Large Language Models for Explainable Recommendations
In this study, we propose LLMXRec, a simple yet effective two-stage explainable recommendation framework aimed at further boosting the explanation quality by employing LLMs.
LARP: Language-Agent Role Play for Open-World Games
Language agents have shown impressive problem-solving skills within defined settings and brief timelines.
De novo Drug Design using Reinforcement Learning with Multiple GPT Agents
A central challenge in this field is to generate molecules with specific properties while also producing a wide range of diverse candidates.
Beyond 1D and oversimplified kinematics: A generic analytical framework for surrogate safety measures
This paper presents a generic analytical framework tailored for surrogate safety measures (SSMs) that is versatile across various highway geometries, capable of encompassing vehicle dynamics of differing dimensionality and fidelity, and suitable for dynamic, real-world environments.
Combined Invariant Subspace \& Frequency-Domain Subspace Method for Identification of Discrete-Time MIMO Linear Systems
Recently, a novel system identification method based on invariant subspace theory is introduced, aiming to address the identification problem of continuous-time (CT) linear time-invariant (LTI) systems by combining time-domain and frequency-domain methods.
Interfacing Foundation Models' Embeddings
The proposed interface is adaptive to new tasks, and new models.
CSL: Class-Agnostic Structure-Constrained Learning for Segmentation Including the Unseen
Addressing Out-Of-Distribution (OOD) Segmentation and Zero-Shot Semantic Segmentation (ZS3) is challenging, necessitating segmenting unseen classes.
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
To address this issue, we have created GVC data that allows for the combination of grounding and chat capabilities.
Slice3D: Multi-Slice, Occlusion-Revealing, Single View 3D Reconstruction
Our key observation is that object slicing is more advantageous than altering views to reveal occluded structures.
Revisiting Single Image Reflection Removal In the Wild
This research focuses on the issue of single-image reflection removal (SIRR) in real-world conditions, examining it from two angles: the collection pipeline of real reflection pairs and the perception of real reflection locations.
Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges
We further present evaluation datasets used in existing methods, and finally conclude this survey by presenting the ongoing challenges and future directions.
Visual In-Context Prompting
In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain.
DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation
We present an unsupervised 3D shape co-segmentation method which learns a set of deformable part templates from a shape collection.
Generalization and Hallucination of Large Vision-Language Models through a Camouflaged Lens
Large Vision-Language Model (LVLM) has seen burgeoning development and increasing attention recently.
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
LLaVA-Plus is a general-purpose multimodal assistant that expands the capabilities of large multimodal models.
Ranked #1 on
LMM real-life tasks
on Leaderboard
Interpretable Geoscience Artificial Intelligence (XGeoS-AI): Application to Demystify Image Recognition
Experimental results demonstrate that the effectiveness, versatility, and heuristics of the proposed framework have great potential in solving geoscience image recognition problems.
Spatial Process Approximations: Assessing Their Necessity
In spatial statistics and machine learning, the kernel matrix plays a pivotal role in prediction, classification, and maximum likelihood estimation.
Inner-IoU: More Effective Intersection over Union Loss with Auxiliary Bounding Box
Based on the above, we first analyzed the BBR model and concluded that distinguishing different regression samples and using different scales of auxiliary bounding boxes to calculate losses can effectively accelerate the bounding box regression process.
Ranked #1 on
Object Detection
on AI-TOD
(mAP50 metric)
Few-shot Learning using Data Augmentation and Time-Frequency Transformation for Time Series Classification
Deep neural networks (DNNs) that tackle the time series classification (TSC) task have provided a promising framework in signal processing.
Towards Automatic Sampling of User Behaviors for Sequential Recommender Systems
Sequential recommender systems (SRS) have gained widespread popularity in recommendation due to their ability to effectively capture dynamic user preferences.
Sentence Bag Graph Formulation for Biomedical Distant Supervision Relation Extraction
We introduce a novel graph-based framework for alleviating key challenges in distantly-supervised relation extraction and demonstrate its effectiveness in the challenging and important domain of biomedical data.
TLM: Token-Level Masking for Transformers
Structured dropout approaches, such as attention dropout and DropHead, have been investigated to regularize the multi-head attention mechanism in Transformers.
Open-NeRF: Towards Open Vocabulary NeRF Decomposition
Current techniques for NeRF decomposition involve a trade-off between the flexibility of processing open-vocabulary queries and the accuracy of 3D segmentation.
Interaction-Driven Active 3D Reconstruction with Object Interiors
We introduce an active 3D reconstruction method which integrates visual perception, robot-object interaction, and 3D scanning to recover both the exterior and interior, i. e., unexposed, geometries of a target 3D object.
Long-Form Speech Translation through Segmentation with Finite-State Decoding Constraints on Large Language Models
One challenge in speech translation is that plenty of spoken content is long-form, but short units are necessary for obtaining high-quality translations.
Experimental Results of Underwater Sound Speed Profile Inversion by Few-shot Multi-task Learning
Underwater Sound Speed Profile (SSP) distribution has great influence on the propagation mode of acoustic signal, thus the fast and accurate estimation of SSP is of great importance in building underwater observation systems.
Neural Packing: from Visual Sensing to Reinforcement Learning
We present a novel learning framework to solve the transport-and-packing (TAP) problem in 3D.
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
We present Set-of-Mark (SoM), a new visual prompting method, to unleash the visual grounding abilities of large multimodal models (LMMs), such as GPT-4V.
Explaining How a Neural Network Play the Go Game and Let People Learn
The AI model has surpassed human players in the game of Go, and it is widely believed that the AI model has encoded new knowledge about the Go game beyond human players.
From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models
By simply equipping it with an MLP layer for alignment, DINO surpasses CLIP in fine-grained related perception tasks.
Underwater Sound Speed Profile Construction: A Review
The proposal of SSP inversion method greatly improves the convenience and real--time performance, but the accuracy is not as good as the direct measurement method.
Fast Ray-Tracing-Based Precise Underwater Acoustic Localization without Prior Acknowledgment of Target Depth
To tackle this issue, we propose an iterative ray tracing 3D underwater localization (IRTUL) method for stratification compensation.
Online Speculative Decoding
We develop a prototype of online speculative decoding based on online knowledge distillation and evaluate it using both synthetic and real query data on several popular LLMs.
Open-Vocabulary Animal Keypoint Detection with Semantic-feature Matching
Current image-based keypoint detection methods for animal (including human) bodies and faces are generally divided into full-supervised and few-shot class-agnostic approaches.
DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training
FlashAttention (Dao, 2023) effectively reduces the quadratic peak memory usage to linear in training transformer-based large language models (LLMs) on a single GPU.
Tuning Large language model for End-to-end Speech Translation
The training of LST consists of two stages: (1) Modality adjustment, where the adapter is tuned to align speech representation with text embedding space, and (2) Downstream task fine-tuning, where both the adapter and LLM model are trained to optimize performance on the E2EST task.
Advancing Acoustic Howling Suppression through Recursive Training of Neural Networks
In this paper, we introduce a novel training framework designed to comprehensively address the acoustic howling issue by examining its fundamental formation process.
Neural Network Augmented Kalman Filter for Robust Acoustic Howling Suppression
Acoustic howling suppression (AHS) is a critical challenge in audio communication systems.
Boundary-Aware Proposal Generation Method for Temporal Action Localization
More importantly, few works consider the background frames that are similar to action frames in pixels but dissimilar in semantics, which also leads to inaccurate temporal boundaries.
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Studying how people interact with large language models (LLMs) in real-world scenarios is increasingly important due to their widespread use in various applications.
LMDX: Language Model-based Document Information Extraction and Localization
Large Language Models (LLM) have revolutionized Natural Language Processing (NLP), improving state-of-the-art on many existing tasks and exhibiting emergent capabilities.
Reformulating Sequential Recommendation: Learning Dynamic User Interest with Content-enriched Language Modeling
Recommender systems are indispensable in the realm of online applications, and sequential recommendation has enjoyed considerable prevalence due to its capacity to encapsulate the dynamic shifts in user interests.
Source-free Active Domain Adaptation for Diabetic Retinopathy Grading Based on Ultra-wide-field Fundus Image
Domain adaptation (DA) has been widely applied in the diabetic retinopathy (DR) grading of unannotated ultra-wide-field (UWF) fundus images, which can transfer annotated knowledge from labeled color fundus images.
Unifying Robustness and Fidelity: A Comprehensive Study of Pretrained Generative Methods for Speech Enhancement in Adverse Conditions
Enhancing speech signal quality in adverse acoustic environments is a persistent challenge in speech processing.
Text-Guided Generation and Editing of Compositional 3D Avatars
We argue that existing methods are limited because they employ a monolithic modeling approach, using a single representation for the head, face, hair, and accessories.
When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System
Critically, the scalability and generalizability of GFMs empower them to address a wide array of prediction, simulation, and decision tasks related to the intricate interactions among Earth system components.
Efficient Memory Management for Large Language Model Serving with PagedAttention
On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage.
Diff-Retinex: Rethinking Low-light Image Enhancement with A Generative Diffusion Model
Therefore, Diff-Retinex formulates the low-light image enhancement problem into Retinex decomposition and conditional image generation.
Text Injection for Capitalization and Turn-Taking Prediction in Speech Models
Text injection for automatic speech recognition (ASR), wherein unpaired text-only data is used to supplement paired audio-text data, has shown promising improvements for word error rate.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
DFA3D: 3D Deformable Attention For 2D-to-3D Feature Lifting
Existing feature lifting approaches, such as Lift-Splat-based and 2D attention-based, either use estimated depth to get pseudo LiDAR features and then splat them to a 3D space, which is a one-pass operation without feature refinement, or ignore depth and lift features by 2D attention mechanisms, which achieve finer semantics while suffering from a depth ambiguity problem.
Semantic-SAM: Segment and Recognize Anything at Any Granularity
In this paper, we introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity.
Computron: Serving Distributed Deep Learning Models with Model Parallel Swapping
In anticipation of workloads that involve serving many of such large models to handle different tasks, we develop Computron, a system that uses memory swapping to serve multiple distributed models on a shared GPU cluster.
TSNet-SAC: Leveraging Transformers for Efficient Task Scheduling
In future 6G Mobile Edge Computing (MEC), autopilot systems require the capability of processing multimodal data with strong interdependencies.
Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks
Text-to-image (T2I) diffusion models (DMs) have shown promise in generating high-quality images from textual descriptions.
CLIPXPlore: Coupled CLIP and Shape Spaces for 3D Shape Exploration
This paper presents CLIPXPlore, a new framework that leverages a vision-language model to guide the exploration of the 3D shape space.
detrex: Benchmarking Detection Transformers
To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation.
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences.
Ranked #3 on
Long-Context Understanding
on Ada-LEval (TSort)
How Can Recommender Systems Benefit from Large Language Models: A Survey
In this paper, we conduct a comprehensive survey on this research direction from the perspective of the whole pipeline in real-world recommender systems.
ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Rendering
Furthermore, we showcase the ability of our method to learn geometric details and textures from shapes reconstructed from real-world photos.
DiViNeT: 3D Reconstruction from Disparate Views via Neural Template Regularization
We demonstrate that our approach is not only able to complete the surface geometry but also reconstructs surface details to a reasonable extent from a few disparate input views.
FaceDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models
The ability to create high-quality 3D faces from a single image has become increasingly important with wide applications in video conferencing, AR/VR, and advanced video editing in movie industries.
MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction
Given a query, the task of Natural Language Video Localization (NLVL) is to localize a temporal moment in an untrimmed video that semantically matches the query.
BRICS: Bi-level feature Representation of Image CollectionS
Our key codes and feature grids are jointly trained continuously with well-defined gradient flows, leading to high usage rates of the feature grids and improved generative modeling compared to discrete Vector Quantization (VQ).
Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR
We address this limitation by distilling punctuation knowledge from a bidirectional teacher language model (LM) trained on written, punctuated text.
Mobile Safety Application for Pedestrians
The work we discuss in this paper is related to a mobile application that utilizes the mobile phone sensors and Bluetooth communication to implement Personal Safety Message (PSM) broadcast using the SAE J2735 standard to create a Pedestrian to Vehicle (P2V) based safety warning structure.
Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model
The detection of machine-generated text, especially from large language models (LLMs), is crucial in preventing serious social problems resulting from their misuse.
NoisywikiHow: A Benchmark for Learning with Real-world Noisy Labels in Natural Language Processing
To mitigate the effects of label noise, learning with noisy labels (LNL) methods are designed to achieve better generalization performance.
Hybrid AHS: A Hybrid of Kalman Filter and Deep Learning for Acoustic Howling Suppression
During offline training, a pre-processed signal obtained from the Kalman filter and an ideal microphone signal generated via teacher-forced training strategy are used to train the deep neural network (DNN).
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction
In FormNetV2, we introduce a centralized multimodal graph contrastive learning strategy to unify self-supervised pre-training for all modalities in one loss.
Deep Learning for Joint Acoustic Echo and Acoustic Howling Suppression in Hybrid Meetings
In particular, the interplay between acoustic echo and acoustic howling in a hybrid meeting makes the joint suppression of them difficult.
A Strong and Reproducible Object Detector with Only Public Datasets
This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64. 6 AP on COCO val2017 and 64. 8 AP on COCO test-dev using only 700M parameters without any test time augmentation.
Ranked #5 on
Object Detection
on COCO minival
(using extra training data)
Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning
However, the final model often performs worse on the MT task than the MT model trained alone, which means that the knowledge transfer ability of this method is also limited.
Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translation
Existing techniques often attempt to make knowledge transfer from a powerful machine translation (MT) to speech translation (ST) model with some elaborate techniques, which often requires transcription as extra input during training.
DropDim: A Regularization Method for Transformer Networks
In contrast to the general dropout method, which randomly drops neurons, DropDim drops part of the embedding dimensions.
MS-LSTM: Exploring Spatiotemporal Multiscale Representations in Video Prediction Domain
The drastic variation of motion in spatial and temporal dimensions makes the video prediction task extremely challenging.
Segment Everything Everywhere All at Once
In SEEM, we propose a novel decoding mechanism that enables diverse prompting for all types of segmentation tasks, aiming at a universal segmentation interface that behaves like large language models (LLMs).
RoSI: Recovering 3D Shape Interiors from Few Articulation Images
The dominant majority of 3D models that appear in gaming, VR/AR, and those we use to train geometric deep learning algorithms are incomplete, since they are modeled as surface meshes and missing their interior structures.
Detection Transformer with Stable Matching
We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.
SpanRE: Entities and Overlapping Relations Extraction Based on Spans and Entity Attention
Then we present a labeled span mechanism to extract the objects and relations simultaneously, we use the labeled span mechanism to generate labeled spans whose start and end positions indicate the objects, and whose labels correspond to relations of subject and objects.
UKP-SQuARE v3: A Platform for Multi-Agent QA Research
To ease research in multi-agent models, we extend UKP-SQuARE, an online platform for QA research, to support three families of multi-agent systems: i) agent selection, ii) early-fusion of agents, and iii) late-fusion of agents.
Active Coarse-to-Fine Segmentation of Moveable Parts from Real Images
We introduce the first active learning (AL) framework for high-accuracy instance segmentation of moveable parts from RGB images of real indoor scenes.
A Region-Prompted Adapter Tuning for Visual Abductive Reasoning
Existing works highlight cues utilizing a specific prompt (e. g., colorful prompt).
Ranked #1 on
Visual Abductive Reasoning
on SHERLOCK
SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environments
Existing work has typically been tailored for particular environments or representations.
DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion
We introduce a novel method to automatically generate an artistic typography by stylizing one or more letter fonts to visually convey the semantics of an input word, while ensuring that the output remains readable.
A Simple Framework for Open-Vocabulary Segmentation and Detection
We present OpenSeeD, a simple Open-vocabulary Segmentation and Detection framework that jointly learns from different segmentation and detection datasets.
Ranked #2 on
Instance Segmentation
on ADE20K val
(using extra training data)
Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETR
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance.
MP-Former: Mask-Piloted Transformer for Image Segmentation
We present a mask-piloted Transformer which improves masked-attention in Mask2Former for image segmentation.
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.
Ranked #1 on
Zero-Shot Object Detection
on MSCOCO
ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishing
Zero-shot capability has been considered as a new revolution of deep learning, letting machines work on tasks without curated training data.
TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders
In this work, we propose TimeMAE, a novel self-supervised paradigm for learning transferrable time series representations based on transformer networks.
Concept-Level Explanation for the Generalization of a DNN
Therefore, in this paper, we investigate the generalization power of each interactive concept, and we use the generalization power of different interactive concepts to explain the generalization power of the entire DNN.
Introducing Depth into Transformer-based 3D Object Detection
To address the second issue, we introduce an auxiliary learning task called Depth-aware Negative Suppression loss.
AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving
Model parallelism is conventionally viewed as a method to scale a single large deep learning model beyond the memory limits of a single device.
Deep AHS: A Deep Learning Approach to Acoustic Howling Suppression
In this paper, we formulate acoustic howling suppression (AHS) as a supervised learning problem and propose a deep learning approach, called Deep AHS, to address it.
Futuristic Variations and Analysis in Fundus Images Corresponding to Biological Traits
Our proposed DL models, named FAG-Net and FGC-Net, correspondingly estimate biological traits (age and gender) and generates fundus images.
NeuralKalman: A Learnable Kalman Filter for Acoustic Echo Cancellation
The robustness of the Kalman filter to double talk and its rapid convergence make it a popular approach for addressing acoustic echo cancellation (AEC) challenges.
HAL3D: Hierarchical Active Learning for Fine-Grained 3D Part Labeling
We present the first active learning tool for fine-grained 3D part labeling, a problem which challenges even the most advanced deep learning (DL) methods due to the significant structural variations among the small and intricate parts.
A Method For Eliminating Contour Errors In Self-Encoder Reconstructed Images
In this paper, we propose a self-supervised twin network approach based on this a priori.
See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning
The see stage scans the image and grounds the visual concept candidates with a visual perception model.
CA$^2$T-Net: Category-Agnostic 3D Articulation Transfer from Single Image
We present a neural network approach to transfer the motion from a single image of an articulated object to a rest-state (i. e., unarticulated) 3D model.
Lite DETR: An Interleaved Multi-Scale Encoder for Efficient DETR
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance.
CC-FedAvg: Computationally Customized Federated Averaging
Federated learning (FL) is an emerging paradigm to train model with distributed data from numerous Internet of Things (IoT) devices.
Improved Long-Form Spoken Language Translation with Large Language Models
A challenge in spoken language translation is that plenty of spoken content is long-form, but short units are necessary for obtaining high-quality translations.
ARO-Net: Learning Implicit Fields from Anchored Radial Observations
We introduce anchored radial observations (ARO), a novel shape encoding for learning implicit field representation of 3D shapes that is category-agnostic and generalizable amid significant shape variations.
Coordinating Cross-modal Distillation for Molecular Property Prediction
The existing graph methods have demonstrated that 3D geometric information is significant for better performance in MPP.
DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding
As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text for object prediction and phrase mask prediction.
Ranked #7 on
Referring Expression Comprehension
on RefCOCO
FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fields
This paper presents the first significant work on directly predicting 3D face landmarks on neural radiance fields (NeRFs).
A Unified Mutual Supervision Framework for Referring Expression Segmentation and Generation
Our mutual supervision contains two directions.
QueryForm: A Simple Zero-shot Form Entity Query Framework
Zero-shot transfer learning for document understanding is a crucial yet under-investigated scenario to help reduce the high cost involved in annotating document entities.
On Optimizing the Communication of Model Parallelism
This pattern emerges when the two paradigms of model parallelism - intra-operator and inter-operator parallelism - are combined to support large models on large clusters.
FF2: A Feature Fusion Two-Stream Framework for Punctuation Restoration
To accomplish punctuation restoration, most existing methods focus on introducing extra information (e. g., part-of-speech) or addressing the class imbalance problem.
MPCFormer: fast, performant and private Transformer inference with MPC
Through extensive evaluations, we show that MPCFORMER significantly speeds up Transformer inference in MPC settings while achieving similar ML performance to the input model.
Neural Eigenfunctions Are Structured Representation Learners
Unlike prior spectral methods such as Laplacian Eigenmap that operate in a nonparametric manner, Neural Eigenmap leverages NeuralEF to parametrically model eigenfunctions using a neural network.
NIFT: Neural Interaction Field and Template for Object Manipulation
Given a few object manipulation demos, NIFT guides the generation of the interaction imitation for a new object instance by matching the Neural Interaction Template (NIT) extracted from the demos in the target Neural Interaction Field (NIF) defined for the new object.
Language Model Decomposition: Quantifying the Dependency and Correlation of Language Models
A goodness-of-fit metric for LMD similar to the coefficient of determination is defined and used to measure the linear dependency of a set of LMs.
Regularized Data Programming with Automated Bayesian Prior Selection
The cost of manual data labeling can be a significant obstacle in supervised learning.
AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness
Scaling up model sizes can lead to fundamentally new capabilities in many machine learning (ML) tasks.
Application of Deep Learning on Single-Cell RNA-sequencing Data Analysis: A Review
Single-cell RNA-sequencing (scRNA-seq) has become a routinely used technique to quantify the gene expression profile of thousands of single cells simultaneously.
Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planning
Automaton based approaches have enabled robots to perform various complex tasks.
Physical Interaction: Reconstructing Hand-object Interactions with Physics
Single view-based reconstruction of hand-object interaction is challenging due to the severe observation missing caused by occlusions.
Learning Reconstructability for Drone Aerial Path Planning
To reconstruct a new urban scene, we first build the 3D scene proxy, then rely on the predicted reconstruction quality and uncertainty measures by our network, based off of the proxy geometry, to guide the drone path planning.
DiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discrimination
To relieve this issue, curriculum learning is proposed to improve model performance and generalization by ordering training samples in a meaningful (e. g., easy to hard) sequence.
UKP-SQuARE v2: Explainability and Adversarial Attacks for Trustworthy QA
In this paper, we introduce SQuARE v2, the new version of SQuARE, to provide an explainability infrastructure for comparing models based on methods such as saliency maps and graph-based explanations.
PhyGNNet: Solving spatiotemporal PDEs with Physics-informed Graph Neural Network
Solving partial differential equations (PDEs) is an important research means in the fields of physics, biology, and chemistry.
Parameterization of Cross-Token Relations with Relative Positional Encoding for Vision MLP
They use token-mixing layers to capture cross-token interactions, as opposed to the multi-head self-attention mechanism used by Transformers.
Long-term Leap Attention, Short-term Periodic Shift for Video Classification
By replacing a vanilla 2D attention with the LAPS, we could adapt a static transformer into a video one, with zero extra parameters and neglectable computation overhead ($\sim$2. 6\%).
Data-and-Knowledge Dual-Driven Automatic Modulation Recognition for Wireless Communication Networks
In order to tackle these problems, a novel data-and-knowledge dual-driven automatic modulation classification scheme based on radio frequency machine learning is proposed by exploiting the attribute features of different modulations.
Wavelet Regularization Benefits Adversarial Training
Since adversarial vulnerability can be regarded as a high-frequency phenomenon, it is essential to regulate the adversarially-trained neural network models in the frequency domain.
MS-RNN: A Flexible Multi-Scale Framework for Spatiotemporal Predictive Learning
Spatiotemporal predictive learning, which predicts future frames through historical prior knowledge with the aid of deep learning, is widely used in many fields.
DETR++: Taming Your Multi-Scale Detection Transformer
Convolutional Neural Networks (CNN) have dominated the field of detection ever since the success of AlexNet in ImageNet classification [12].
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation
In this paper we present Mask DINO, a unified object detection and segmentation framework.
Ranked #1 on
Panoptic Segmentation
on COCO test-dev
Why Adversarial Training of ReLU Networks Is Difficult?
We also prove that adversarial training tends to strengthen the influence of unconfident input samples with large gradient norms in an exponential manner.
GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesis
In this paper, we combine the above two paradigms together and propose a novel Generalizable Logic Synthesis (GALOIS) framework to synthesize hierarchical and strict cause-effect logic programs.
Active Domain Adaptation with Multi-level Contrastive Units for Semantic Segmentation
A simple pixel selection strategy followed with the construction of multi-level contrastive units is introduced to optimize the model for both domain adaptation and active supervised learning.
Downstream Transformer Generation of Question-Answer Pairs with Preprocessing and Postprocessing Pipelines
We present a system called TP3 to perform a downstream task of transformers on generating question-answer pairs (QAPs) from a given article.
Block Modulating Video Compression: An Ultra Low Complexity Image Compression Encoder for Resource Limited Platforms
We consider the image and video compression on resource limited platforms.
New-Onset Diabetes Assessment Using Artificial Intelligence-Enhanced Electrocardiography
We found ECG-based assessment outperforms the ADA Risk test, achieving a higher area under the curve (0. 80 vs. 0. 68) and positive predictive value (13% vs. 9%) -- 2. 6 times the prevalence of diabetes in the cohort.
Adaptive Split-Fusion Transformer
Neural networks for visual content understanding have recently evolved from convolutional ones (CNNs) to transformers.
FedCos: A Scene-adaptive Federated Optimization Enhancement for Performance Improvement
On the other hand, it enlarges the distances between local models, resulting in an aggregated global model with poor performance.
Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection
Encouraged by this inspiring theoretical result on heterogeneous networks, we directly integrate conventional and quadratic neurons in an autoencoder to make a new type of heterogeneous autoencoders.
Randomized Sharpness-Aware Training for Boosting Computational Efficiency in Deep Learning
Optimizers in RST would perform a Bernoulli trial at each iteration to choose randomly from base algorithms (SGD) and sharpness-aware algorithms (SAM) with a probability arranged by a predefined scheduling function.
Group Contextualization for Video Recognition
By utilizing calibrators to embed feature with four different kinds of contexts in parallel, the learnt representation is expected to be more resilient to diverse types of activities.
Ranked #3 on
Egocentric Activity Recognition
on EGTEA
Boilerplate Detection via Semantic Classification of TextBlocks
We present a hierarchical neural network model called SemText to detect HTML boilerplate based on a novel semantic representation of HTML tags, class names, and text blocks.
Contextual Networks and Unsupervised Ranking of Sentences
We construct a contextual network to represent a document with syntactic and semantic relations between word-sentence pairs, based on which we devise an unsupervised algorithm called CNATAR (Contextual Network And Text Analysis Rank) to score sentences, and rank them through a bi-objective 0-1 knapsack maximization problem over topic analysis and sentence scores.
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results.
Ranked #1 on
Real-Time Object Detection
on COCO 2017 val
Vision-Language Intelligence: Tasks, Representation Learning, and Large Models
This survey is inspired by the remarkable progress in both computer vision and natural language processing, and recent trends shifting from single modality processing to multiple modality comprehension.
DN-DETR: Accelerate DETR Training by Introducing Query DeNoising
Our method is universal and can be easily plugged into any DETR-like methods by adding dozens of lines of code to achieve a remarkable improvement.
Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model
Capitalization normalization (truecasing) is the task of restoring the correct case (uppercase or lowercase) of noisy text.
Hierarchical Point Cloud Encoding and Decoding with Lightweight Self-Attention based Model
In this paper we present SA-CNN, a hierarchical and lightweight self-attention based encoding and decoding architecture for representation learning of point cloud data.
Multi-relation Message Passing for Multi-label Text Classification
These examples motivate the modelling of multiple types of bi-directional relationships between labels.
Multi-Label Classification
Multi-Label Image Classification
+4
Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning
In this paper, we propose an effective method to improve the model generalization by additionally penalizing the gradient norm of loss function during optimization.
A Variational Edge Partition Model for Supervised Graph Representation Learning
This paper introduces a graph generative process to model how the observed edges are generated by aggregating the node interactions over a set of overlapping node communities, each of which contributes to the edges via a logical OR mechanism.
Neural Dual Contouring
We introduce neural dual contouring (NDC), a new data-driven approach to mesh reconstruction based on dual contouring (DC).
RIM-Net: Recursive Implicit Fields for Unsupervised Learning of Hierarchical Shape Structures
Each level of the tree corresponds to an assembly of shape parts, represented as implicit functions, to reconstruct the input shape.
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning
Existing model-parallel training systems either require users to manually create a parallelization plan or automatically generate one from a limited space of model parallelism configurations.
DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR
We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR.
Ranked #11 on
2D Object Detection
on SARDet-100K
Temporal Sentence Grounding in Videos: A Survey and Future Directions
Temporal sentence grounding in videos (TSGV), \aka natural language video localization (NLVL) or video moment retrieval (VMR), aims to retrieve a temporal moment that semantically corresponds to a language query from an untrimmed video.
A Privacy-Preserving Unsupervised Domain Adaptation Framework for Clinical Text Analysis
To evaluate our proposed method's utility and privacy loss, we apply our model on a medical report disease label classification task using two noisy challenging clinical text datasets.
