Search Results for author:
Found 9 papers, 6 papers with code
Simple Drop-in LoRA Conditioning on Attention Layers Will Improve Your Diffusion Model
Current state-of-the-art diffusion models employ U-Net architectures containing convolutional and (qkv) self-attention layers.
Improved Communication-Privacy Trade-offs in $L_2$ Mean Estimation under Streaming Differential Privacy
We study $L_2$ mean estimation under central differential privacy and communication constraints, and address two key challenges: firstly, existing mean estimation schemes that simultaneously handle both constraints are usually optimized for $L_\infty$ geometry and rely on random rotation or Kashin's representation to adapt to $L_2$ geometry, resulting in suboptimal leading constants in mean square errors (MSEs); secondly, schemes achieving order-optimal communication-privacy trade-offs do not extend seamlessly to streaming differential privacy (DP) settings (e. g., tree aggregation or matrix factorization), rendering them incompatible with DP-FTRL type optimizers.
Fully Quantized Always-on Face Detector Considering Mobile Image Sensors
In this study, we aim to bridge the gap by exploring extremely low-bit lightweight face detectors, focusing on the always-on face detection scenario for mobile image sensor applications.
Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing.
PyNET-QxQ: An Efficient PyNET Variant for QxQ Bayer Pattern Demosaicing in CMOS Image Sensors
Additionally, modern mobile cameras employ non-Bayer color filter arrays (CFA) such as Quad Bayer, Nona Bayer, and QxQ Bayer to enhance image quality, yet most existing deep learning-based ISP (or demosaicing) models focus primarily on standard Bayer CFAs.
Neural Tangent Kernel Analysis of Deep Narrow Neural Networks
The tremendous recent progress in analyzing the training dynamics of overparameterized neural networks has primarily focused on wide networks and therefore does not sufficiently address the role of depth in deep learning.
Prune Your Model Before Distill It
Recent results suggest that the student-friendly teacher is more appropriate to distill since it provides more transferable knowledge.
An Information-Theoretic Justification for Model Pruning
We study the neural network (NN) compression problem, viewing the tension between the compression ratio and NN performance through the lens of rate-distortion theory.
WGAN with an Infinitely Wide Generator Has No Spurious Stationary Points
Generative adversarial networks (GAN) are a widely used class of deep generative models, but their minimax training dynamics are not understood very well.
