1 code implementation • 27 Apr 2024 • Shujian Yu, Xi Yu, Sigurd Løkse, Robert Jenssen, Jose C. Principe
The information bottleneck (IB) approach is popular to improve the generalization, robustness and explainability of deep neural networks.
no code implementations • 5 Feb 2024 • Benjamin Colburn, Luis G. Sanchez Giraldo, Kan Li, Jose C. Principe
We provide an extended functional Wiener equation, and present a solution to this equation in an explicit, finite dimensional, data-dependent RKHS.
no code implementations • 20 Jan 2024 • Isaac J. Sledge, Dominic M. Byrne, Jonathan L. King, Steven H. Ostertag, Denton L. Woods, James L. Prater, Jermaine L. Kennedy, Timothy M. Marston, Jose C. Principe
We propose a weakly-supervised framework for the semantic segmentation of circular-scan synthetic-aperture-sonar (CSAS) imagery.
no code implementations • 19 Dec 2023 • Benjamin Colburn, Jose C. Principe, Luis G. Sanchez Giraldo
Kernel Adaptive Filtering (KAF) are mathematically principled methods which search for a function in a Reproducing Kernel Hilbert Space.
no code implementations • 31 Dec 2022 • Benjamin Colburn, Luis G. Sanchez Giraldo, Jose C. Principe
Because of the lack of congruence between the Gaussian RKHS and the space of time series, we compare performance of two pre-imaging algorithms: a fixed-point optimization (FWFFP) that finds and approximate solution in the RKHS, and a local model implementation named FWFLM.
no code implementations • 20 Dec 2022 • Isaac J. Sledge, Jose C. Principe
We then illustrate that these trends hold for deep, value-of-information-based agents that learn to play ten simple games and over forty more complicated games for the Nintendo GameBoy system.
no code implementations • 9 Dec 2022 • Bo Hu, Jose C. Principe
We mathematically prove that FMCA learns the dominant eigenvalues and eigenfunctions of NCD directly from realizations.
no code implementations • 3 Nov 2022 • Rishabh Singh, Jose C. Principe
We present a simple framework for high-resolution predictive uncertainty quantification of semantic segmentation models that leverages a multi-moment functional definition of uncertainty associated with the model's feature space in the reproducing kernel Hilbert space (RKHS).
no code implementations • 3 Nov 2022 • Rishabh Singh, Jose C. Principe
Being based on the Gaussian RKHS, our approach is robust towards outliers and monotone transformations of data, while the multiple moments of uncertainty provide high resolution and interpretability of the type of dependence being quantified.
1 code implementation • 31 May 2022 • Shujian Yu, Francesco Alesiani, Wenzhe Yin, Robert Jenssen, Jose C. Principe
Graph sparsification aims to reduce the number of edges of a graph while maintaining its structural properties.
1 code implementation • 7 Feb 2022 • Hongming Li, Shujian Yu, Jose C. Principe
We develop a new neural network based independent component analysis (ICA) method by directly minimizing the dependence amongst all extracted components.
1 code implementation • 12 Oct 2021 • Bo Hu, Shujian Yu, Jose C. Principe
We test the framework for estimation of mutual information and compare the results with the mutual information neural estimation (MINE), for density estimation, for conditional probability estimation in Markov models as well as for training adversarial networks.
1 code implementation • 24 Sep 2021 • Isaac J. Sledge, Jose C. Principe
This yields matrix-based estimators of R\'enyi's $\alpha$-cross-entropies.
no code implementations • 22 Sep 2021 • Rishabh Singh, Jose C. Principe
The RKHS projection of model weights yields a potential field based interpretation of model weight PDF which consequently allows the definition of a functional operator, inspired by perturbation theory in physics, that performs a moment decomposition of the model weight PDF (the potential field) at a specific model output to quantify its uncertainty.
no code implementations • 2 Sep 2021 • Raheleh Baharloo, Jose C. Principe, Parisa Rashidi, Patrick J. Tighe
The dynamics of patients' physiological responses to these surgical events are linked to long-term post-operative pain development.
no code implementations • 22 Jul 2021 • Isaac J. Sledge, Christopher D. Toole, Joseph A. Maestri, Jose C. Principe
We propose a memory-based framework for real-time, data-efficient target analysis in forward-looking-sonar (FLS) imagery.
no code implementations • 5 Jul 2021 • Isaac J. Sledge, Jose C. Principe
A fundamental problem when aggregating Markov chains is the specification of the number of state groups.
no code implementations • 19 Apr 2021 • Shiyu Duan, Spencer Chang, Jose C. Principe
We call this statistic "sufficiently-labeled data" and prove its sufficiency and efficiency for finding the optimal hidden representations, on which competent classifier heads can be trained using as few as a single randomly-chosen fully-labeled example per class.
no code implementations • 2 Mar 2021 • Rishabh Singh, Jose C. Principe
We therefore propose a framework for predictive uncertainty quantification of a trained neural network that explicitly estimates the PDF of its raw prediction space (before activation), p(y'|x, w), which we refer to as the model PDF, in a Gaussian reproducing kernel Hilbert space (RKHS).
no code implementations • 24 Feb 2021 • Isaac J. Sledge, Darshan W. Bryner, Jose C. Principe
We have previously developed a means of constraining, and hence speeding up, the search process through the use of motion primitives; motion primitives are sequences of pre-specified actions taken across a state series.
1 code implementation • 31 Jan 2021 • Xi Yu, Shujian Yu, Jose C. Principe
We introduce the matrix-based Renyi's $\alpha$-order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle with a neural network.
1 code implementation • 25 Jan 2021 • Shujian Yu, Francesco Alesiani, Xi Yu, Robert Jenssen, Jose C. Principe
Measuring the dependence of data plays a central role in statistics and machine learning.
no code implementations • 18 Jan 2021 • Isaac J. Sledge, Jose C. Principe
It yields unsupervised object recognition that surpass convolutional autoencoders and are on par with convolutional networks trained in a supervised manner.
no code implementations • 10 Jan 2021 • Isaac J. Sledge, Matthew S. Emigh, Jonathan L. King, Denton L. Woods, J. Tory Cobb, Jose C. Principe
We show that our framework outperforms supervised, deep-saliency networks designed for natural imagery.
no code implementations • 9 Jan 2021 • Shiyu Duan, Jose C. Principe
This tutorial paper surveys provably optimal alternatives to end-to-end backpropagation (E2EBP) -- the de facto standard for training deep architectures.
no code implementations • 16 Nov 2020 • Shailaja Akella, Ali Mohebi, Kiersten Riels, Andreas Keil, Karim Oweiss, Jose C. Principe
A detailed analysis of the power - specific marked features of neuromodulations confirm high correlation between power spectral density and power in neuromodulations establishing the aptness of MPP spectrogram as a finer measure of power where it is able to track local variations in power while preserving the global structure of signal power distribution.
no code implementations • 18 Oct 2020 • Ryan Burt, Nina N. Thigpen, Andreas Keil, Jose C. Principe
The results in foveated vision show that Gamma saliency is comparable to the best and computationally faster.
1 code implementation • 21 Jul 2020 • Feiya Lv, Shujian Yu, Chenglin Wen, Jose C. Principe
This paper presents a novel mutual information (MI) matrix based method for fault detection.
no code implementations • 13 Jul 2020 • Yanjun Li, Shujian Yu, Jose C. Principe, Xiaolin Li, Dapeng Wu
Although substantial efforts have been made to learn disentangled representations under the variational autoencoder (VAE) framework, the fundamental properties to the dynamics of learning of most VAE models still remain unknown and under-investigated.
1 code implementation • 5 May 2020 • Shujian Yu, Ammar Shaker, Francesco Alesiani, Jose C. Principe
We propose a simple yet powerful test statistic to quantify the discrepancy between two conditional distributions.
no code implementations • 30 Jan 2020 • Rishabh Singh, Jose C. Principe
This paper introduces a new framework for quantifying predictive uncertainty for both data and models that relies on projecting the data into a Gaussian reproducing kernel Hilbert space (RKHS) and transforming the data probability density function (PDF) in a way that quantifies the flow of its gradient as a topological potential field quantified at all points in the sample space.
no code implementations • 1 Jan 2020 • Kan Li, Jose C. Principe
The inner product defined by the feature mapping corresponds to a positive-definite finite-rank kernel that induces a finite-dimensional reproducing kernel Hilbert space (RKHS).
no code implementations • 10 Dec 2019 • Kan Li, Jose C. Principe
Without loss of generality, we apply this approach to classical adaptive filtering algorithms and validate the methodology to show that deterministic features are faster to generate and outperform state-of-the-art kernel methods based on random Fourier features.
no code implementations • 24 Nov 2019 • Kan Li, Jose C. Principe
We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS).
no code implementations • 8 Nov 2019 • Chi Ding, Zheng Cao, Matthew S. Emigh, Jose C. Principe, Bing Ouyang, Anni Vuorenkoski, Fraser Dalgleish, Brian Ramos, Yanjun Li
To fully understand interactions between marine hydrokinetic (MHK) equipment and marine animals, a fast and effective monitoring system is required to capture relevant information whenever underwater animals appear.
1 code implementation • 13 Jul 2019 • Yantao Wei, Shujian Yu, Luis Sanchez Giraldo, Jose C. Principe
This paper proposes a novel architecture, termed multiscale principle of relevant information (MPRI), to learn discriminative spectral-spatial features for hyperspectral image (HSI) classification.
no code implementations • 13 Apr 2019 • Badong Chen, Xin Wang, Yingsong Li, Jose C. Principe
The kernel function in correntropy is usually restricted to the Gaussian function with center located at zero.
no code implementations • 22 Jan 2019 • Isaac J. Sledge, Jose C. Principe
An important property of our radial basis function networks is that they are guaranteed to yield the same responses as conventional radial-basis networks trained on a corresponding vector realization of the relationships encoded by the adjacency-matrix.
no code implementations • 31 Dec 2018 • Gabriel Nallathambi, Jose C. Principe
The integrate and fire converter transforms an analog signal into train of biphasic pulses.
no code implementations • 29 Nov 2018 • Shujian Yu, Jose C. Principe
Feature selection aims to select the smallest feature subset that yields the minimum generalization error.
1 code implementation • 23 Aug 2018 • Shujian Yu, Luis Gonzalo Sanchez Giraldo, Robert Jenssen, Jose C. Principe
The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).
no code implementations • 25 Jun 2018 • Shujian Yu, Xiaoyang Wang, Jose C. Principe
In this paper, a novel Hierarchical Hypothesis Testing framework with Request-and-Reverify strategy is developed to detect concept drifts by requesting labels only when necessary.
no code implementations • 18 Apr 2018 • Shujian Yu, Kristoffer Wickstrøm, Robert Jenssen, Jose C. Principe
The matrix-based Renyi's \alpha-entropy functional and its multivariate extension were recently developed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).
no code implementations • 30 Mar 2018 • Shujian Yu, Jose C. Principe
Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks.
no code implementations • 5 Feb 2018 • Isaac J. Sledge, Matthew S. Emigh, Jose C. Principe
The value of information yields a stochastic routine for choosing actions during learning that can explore the policy space in a coarse to fine manner.
no code implementations • 1 Feb 2018 • Zhengda Qin, Badong Chen, Nanning Zheng, Jose C. Principe
In this paper, we propose a linear model called Augmented Space Linear Model (ASLM), which uses the full joint space of input and desired signal as the projection space and approaches the performance of nonlinear models.
no code implementations • 28 Oct 2017 • Isaac J. Sledge, Jose C. Principe
In this paper, we provide an approach to clustering relational matrices whose entries correspond to either similarities or dissimilarities between objects.
no code implementations • 8 Oct 2017 • Isaac J. Sledge, Jose C. Principe
High amounts of policy information are associated with exploration-dominant searches of the space and yield high rewards.
2 code implementations • 11 Sep 2017 • Pablo Huijse, Pablo A. Estevez, Francisco Forster, Scott F. Daniel, Andrew J. Connolly, Pavlos Protopapas, Rodrigo Carrasco, Jose C. Principe
Robust and efficient methods that can aggregate data from multidimensional sparsely-sampled time series are needed.
Instrumentation and Methods for Astrophysics Information Theory Information Theory
no code implementations • 4 Aug 2017 • Jianji Wang, Nanning Zheng, Badong Chen, Jose C. Principe
Moreover, for a target vector, the ratio of the corresponding affine parameters in the MSE-based linear decomposition scheme and the SSIM-based scheme is a constant, which is just the value of PCC between the target vector and its estimated vector.
no code implementations • CVPR 2017 • Luis G. Sanchez Giraldo, Erion Hasanbelliu, Murali Rao, Jose C. Principe
In this paper, we describe a set of robust algorithms for group-wise registration using both rigid and non-rigid transformations of multiple unlabelled point-sets with no bias toward a given set.
no code implementations • 23 Mar 2017 • Badong Chen, Lei Xing, Haiquan Zhao, Bin Xu, Jose C. Principe
The maximum correntropy criterion (MCC) has recently been successfully applied in robust regression, classification and adaptive filtering, where the correntropy is maximized instead of minimizing the well-known mean square error (MSE) to improve the robustness with respect to outliers (or impulsive noises).
no code implementations • 28 Feb 2017 • Isaac J. Sledge, Jose C. Principe
This cost function is the value of information, which provides the optimal trade-off between the expected return of a policy and the policy's complexity; policy complexity is measured by number of bits and controlled by a single hyperparameter on the cost function.
no code implementations • 31 Oct 2016 • Eder Santana, Matthew Emigh, Pablo Zegers, Jose C. Principe
We propose a convolutional recurrent neural network, with Winner-Take-All dropout for high dimensional unsupervised feature learning in multi-dimensional time series.
no code implementations • 1 Aug 2016 • Badong Chen, Lei Xing, Bin Xu, Haiquan Zhao, Nanning Zheng, Jose C. Principe
Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher-order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non-Gaussian signal processing and machine learning.
no code implementations • 22 Mar 2016 • Eder Santana, Matthew Emigh, Jose C. Principe
We propose Information Theoretic-Learning (ITL) divergence measures for variational regularization of neural networks.
no code implementations • 25 Sep 2015 • Pablo Huijse, Pablo A. Estevez, Pavlos Protopapas, Jose C. Principe, Pablo Zegers
In this article we present an overview of machine learning and computational intelligence applications to TDA.
no code implementations • 23 Jan 2014 • Badong Chen, Junli Liang, Nanning Zheng, Jose C. Principe
Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS).
no code implementations • 28 Dec 2013 • Luis G. Sanchez Giraldo, Jose C. Principe
Here, we propose a learning algorithm for auto-encoders based on a rate-distortion objective that minimizes the mutual information between the inputs and the outputs of the auto-encoder subject to a fidelity constraint.
no code implementations • 16 Jan 2013 • Luis G. Sanchez Giraldo, Jose C. Principe
In this paper, we develop a framework for information theoretic learning based on infinitely divisible matrices.
no code implementations • 11 Nov 2012 • Luis G. Sanchez Giraldo, Murali Rao, Jose C. Principe
In this way, capitalizing on both the axiomatic definition of entropy and on the representation power of positive definite kernels, the proposed measure of entropy avoids the estimation of the probability distribution underlying the data.