Invariant risk minimization (IRM) has recently emerged as a promising alternative for domain generalization. Nevertheless, the loss function is difficult to optimize nonlinear classifiers and original optimization objective could fail when pseudo-invariant features geometric skews exist. Inspired by IRM, in this paper we propose novel formulation generalization, dubbed invariant information bot...

Buffers decouple fluctuations in the material flow. It is common wisdom in industry that a full buffer indicates a downstream bottleneck and an empty buffer indicates an upstream bottleneck. Numerous different bottleneck detection methods use this approach to detect the bottlenecks. However, so far this common wisdom on the shop floor has not yet been verified academically. The authors tested t...

The information bottleneck (IB) method is an unsupervised model independent data organization technique. Given a joint distribution, p(X, Y), this method constructs a new variable, T, that extracts partitions, or clusters, over the values of X that are informative about Y. Algorithms that are motivated by the IB method have already been applied to text classification, gene expression, neural co...

Given a pair of random variables (X,Y ) ∼ PXY and two convex functions f1 and f2, we introduce two bottleneck functionals as the lower and upper boundaries of the two-dimensional convex set that consists of the pairs (If1(W ;X), If2(W ;Y )), where If denotes f -information and W varies over the set of all discrete random variables satisfying the Markov condition W → X → Y . Applying Witsenhause...

We propose a neural network framework for high-dimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input x and target y, where both are highdimensional. The key to effectively train BCDEs is the hybrid blending of ...