Training of autoencoders using the back-propagation algorithm is challenging for non-differential channel models or in an experimental environment where gradients cannot be computed. In this paper, we study a gradient–free training method based on cubature Kalman filter. To numerically validate method, autoencoder employed to perform geometric constellation shaping differentiable communication ...

We introduce physics informed neural networks – neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. In this two part treatise, we present our developments in the context of solving two main classes of problems: data-driven solution and data-driven discovery of partial differe...

We discuss a general technique that can be used to form a differentiable bound on the optima of non-differentiable or discrete objective functions. We form a unified description of these methods and consider under which circumstances the bound is concave. In particular we consider two concrete applications of the method, namely sparse learning and support vector classification. 1 Optimization b...

We consider the minimization of a function G defined on R , which is the sum of a (non necessarily convex) differentiable function and a (non necessarily differentiable) convex function. Moreover, we assume that G satisfies the KurdykaLojasiewicz property. Such a problem can be solved with the Forward-Backward algorithm. However, the latter algorithm may suffer from slow convergence. We propose...

The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems, with differentiable or non-differentiable objective functions. Unfortunately, its performance is highly sensitive to a penalty parameter, which makes ADMM often unreliable and hard to automate for a non-expert user. We tackle this weakness of ADMM by proposin...

Recently, Huang and Ng presented second-order sufficient conditions for error bounds of continuous and Gâteaux differentiable functions in Banach spaces. Wu and Ye dropped the assumption of Huang and Ng on Gâteaux differentiability but required the space to be a Hilbert space. We carry on this research in two directions. First we extend Wu and Ye’s result to some non-Hilbert spaces; second, sam...

We study the properties of locally uniformly differentiable functions on N, a non-Archimedean field extension of the real numbers that is real closed and Cauchy complete in the topology induced by the order. In particular, we show that locally uniformly differentiable functions are C1, they include all polynomial functions, and they are closed under addition, multiplication, and composition. Th...