Universally Attainable Error-Exponents for Rate-Constrained Denoising of Noisy Sources

Consider the problem of rate-constrained reconstruction of a finite-alphabet discrete memoryless signal X = (X1, . . . , Xn), based on a noise-corrupted observation sequence Z , which is the finite-alphabet output of a Discrete Memoryless Channel (DMC) whose input is X. Suppose that there is some uncertainty in the source distribution, in the channel characteristics, or in both. Equivalently, suppose that the distribution of the pairs (Xi, Zi), rather than completely being known, is only known to belong to a set Θ. Suppose further that the relevant performance criterion is the probability of excess distortion, i.e., letting X̂(Z) denote the reconstruction, we are interested in the behavior of Pθ ( ρ(X, X̂(Z)) > dθ ) , where ρ is a (normalized) block distortion induced by a single-letter distortion measure and Pθ denotes the probability measure corresponding to the case where (Xi, Zi) ∼ θ, θ ∈ Θ. Since typically this probability will either not decay at all or do so at an exponential rate, it is the rate of this decay which we focus on. More concretely, for a given rate R ≥ 0 and a family of distortion levels {dθ}θ∈Θ, we are interested in families of exponential levels {Iθ}θ∈Θ which are achievable in the sense that for large n there exist rate-R schemes satisfying − 1 n logPθ ( ρ(X, X̂(Z)) > dθ ) ≥ Iθ for all θ ∈ Θ. Our main result is a complete “single-letter” characterization of achievable levels {Iθ}θ∈Θ per any given triple (Θ, R, {dθ}θ∈Θ). Equipped with this result, we later turn to addressing the question of the “right” choice of {Iθ}θ∈Θ. Relying on methodology recently put forth by Feder and Merhav in the context of the composite hypothesis testing problem, we propose a competitive minimax approach for the choice of these levels and apply our main result for characterizing the associated key quantities. Subsequently, we apply the main result to characterize optimal performance in a Neyman-Pearson-like setting, where there are two possible noise-corrupted signals. In this problem, the goal of the observer of the noisy signal, rather than having to determine which of the two it is (as in the hypothesis testing problem), is to reproduce the underlying clean signal with as high a fidelity as possible (e.g., lowest number of symbol errors when distortion measure is Hamming), under the assumption that one source is active, while operating at a limited information rate R and subject to a constraint on the fidelity of reconstruction when the other source is active. Finally, we apply our result to characterize a sufficient condition for the source class, Θ, to be universally encodable in the sense of the existence of schemes attaining the optimal distribution-dependent exponent, simultaneously for all sources in the class. This condition was shown in an earlier work to suffice for universality in expectation.