Screening in Operations Research and Statistics • Partially Observable Markov Decision Process POMDP (Ayer et.al.) • Bayesian Optimal Design (Rizopoulos et.al.) • Pros: Principled search for optimal policies • Cons: Markov/Semi-Markov, Stationary Screening in Medical Literature • Stochastic simulation based (Frazie et.al. ) • Pros: No assumptions on disease models • Cons: Not personalized, Comp...

Optimization models are necessary for efficient and cost-effective maintenance of a road network. In this regard, road deterioration is commonly modeled as a discrete-time Markov process such that an optimal maintenance policy can be obtained based on the Markov decision process, or as a renewal process such that an optimal maintenance policy can be obtained based on the renewal theory. However...

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...

This paper studies the off-policy evaluation problem, where one aims to estimate the value of a target policy based on a sample of observations collected by another policy. We first consider the multi-armed bandit case, establish a minimax risk lower bound, and analyze the risk of two standard estimators. It is shown, and verified in simulation, that one is minimax optimal up to a constant, whi...