Appropriately designing sampling policies is highly important for obtaining better control policies in reinforcement learning. In this paper, we first show that the least-squares policy iteration (LSPI) framework allows us to employ statistical active learning methods for linear regression. Then we propose a design method of good sampling policies for efficient exploration, which is particularl...

We develop a novel, fundamental and surprisingly simple randomized iterative method for solving consistent linear systems. Our method has five different but equivalent interpretations: sketch-and-project, constrain-and-approximate, random intersect, random linear solve and random fixed point. By varying its two parameters—a positive definite matrix (defining geometry), and a random matrix (samp...

This paper describes a two pass algorithm capable of computing solutions to the global illumination in general environments (diffuse or glossy surfaces, anisotropically scattering participating media) faster than previous methods, by combining the strengths of finite element and Monte Carlo methods. A quick coarse solution is first computed with a clustered directional hierarchical method. This...

This paper, deals with detecting and identifying unknown scatters (e.g., obstacles) in an elastic background solid through the use of elastic illuminating waves. In this regards, the Linear Sampling Method (LSM) for the reconstruction of the underground obstacles from near-field surface seismic measurements in the time domain is explained. The LSM is an effective approach to image the geometric...

We present two methods for barrierless equilibrium sampling of molecular systems based on the recently proposed Kirkwood method (J. Chem. Phys. 2009, 130, 134102). Kirkwood sampling employs low-order correlations among internal coordinates of a molecule for random (or non-Markovian) sampling of the high dimensional conformational space. This is a geometrical sampling method independent of the p...

A random simulation method was used for treatment of systems of Volterra integral equations of the second kind. Firstly, a linear algebra system was obtained by discretization using quadrature formula. Secondly, this algebra system was solved by using relaxed Monte Carlo method with importance sampling and numerical approximation solutions of the integral equations system were achieved. It is t...

We consider two-component block Gibbs sampling for a Bayesian hierarchical version of the normal theory general linear model. This model is practically relevant in the sense that it is general enough to have many applications and in that it is not straightforward to sample directly from the corresponding posterior distribution. There are two possible orders in which to update the components of ...

Granger causality (GC) is a powerful method for causal inference for time series. In general, the GC value is computed using discrete time series sampled from continuous-time processes with a certain sampling interval length τ, i.e., the GC value is a function of τ. Using the GC analysis for the topology extraction of the simplest integrate-and-fire neuronal network of two neurons, we discuss b...

Multistage stochastic programming deals with operational and planning problems that involve a sequence of decisions over time while responding to an uncertain future. Algorithms designed address multistage linear (MSLP) often rely upon scenario trees represent the underlying process. When this process exhibits stagewise independence, sampling-based techniques, particularly dual dynamic algorith...