On the Asymptotic Validity of Fully Sequential Selection Procedures for Steady-State Simulation

Statistical procedures have been proposed to select the simulated system with the largest or smallest long-run average performance from among a finite number of alternative systems. In this paper, we focus on the indifference-zone formulation of the selection problem, meaning that we desire procedures that are guaranteed to select the best system with high probability when the long-run average performance of the best is at least a given amount better than the rest (see Chen et al. 2000 and Chick and Inoue 2001 for alternative formulations). Many of these procedures have been constructed under the assumption that the output data generated by each system are independent and identically distributed (i.i.d.), and also marginally normal. The assumption of independence within each system’s output, which is appropriate for terminating simulations, is the biggest barrier to applying these procedures directly to steady-state simulation experiments. The outputs within a single replication of a steady-state simulation are typically dependent. For instance, the delays in queue of successive parts processed at a work center may be dependent because each part must wait for the ones ahead of it. We can apply procedures for i.i.d. normal data to steadystate simulation experiments if we make multiple replications of each alternative, use the within-replication averages as the basic observations, and make the replications long enough that the within-replication averages are approximately normally distributed. Or, we can generate a single long replication of each alternative and use batch means of many individual outputs as the basic observations. In typical simulation output processes, the batch means are much less dependent and more nearly normally distributed than the individual outputs if the batch size is large enough. See Law and Kelton (2000) for a general discussion of replication versus batching, Goldsman and Nelson (1998) for a presentation specialized to selection procedures, and Glynn and Iglehart (1990) for conditions under which the batch means method is asymptotically valid. Unfortunately, both of these remedies for dependent data have disadvantages. If we make replications, then we have to discard the so-called warm-up period from each one; this will be very inefficient if a large number of observations need to be deleted. Batching within a replication may also be inefficient for the following reason: Selection procedures attempt to minimize the simulation data required to obtain a “correct selection” by working sequentially—meaning two or more stages of sampling—with decisions on how much, if any, additional sampling is needed made at the end of each stage. If a “stage” is defined by batch means, rather than individual observations, then the simulation effort consumed by a stage is a multiple of the batch size. When a large batch size is required to achieve approximate independence—and batch sizes of several thousand are common—then the selection procedure is forced to make decisions at long intervals, wasting observations and time. These disadvantages have fostered efforts to develop new procedures designed specifically for steady-state simulation, procedures that can be applied to a single replication from each alternative and that use basic outputs rather than batch means. Goldsman and Marshall (1999) extended Rinott’s (1978) procedure for use in steady-state simulation. Nakayama (1997) presented