This chapter collects several probabilistic tools that proved to be useful in the analysis of randomized search heuristics. This includes classic material like Markov, Chebyshev and Chernoff inequalities, but also lesser known topics like stochastic domination and coupling or Chernoff bounds for geometrically distributed random variables and for negatively correlated random variables. Almost al...

We introduce the Bounded Axiom A Forcing Axiom(BAAFA). It turns out that it is equiconsistent with the existence of a regular Σ2-correct cardinal and hence also equiconsistent with BPFA. Furthermore we show that, if consistent, it does not imply the Bounded Proper Forcing Axiom(BPFA).

We have seen the Chernoff+Union bound in action in the previous section when we analyzed the outcome of reservoir sampling for items in [1, 100] over m iterations. There the bad event Badi represents the event that item i is not sampled in the range m 100 ± m 200 . Using the Chernoff bound, for each i Pr[Badi] is minuscule. Therefore, the probability that at least one of the bad event happens w...

When dealing with modern big data sets, a very common theme is reducing the set through a random process. These generally work by making “many simple estimates” of the full data set, and then judging them as a whole. Perhaps magically, these “many simple estimates” can provide a very accurate and small representation of the large data set. The key tool in showing how many of these simple estima...

This week, we consider a very simple load-balancing problem. Suppose you have n machines and m jobs. You want to assign the jobs to machines such that all machines have approximately the same load. Of course, there is a solution with load at most dmn e on every machine, but that requires central coordination. Without central coordination, the easiest thing you can do is let each job drawn one m...

i=1 `(f(xi), yi)), f̄ := ”argminf∈F”R(f), f̄n := ”argminf∈F”R̂(f), f̂ output of optimization algorithm, ḡ := ”argming measurable”R(g). (As usual, argmin has technical issues we are avoiding, hence the quotes.) The goal in a machine learning problem is to make an algorithm that outputs f̂ so that R(f̂)−R(ḡ) is small. We can decompose this error into the following pieces: R(f̂)−R(ḡ) = R(f̂)− R̂(f̂) (4) + R̂...