We present a novel method for frequentist statistical inference in M -estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling. An intuitive analysis using the OrnsteinUhlenbeck process suggests that such averages are asymptotically normal. From a prac...

In this chapter, we introduce two techniques of statistical inference: hypothesis testing and computing confindence intervals. Following this chapter, we will frequently use these techniques to answer questions about linear models. We begin the chapter by exploring two types of continuous distributions commonly used in statistical inference: normal and t distributions. After that, we show how t...

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 General Probability Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . ...