ویراساینس.
سند علمی

G : Bandits , Experts and Games 11 / 07 / 16 Lecture 10 : Contextual Bandits

Alex Slivkins, Guowei Sun, Cheng Jie

چکیده

• algorithm observes a “context” xt, • algorithm picks an arm at, • reward rt ∈ [0, 1] is realized. The reward rt depends both on the context xt and the chosen action at. Formally, make the IID assumption: rt is drawn independently from some distribution that depends on the (xt, at) pair but not on t. The expected reward of action a given context x is denoted μ(a|x). This setting allows a limited amount of “change over time”, but this change is completely “explained” by the observable contexts. Throughout, we assume contexts x1, x2, . . . are chosen by an oblivious adversary. The main motivation is that a user with a known “user profile” arrives in each round, and the context is the user profile. Some of the natural scenarios include: choosing which news articles to display to a given reader, which products to recommend to a given customer, and which interface options (e.g., font sizes and colors, page layout, etc.) to use for a given website visitor. Rewards are often determined by user clicks, possibly in conjunction with other observable signals that correlate with revenue and/or user satisfaction. Naturally, rewards for the same action may be different for different users. Contexts can include other things apart from (and instead of) the user profiles. First, known features of the environment, such as day of the week, time of the day, season (e.g., Summer, preChristmas shopping season), proximity to a major event (e.g., Olympics, elections). Second, the set of feasible actions: only a subset of actions may be allowed in a given round and/or for a given user. Third, actions can come with features of their own, and it may be convenient to include this information into the context, esp. if the action features change over time. For ease of exposition, we assume a fixed and known time horizon T . The set of actions is denoted by A, and K = |A| is the number of actions. X denotes the set of all contexts. For brevity, we will use CB instead of “contextual bandits”. The reward of an algorithm ALG is REW(ALG) = ∑T t=1 rt, so that the expected reward is

اسناد مشابه

  • G : Bandits , Experts and Games 09 / 12 / 16 Lecture 2 : Multi - arm Bandits with i . i . d Rewards

    سند علمی۲۰۱۶Amr Sharaf, Liqian Zhang

    In the multi-arm bandit problem with iid rewards, the learner selects an arm a ∈ A at every time step t. Learning proceeds in rounds, and we assume that the number of rounds is fixed, and indexed by t = 1 · · ·T . At each round, the algorithm chooses one action at (we’ll use the

  • G : Bandits , Experts and Games 09 / 12 / 16 Lecture 4 : Lower Bounds ( ending ) ; Thompson Sampling

    سند علمی۲۰۱۶Alex Slivkins, Guowei Sun, Cheng

    Here is a parameter to be adjusted in the analysis. Recall that K is the number of arms. We considered a “bandits with predictions” problem, and proved that it is impossible to make an accurate prediction with high probability if the time horizon is too small, regardless of what

  • Linear game non-contextuality and Bell inequalities—a graph-theoretic approach

    سند علمی۲۰۱۶R Ramanathan, S Severini

    We study the classical and quantumvalues of a class of oneand two-party unique games, that generalizes thewell-knownXORgames to the case of non-binary outcomes. In the bipartite case the generalizedXOR (XOR-d) gameswe study are a subclass of thewell-known linear games.We introduc

  • Expert games: means to acquire expert knowledge

    سند علمی۱۹۹۳The Computer Science Journal of MoldovaGennady Ginkul

    This article describes the idea and use of non-traditional technique designed to acquire Expert's knowledge for Expert Systems. The procedures, called " Expert Games " , look outwardly as well-known computer " playthings " and at the same time are strong means for Knowledge Acqui

  • Enhancing speech recognition in fast-paced educational games using contextual cues

    سند علمی۲۰۱۳SLaTECarrie J. Cai, Rob Miller, Stephanie Seneff

    Arcade-style games like Tetris and Pacman are often difficult to adapt for educational purposes because their fast-paced intensity and keystroke-heavy nature leave little room for simultaneous practice of other skills. Incorporating spoken language technology could make it possib

  • Novices Vs. Experts; Game-Based Learning and the Heterogeneous Classroom Audience

    سند علمی۲۰۱۵Björn Berg Marklund

    This paper examines how the heterogeneity of K-12 students, as game audiences, affect the way games can be used as educational tools in formal education. When discussing the application of games in educational contexts, the realities of the formal educational environment are seld