Delayed Feedback Effects 1 Delayed Feedback Disrupts the Procedural-Learning System But Not the Hypothesis- Testing System in Perceptual Category Learning
چکیده
Maddox, Ashby and Bohil (2003) found that delayed feedback adversely affects information-integration, but not rule-based category learning in support of a multiple systems approach to category learning. However, differences in the number of stimulus dimensions relevant to solving the task and perceptual similarity failed to rule out two single-system interpretations. An experiment was conducted that remedied these problems, and replicated Maddox et al.’s findings. The experiment revealed a strong performance decrement for information-integration, but not rule-based category learning under delayed feedback that was due to an increase in the number of observers using hypothesis-testing strategies to solve the information-integration task, and lower accuracy rates for the few observers using information-integration strategies. Maddox, Ashby and Bohil (2003) examined the effects of delayed feedback on rulebased and information-integration category learning. Rule-based category learning tasks are those in which the category structures can be learned via some explicit reasoning process. Frequently, the rule that maximizes accuracy (i.e., the optimal rule) is easy to describe verbally. Informationintegration category learning tasks are those in which accuracy is maximized only if information from two or more stimulus components (or dimensions) is integrated at some pre-decisional stage (Ashby & Gott, 1988). For example, the perceptual integration might take the form of a weighted linear combination of the dimensional values. Maddox et al. (2003) offered a critical test of a prediction derived from Ashby, AlfonsoReese, Turken, and Waldron’s (1998) COmpetition between Verbal and Implicit Systems 1 This research was supported in part by National Institute of Health Grant R01 MH59196, and by the Center for Perceptual Systems at the University of Texas, Austin. We thank Kelli Hejl and Mina WilcoxGhanoonparvar for help with data collection, and two anonymous reviewers for comments on an earlier version of this article. Correspondence should be addressed to W. Todd Maddox, Department of Psychology, 1 University Station A8000, University of Texas, Austin, Texas, 78712 (e-mail: maddox@psy.utexas.edu). 2 However, a conjunctive task (e.g., respond A if the stimulus is small on dimension x and small on dimension y) is rule-based rather than an information-integration task because separate decisions are first made about each dimension (e.g., small or large) and then the outcome of these decisions is combined (integration is post-decisional). Notice that the optimal rule for the conjunctive task is amenable to verbal description. In contrast, the optimal rule in information-integration tasks is often difficult or impossible to describe verbally. Delayed Feedback Effects 2 (COVIS) model of category learning. COVIS, like other recently proposed multiple systems models (e.g., Erickson & Kruschke, 1998; Pickering, 1997; Reber & Squire, 1994; Smith, Patalano, & Jonides, 1998; however see Nosofsky & Johansen, 2000), assumes that in humans, the learning of different types of category structures is mediated by different categorization and memory systems. COVIS also makes assumptions about the underlying neurobiology that mediates learning in these systems. Here we review these assumptions in enough detail to motivate the current study. More thorough reviews can be found in other sources (e.g., Ashby, et al., 1998; Ashby & Ell, 2001; Maddox & Ashby, in press). COVIS assumes that learning in rule-based tasks is dominated by an explicit hypothesistesting system that uses working memory and executive attention and is mediated by a circuit that includes the anterior cingulate, the prefrontal cortex, and the head of the caudate nucleus. This system learns through a conscious process of hypothesis generation and testing. In contrast, learning in information-integration tasks is assumed to be dominated by an implicit, procedurallearning-based system that depends on a reward signal to strengthen the appropriate (stimuluscategory) associations in a relatively automatic fashion (Ashby et al., 1998; Ashby & Ell, 2001). The procedural-learning system is mediated largely within the tail of the caudate nucleus (with visual stimuli). In primates, all of extrastriate visual cortex projects directly to the tail of the caudate nucleus, with about 10,000 visual cortical synapses converging onto each medium spiny cell in the caudate (Wilson, 1995). These medium spiny cells then project to prefrontal and premotor cortex (via the globus pallidus and thalamus; e.g., Alexander, DeLong, & Strick, 1986). A dopamine-mediated reward signal is critical for learning in this system. The idea is that an unexpected reward causes substantia nigra neurons to release dopamine from their terminals in the caudate nucleus (Hollerman & Schultz, 1997; Schultz, 1992), and that the presence of this dopamine strengthens recently active synapses (Arbuthnott, Ingham, & Wickens, 2000; Kerr, & Wickens, 2001). In reward-mediated learning, it is essential to strengthen those (and only those) synapses that actively participated in the response that elicited the reward. Because there is necessarily some delay between response and reward delivery, a chemical trace must be maintained that signals which spines were recently active. Toni et al. (1999) demonstrated that calcium catalyzes morphological changes in cells that have spiny dendritic complexes (see also Calabresi et al., 1996). The morphology of the dendritic spines allows this calcium dependent trace to exist for some time after the response is initiated (Gamble & Koch, 1987; MacDermott, Mayer, Westbrook, Smith, & Barker, 1986). Under ideal conditions, the dopamine-mediated reward signal will arrive during this time and act synergistically with the calcium-mediated perceptual trace, eventually resulting in structural changes that result in learning. If the reward is delayed, then the perceptual signal will degrade and the ensuing dopamine release will modify inappropriate synapses, which causes learning to be adversely affected. Thus, COVIS predicts that the information learned, and the processing of the feedback signal should be very different in the two systems. In the explicit system, verbalizable rules that can be stored in working memory until feedback is provided are learned, even if an extended period elapses between the response and feedback. Once the feedback is presented, time and attention are required to process it. In the implicit, procedural-learning system, stimulus-tocategory associations are learned that require a close temporal correspondence between the perceptual signal and the feedback; but once the feedback signal is presented, feedback processing is essentially automatic. Maddox et al. (2003) tested the prediction that a close temporal correspondence between the perceptual signal and the feedback is required for efficient learning in the procedural-learning system but not in the hypothesis-testing system by comparing rule-based and informationintegration category learning across immediate and delayed feedback conditions. In both category learning tasks (and in the current study) the stimulus on every trial was a single sine-wave grating (i.e., Gabor stimulus) that varied across trials in spatial frequency and spatial orientation. In the rule-based condition, spatial frequency was relevant and spatial orientation was irrelevant, and the Delayed Feedback Effects 3 optimal decision rule was to respond A if the spatial frequency was low and to respond B if the spatial frequency was high. In the information-integration condition, the optimal decision rule required a linear integration of the spatial frequency and orientation values and was not verbalizable. In line with predictions from COVIS, learning under delayed feedback conditions was as effective as under immediate feedback conditions with rule-based categories, but with information-integration categories, delayed feedback led to a significant decrease in category learning performance. This result is in line with the COVIS prediction that a temporal delay in feedback presentation should lead to synaptic activity in the tail of the caudate that is non-specific to the stimulus, resulting in a modification of inappropriate synapses, and finally leading to poor information-integration category learning. It is important to note that under ideal conditions, rulebased tasks will be solved by the explicit, hypothesis-testing system and information-integration tasks will be solved by the implicit, procedural-learning-based system; but under non-optimal training conditions such as delayed feedback, COVIS predicts that the participant will likely rely on the explicit, hypothesis-testing system to solve the information-integration task, even though hypothesis-testing strategies are not optimal. As a test of this prediction, Maddox et al. (2003) applied a set of models that tested for the presence of hypothesis-testing or informationintegration strategies. Model sets were fit individually to each participant’s data. As predicted, they found that participants were much more likely to use hypothesis-testing strategies to solve the information-integration task in delayed feedback conditions as compared to immediate feedback conditions. Although these data provide strong support for a theory of multiple systems, there are at least two alternative single system explanations for these results. First, one might argue that a single category learning system solves both tasks, but that the information-integration task is more complex or difficult because it requires the observer to represent both stimulus dimensions whereas the rule-based task requires the observer to represent only one dimension. Delaying the feedback to this single category learning system does not affect a task that requires a unidimensional representation, but does affect a task that requires a two-dimensional representation. One might argue against this hypothesis by pointing out that rule-based and informationintegration category learning performance was equated in the immediate feedback condition to ensure that task difficulty differences could not be used to explain the observed performance dissociation. However, to equate immediate feedback performance, Maddox et al. (2003) lowered the category discriminability in the rule-based condition relative to the informationintegration condition. This leads to the second single system explanation of the results which states that the rule-based and information-integration categories were difficult for different reasons. Rule-based category learning was difficult because the stimuli were perceptually very similar whereas information-integration category learning was difficult because observers must learn to associate different regions of stimulus space with each category. Because the two tasks were difficult for different reasons, the delayed feedback effect might not be attributable to separate category learning systems. Experiment Both of these alternatives are reasonable, and must be ruled-out before a multiple system interpretation of the delayed feedback prediction is accepted. To achieve this goal we took the following approach. First, whereas the optimal rule in Maddox, et al’s (2003) rule-based task required only a uni-dimensional representation, the rule-based task used in the current study required a two-dimensional representation (see Figure 1). In particular, the optimal rule required the observer to set a criterion on the spatial frequency dimension to determine whether the stimulus had a “low” or “high” spatial frequency, and to set a separate criterion on the spatial orientation dimension to determine whether the stimulus had a “shallow” or “steep” orientation. These decisions where then integrated to generate a categorization response (post-decisional integration). The information-integration task also required a two-dimensional representation, but Delayed Feedback Effects 4 in this case the integration was pre-decisional. Notice from Figure 1 that both the rule-based and information-integration tasks require the use of two decision rules (or bounds) and utilize four categories instead of two. Because few category learning studies have examined learning with more than two categories (however see Maddox, Filoteo, Hejl, & Ing, 2004), this is a unique extension of the current work. Second, whereas the rule-based discriminability was very low in Maddox, et al, and many stimuli from each category where perceptually similar possibly leading to problems with perceptual discrimination, in the current study we utilized rule-based category structures that were more perceptually dissimilar (see Figure 1). In fact, to equate rule-based and information-integration category learning in the immediate feedback conditions, we had to increase the rule-based category discriminability relative to information-integration category discriminability. The rule-based and information-integration category structures used in the Experiment are depicted in Figure 1 along with the decision bounds that maximize accuracy. The distribution parameters are outlined in Table 1. In the rule-based (RB) task the optimal bounds require the participant to use the conjunctive rules: “Respond A if the frequency is low and the orientation is shallow, respond B if the frequency is high and the orientation is steep, respond C if the frequency is low and the orientation is shallow, or respond D if the frequency is high and the orientation is steep”. In the information-integration (II) task, the category distributions were constructed by rotating the RB distributions clockwise 45 degrees around the center of the frequency-orientation space and then reducing discriminability in order to equate performance in the immediate feedback condition. (A series of small sample pilot studies were conducted to determine the II category distributions that met this criterion.) The optimal rule in the II condition has no simple verbal description. INSERT FIGURE 1 AND TABLE 1 ABOUT HERE Method Participants and Design. One-hundred-twelve participants were solicited from the University of Texas community and received course credit for participation. Twenty-seven and 29 participants completed the RB and II immediate feedback conditions, respectively; and 29 and 27 completed the RB and II delayed feedback conditions, respectively. No participant completed more than one experimental condition. All participants were tested for 20/20 vision using a Snelling eye chart. In nearly all of our current work with two categories we define a “learner” as a participant who achieves 65% accuracy during the final block of trials. Because the Experiment included four categories we lowered the criterion proportionally to 32.5% accuracy during the final block of trials. The data from participants who did not meet this criterion were excluded from all subsequent analyses. This criterion excluded 5 and 3 participants from the RB and II immediate feedback conditions and 4 and 7 from the RB and II delayed feedback conditions. Stimuli and Stimulus Generation. The experiment used the randomization technique introduced by Ashby and Gott (1988). Eighty stimuli (20 from each category) from the RB categories were generated by sampling randomly from four bivariate normal distributions. The stimuli for the II categories were generated by rotating the 80 RB stimuli clockwise by 45° around the center of the frequency-orientation space, and then shifting the stimuli away from the center of the space. Each set of 80 stimuli was displayed in a random order in each of four blocks of trials. 3 Two comments are in order regarding this learning criterion. First, after data collection we examined other learning criterion values including one that excluded no data. The qualitative pattern of results was relatively unchanged across conditions. Second, to determine whether the “non-learners” displayed essentially random responding, or whether they displayed systematic responding, we fit decision bound models to the data and also models that assumed a fixed response probability for all stimuli (indicative of random or biased random responding). In all but a few cases, the models indicative of random or biased random responding provided the best account of the non-learners data. Delayed Feedback Effects 5 The stimuli were computer generated and displayed on a 21” monitor with 1360 X 1024 resolution in a dimly lit room. Each Gabor patch was generated using Matlab routines from Brainard’s (1997) Psychophysics Toolbox. Each random sample (x1, x2) was converted to a stimulus by deriving the frequency, f = .25 + (x1/50), and orientation, o = x2(π/500). The orientation scaling factors were chosen to approximately equate the salience of frequency and orientation. Procedure. The participants were informed that there were four equally-likely categories, that perfect performance was possible and that they should be as accurate as possible and not worry about speed of responding. The procedure for a typical trial was as follows: Immediate feedback condition: Response terminated stimulus display – 500ms Mask – 750ms feedback – 5-sec blank screen ITI Delayed feedback condition: Response terminated stimulus display – 5-sec Mask – 750ms feedback – 500ms blank screen ITI The mask was a Gabor pattern that subtended approximately 11 degrees of visual angle and was of a random frequency and orientation from within the range of stimulus values. Results and Theoretical Analysis Analyses were performed separately on each block of data. In the first section (entitled ANOVA Results) we analyze the accuracy rates using ANOVA. In the second section (entitled Model Results) we introduce the model-based analyses. Results and Discussion