Gender Differences 1 Running head: GENDER DIFFERENCES IN SEARCHING THE WEB Gender Differences in Patterns of Searching the Web
چکیده
There has been a national call for increased use of computers and technology in schools. Currently, however, little is known about how students use and learn from these technologies. This study explores how eighth-grade students use the Web to search for, browse, and find information in response to a specific prompt (how mosquitoes find their prey). A previous analysis (Roy, Taylor & Chi, in press) found that boys performed significantly better on gaining target-specific (information directly related to the prompt) and target-related (information related to mosquitoes in general) knowledge than girls. The current paper explores this difference further by examining how students searched the Web for information. Each student’s search behavior was diagramed out and a series of six different “search moves” were derived. Statistical analysis of these search variables revealed that boys tended to employ a different search pattern from girls and that this variation in search behavior was related to the pattern of performance outcomes. Gender Differences 3 Gender Differences in Patterns of Searching the Web The World Wide Web (Web) is a large distributed hypermedia repository of information where people can search for information, navigate through links, and view pages of documents through browsers. Web users are faced with finding their way through this rich but unstructured information world. Increasingly, students are using this incredible resource from both school and home as a main avenue for finding information for their research projects and papers (Wallance, Kupperman, Krajcik & Soloway, 2000). Thus, students need to navigate this complex world of information effectively. But what are these navigational skills precisely, and are there important individual differences in the ways that students search for information on the Web? Previous research has shown that students vary widely in their ability to find and retrieve information in loosely structured information environments (Chang & McDaniel, 1995). Some factors that predict search success in such environments include level of domain knowledge and search expertise (MaKinster, Beghetto & Plucker, in press; McGregor, 1999), ability (Chang & McDaniel, 1995), gender (ChanLin, 1999), learner control (Diddon & Gabbard, 1998; McGreggor, 1999), learner style (Shute, 1993), and interest (Tobias, 1994). In general, more experienced, knowledgeable, interested, male users who are active learners and oriented towards an internal locus of control are associated with being successful in such environments. While several studies have investigated search behavior in closed hypermedia systems, it has been suggested (Hargittai, 2002) that finding information on the Web may be an even more complex search task in that the Web allows many different search strategies. Students can have difficulty finding useful, relevant, and accurate information on the Web. For example, students often lack the skill and resources to access and manage information intelligently (Salomon, 1998). Moreover, according to Wallace et al. (2000) students use the Web simplistically in that they tend to engage in “information gathering” (simply searching for and retrieving relevant Gender Differences 4 information as an end in itself) rather than in“information seeking” (the iterative and recursive process of finding and using relevant information to refine the search goal until it is met). Such findings suggest that the Web, in and of itself, might not be an appropriate resource for supporting such open ended learning tasks. A similar conclusion was reached by Vansickle and Monaco (2000) in their study of tenth graders use of the Web. They reported that students did not demonstrate much search sophistication. For example, students failed to see connections between the amount and kinds of information they retrieved and the way they searched for information. Search skills definitely improve with domain knowledge. To find information about a specific science concept, searchers with more sophisticated domain knowledge demonstrated greater understanding of the organization of scientific content, greater ability to evaluate Web page descriptors and content, and greater facility formulating appropriate keywords than searchers with less domain knowledge. Students with more impoverished domain knowledge had greater difficulty formulating effective search keywords, selecting appropriate sites or documents to pursue, and demonstrated a greater tendency to become distracted during the search process (Makinster et al., in press). This clearly suggests that search skills depend on content knowledge. But how do students fare when they are asked to use the Web to search for information about a topic for which they have little content knowledge? This would seem to be an important question to address since this is the type of task students are more routinely asked to perform. One individual difference that may be significantly related to successful search behavior is gender. The literature on gender differences in academic achievement in general is extensive. It is widely accepted that females score higher on average than males on tests of verbal abilities (Hyde & Linn, 1988), while males score higher on average than females on tests of mathematical Gender Differences 5 ability (Hyde, Fennema, & Lamon, 1990) and spatial abilities (Linn & Peterson, 1986). One might also expect gender to contribute to student’s ability to successfully search and navigate information on the Web, since there is evidence suggesting that girls lag behind boys in the degree to which they are experienced with and motivated by technology (Ligth et al., 2000; Schumacher & Morahan-Martin, 2001) and their attitude towards computers and experience using the Web (Leong & Hawamdeh, 1999). Such findings suggest that boys might out perform girls in their ability to search the Web since they are more experienced users and have more positive attitudes towards technology in general. But how does gender relate to search performance more specifically? Large, Beheshti, and Rahman (2002) investigated how same-sex collaborative groups of grade six students searched the Web for information to complete a class assignment. They found boys to be generally more active in the way they searched the Web, formulating more queries, clicking on more links per minute, and following up on more hits. On the other hand, girls spent more time reading documents. Although Large et al. (2002) quantified and compared aspects of search behavior across gender, they did not relate these behaviors to any measure of search success. Therefore it is not known if such differences are important in distinguishing successful from unsuccessful search patterns. Furthermore, it is possible that there are important differences between boys and girls not only in terms of the distribution of individual search behaviors, but also in terms of how those behaviors are orchestrated into a coherent search pattern. The current paper explores gender differences in how middle school students search the Web to find information in response to a specific prompt for a case where students possess little domain knowledge. That is, when students are unfamiliar with a topic, are boys and girls equally successful at locating information using the Web? What search patterns do they demonstrate? How do such search patterns relate to success in finding and using the targeted information? Gender Differences 6 The data reported here is part of a larger study comparing the ability of students to search for information on the Web verses in the school Library. In an earlier paper (Roy, Taylor & Chi, in press) we reported that boys and girls did not differ in their ability to search for, locate, and summarize information when using the school Library. However, when using the Web for an identical search task, boys performed significantly better than girls on both target-specific information (i.e., information they were specifically prompted to find) and target-related information (i.e., information that was related to the target topic, but was not the specifically prompted to find). This paper further explores this gender difference in the Web condition by analyzing student’s patterns of search behavior and by attempting to relate these differences to performance outcomes. If variation in the search patterns employed by boys and girls does explain the observed performance differences, then a careful analysis of these differences should inform the design of tools or scaffolds to support student’s navigation. Method Participants Fourteen eighth grade students from a local Pittsburgh middle school between the ages of 13 and 14 volunteered to participate in this study. Equal numbers of boys and girls participated. Materials The experiment was performed in the school computer laboratory on a Macintosh G3 computer running Internet Explorer as the Web browser and Google as the search engine. Target Domain and Knowledge Assessment. Mosquito behavior (i.e., how mosquitoes find their prey) was chosen as the target topic for several reasons. An informal survey of students prior to the study indicated that the topic was of interest and that most students did not know the answer. Furthermore, the question could be answered within a single academic period and there were sufficient available resources on the Web to answer the question successfully. Gender Differences 7 The materials used to measure student knowledge included (1) an open-ended prompt to assess student’s targetspecific knowledge (In your own words, please describe at least three ways that mosquitoes find their prey.), and (2) a set of 13 short answer questions to assess student’s target-related knowledge. The target-specific prompt was administered to assess both pre-search and post-search target-specific knowledge. The set of target-related questions was used to assess each student’s post-search general knowledge about mosquitoes only. These questions were designed to assess knowledge of facts that students were unlikely to know before their search but were likely to come across while performing their search (e.g., Why are mosquito bites itchy?). We asked these questions to assess general information about mosquitoes that students may have acquired incidentally while searching for information to the targeted prompt. Design and Procedure Each student met with two experimenters (one male and one female) in the school computer laboratory for an academic period (50 minutes) to perform the search task. Initially, each student was given five minutes to respond to the target-specific prompt. Following this pretest of target-specific knowledge, each student was given twenty-five minutes to search the Web for answers to the target-specific prompt. Students were permitted to take notes during their search and to "book mark" up to 30 pages of resources (by selecting “Add to Favorites” under the Favorites menu). Students were told that they would be permitted to refer back to these pages or notes during the posttest section. While each student conducted their search, the experimenters observed and recorded the participant’s search behavior and noted the identity, number, and sequence of all search queries executed and all resources that were examined by the student. The experimenters also recorded any bookmark selections or note taking. This information was used to supplement the computer logs of all Web pages accessed by students. Gender Differences 8 After completing the search, participants had ten minutes to answer the target-specific knowledge question again. Following this posttest, students were given the unannounced targetrelated test to measure incidental facts about mosquitoes that they might have gained during their search. Students were not permitted to use their notes for this assessment. Diagramming Student Search Behavior & Movement Using the field notes along with computer logs of all the Web pages accessed by students during their search, we were able to construct diagrams depicting each student’s search behavior. All search moves were categorized into one of the following four categories of search behavior: (1) submitting a search query in the Google search window; (2) scanning the list of returned document excerpts which contain links to documents; (3) Selecting, opening, and browsing a particular document; and (4) book marking a document location or taking notes. We arranged these behaviors in four columns from left to right. Each new instance of behavior within one of these four state categories was depicted on a new row. Finally, we linked these states with sequenced directed arrows in a manner that reflected the temporal order of movement from one state to another. Figures 1a and b below show small excerpts from two search episodes. These give some flavor of two overall patterns of search behavior that will be elaborated below (i.e. horizontal and vertical search moves). --Insert Figures 1a & 1b here--. Horizontal movement across the search diagram columns represents movement between (1) Submitting a search and Scanning pages describing the search results returned; (2) Scanning pages describing the search results to Browsing an actual document; or (3) Browsing documents to Book Marking or Note Taking (presumably after one has located the sought after information). Each horizontal move across search categories of behavior can be conceptualized as a change in the particular stage of a search episode, from early and more general (moving left Gender Differences 9 wards) to more specific or detailed (moving right wards). For example, the bottom of Figure 1a is an instance of a student searching horizontally. In this instance the student Submits his fourth search query (Mosquito prey), then Scans the first page of returned results (arrow 6), then Opens and Browses a document (arrow 7), and Takes Notes about carbon dioxide and octenal (arrow 8). On the other hand, vertical movement in the search diagrams represent instances where the student pursues a search at a particular stage in a depth-wise manner. In these cases the student moves from (1) search to search; (2) one page search result descriptions to another page; or (3) document to document by clicking on links. We can conceptualize each vertical movement within each search stage as an instance where the student pursues a particular line of search (movement down) or as revisiting information already browsed (movement up). For example, Figure 1b above depicts an instance of vertical search behavior. Specifically, the student moves through a sequence of documents (arrows 17 though 20) by clicking on a series of links. Deriving Measures of Search Behavior. Each subject’s total number of search moves was broken into proportions reflecting changes in search stage (submitting a search query to scanning results; scanning results to browsing documents; and browsing documents to taking notes) and changes in search depth within a stage (movement from search to search, results to results, document to document). Thus search behavior was indexed by six variables, three that reflected state changes in search stage and three that reflected changes in search depth within a particular stage. These variables served as the main quantitative indices for describing search patterns. Results We begin our presentation of results by revisiting the gender differences in performance outcomes that students gained from their Web searches, as reported in a previous paper (Roy et al., in press). In that study, although both boys and girls gained significantly from pre to posttest, Gender Differences 10 boys performed significantly higher than girls on both target-specific (Figure 2a) and targetrelated information measures (Figure 2b). --Insert Figures 2a & 2b here--. We know from the pretest analysis that boys and girls did not differ in their prior knowledge of the target domain. Nor were there differences in terms of familiarity or access to computers, with using Google, or with using the Web. Furthermore, boys and girls did not differ in terms of the total number of search moves, nor in terms of the amount of time spent searching. We turn now to an analysis of Search Behavior in an attempt to explain the learning differences. Overall Search Behavior. The six search variables were analyzed together in a one-way MANOVA with gender as the between-subjects variable. Using a MANOVA enabled us to test the gender difference on the overall pattern of all search variables considered simultaneously. There was a significant multivariate effect for gender (F (6,7) = 12.271, p = .002) indicating that the overall pattern of search behavior was different for boys and girls. Search Stages: Horizontal Moves. On average, boys demonstrated significantly more movement early on in the search phase than girls. Specifically, boys showed a greater tendency to move between Submitting searches and Scanning the results pages of returned by search queries (boys M = .306 and girls M = .101; F (1,12) = 25.528, p Chang, C.K., & McDaniel, E. D. (1995). Information search strategies in loosely structured settings. Journal of Educational Computing Research, 12(1), 95-107. ChanLin, L.J. (1999). Gender differences and the need for visual control. International Journal of Instructional Media, 26(3), 329-335. Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A review of the quantitative literature on learner comprehension, control, and style. Review of Educational Research, 68(3), 322-349. Hargittai, E. (2002). Beyond logs and surveys: In-depth measures of people’s Web use skills. Journal of the American Society for Information Science and Technology, 53(14), 1239- 1244. Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin, 107, 139-153. Hyde, J. S., & Linn, M. C. (1988). Gender differences in verbal ability: A meta-analysis. Psychological Bulletin, 104, 53-69. Large, A., Beheshti, J., & Rahman, T. (2002). Gender differences in collaborative Web searching behavior: An elementary school study. Information Processing and Management, 38, 427-443. Leong, S.C., & Hawamdeh, S. (1999). Gender and leaning attitudes in using Web-based science lessons. Retrrived from http://information.net/ir/5-1/paper66.html. Light, P. Littleton, K., Bale, S., Joiner, R., & Messer, D. (2000). Gender and social comparison effects in computer-based problem solving. Learning and Instruction, 10, 483-496. Linn, M. C., & Peterson, A. C. (1986). A meta-analysis of gender differences in spatial ability: Implications for mathematics and science achievement. In J. S. Hyde & M.C. Linn Gender Differences 15 (Eds.), The psychology of gender: Advances through meta-analysis. Baltimore: Johns Hopkins University Press. MacGregor, S. K. (1999). Hypermedia navigation profiles: Cognitive characteristics and information processing strategies. Journal of Educational Computing Research, 20(2), 189-206. MaKinster, J. G., Beghetto, R. A., & Plucker, J. A. (in press). Why can't I find Newton's Third Law?: Case studies of students using of the web as a science resource. Journal of Science Education and Technology. Roy, M., Taylor, R., & Chi, M.T.H. (in press). Searching for information on-line and off- line: Gender differences among middle school students. Journal of Educational Computing Research. Salomon, G. (1998). Novel constructivist learning environments and novel technologies: Some issues to be concerned with. Research Dialogue in Leaning and Instruction, 1(1), 3-12. Schumacher, P., & Morahan-Martin, J. (2001). Gender, Internet and computer attitudes and experiences. Computers in Human Behavior, 17(1), 95-110. Shute, V. (1993). A comparison of learning environments: All that glitters. In S. LaJoie & S. Derry (Eds.), Computers as Cognitive Tools (pp.47-73). Hillsdale, NJ: LEA. Tobias, S. (1994). Interest, prior knowledge, and learning. Review of Educational Research, 64, 37-54. Vansickle, S, & Monaco, M. (2000). Tenth graders’ search knowledge and use of the world wide Web. Unpublished doctoral dissertation, Georgia State University. Wallace, R., Kupperman, J., Krajcik , J., & Soloway, E. (2000). Science on the Web: Students online in a sixth-grade classroom. The Journal of the Learning Sciences, 9(1), 75-104. Gender Differences 16 Author Note Marguerite Roy and Michelene T. H. Chi, Learning Research and Development Center, University of Pittsburgh. We would like to acknowledge the contribution of Roger Taylor who assisted in the collection of the data. In addition, we would like to thank the students and staff who kindly volunteered to participate in this study. The authors are grateful for the support from the Spencer Foundation granted to Michelene T. H. Chi. Correspondence concerning this article should be addressed to either Marguerite Roy (mar982@pitt.edu) or Michelene Chi ( ) Chi@pitt.edu at 3939 O’Hara Street, Learning Research and Development Center, University of Pittsburgh, Pittsburgh, PA 15260. Gender Differences 17 Figure Captions Figure 1. a) An example of an excerpt from a horizontal search. b) An example of an excerpt from a vertical search. Figure 2. a) Pre and post target-specific knowledge scores by gender. b) Target-related knowledge scores by gender. Figure 3. a) Proportion of horizontal search moves by gender. b) Proportion of vertical search moves by gender. Gender Differences 18