Journal of Agricultural Education - Vol. 40

This was a descriptive study to assess inservice education needs of entry-phase agriculture teachers in Texas. The purpose was to compare the ranking of inservice needs as determined by direct assessment to a ranking based on a mean weighted discrepancy score (MWDS), i.e., the Borich model. Via a mail questionnaire, teachers “graded” their performance for I63 competencies and their need for inservice (expressed need) for each competency. Teachers then responded to a second questionnaire by “‘rating” the level of importance for these competencies. Each questionnaire used a Likert-type scale. Subjects responded to 51 “core " competencies and to approximately one-fourth of the remaining items. During Phase I, competencies were ranked based on teachers’ expressed need The importance rating, obtained during Phase II, allowedfor calculation of a MWDS for each competency and ranking of the competencies based on these scores. Only four competencies ranked among the top 15 for both rankings. Because MWDS rankings were more congruent with inset-vice needs identified in the literature, it was concluded that a discrepancy approach does provide a more valid picture of inservice needs. Providers should allocate resources based on MWDS rankings. Future needs assessment studies should incorporate the calculation and ranking of MWDS. Introduction and Theoretical Base There is a general consensus among all educators that resources are precious. Witkin (1984, p. x) stated, “Effective needs assessment provides the basis for decisions on priorities either for program development or retrenchment.” Any difference between “desired status of learners” and “current status of learners equals an educational need” (Popham, 1993, p. 67). Identifiable areas of need may be used as decision rules for determining future resource allocation. Therefore, the method by which needs are identified and prioritized for delivery must be valid. Historically, one of the main functions of collegiate agricultural education departments has been the identification of relevant topics to provide agriculture teachers during inservice training (Bar-rick, Ladewig, & Hedges, 1983). Researchers (Garton & Chung, 1995; Mundt & Connors, 1997) have noted the relationship between problems entry-phase agriculture teachers encounter and opportunities the problems create for providing inservice. According to Birkenholz and Harbstreit (1987 , p . 48), inservice providers should “periodically monitor the needs of beginning teachers as they change over time and provide assistance based upon current needs.” Further, Garton and Chung (1995, p. 78) stated that “research is needed to assess the inservice needs of today’s beginning agriculture teachers.” Many researchers have used direct assessment models for determining inservice needs of entry-phase agriculture teachers (Birkenholz & Harbstreit, 1987; Briers & Edwards, 1998; Claycomb & Petty, 1983; Farrington, 198 1; Miller & Scheid, 1984; Shippy, 1981; Webb, Stoner, & Vaclavik, 1977). Others have used the Borich model (Barrick & Doerfert, 1989; Barrick & Powell, 1986; Garton & Chung, 1995; McDonald Journal of Agricultural Education J 0 40 Vol. 40, No. 3 1999 & Lawver, 1997; Newman & Johnson, 1994). The Borich model is based on a discrepancy score derived from respondent-determined level of importance and level of performance for the specific competency being assessed (Borich, 1980). Borich (1980) concluded the model "...is sufficiently direct that data analysis and instrument construction are no more complex than with any type of follow-up survey; yet it yields more data, and more understandable data, than many other types of follow-up questionnaires” (p. 42). Barrick et al. (1983) stated that to select inservice topics based on one ranking “would be less reliable than selecting topics based upon a combination of rankings” (p. 16), i.e., the Borich model. Further, Barrick et al. (1983, p. 15) “hypothesized that there would be a significant difference among the rankings of the topics [for inservice] by importance scores, knowledge scores, and application scores.” In 1983, Bar-rick et al. tested the Borich model and found the use of only one ranking, whether it is importance, knowledge, or application, “may not be valid” (p. 19), and that “a combination of two or more rankings must be considered to form conclusions regarding inservice education needs” (p. 19). Bar-rick et al. (1983) concluded, “The [Borich] model provided defensible data in identifying important topics in which teachers need further knowledge” (p. 19). Other researchers have supported Bar-rick’s conclusions (Newman & Johnson, 1994). Waters and Haskell (1989) stated the Borich model "...appears to have merit for adding validity to the process of assessing respondents perceptions about the importance of educational program needs.. ." (p. 26). Inservice education for entry-level teachers in Texas was a driving force for this study. However, the specific questions answered in the study were as follows: Will the ranking of competencies be different depending on whether they were ranked based on mean rating scores (expressed need) or were ranked by mean weighted discrepancy score? Which one of these rankings will be more valid? Purpose and Research Questions This study represents Phase II of a twopart investigation whose purpose was to identify inservice needs of entry-phase agriculture teachers in Texas. The purpose of Phase II was to compare the ranking of inservice needs as determined by direct assessment in Phase I (Briers & Edwards, 1998) to a ranking of those needs based on a mean weighted discrepancy score. These research questions guided this phase: (1) What are the rankings for inservice education of these competencies based on mean weighted discrepancy scores? (2) How do mean weighted discrepancy score rankings compare to Phase I rankings for in-service education based on a direct assessment approach (expressed needs for inservice)? Methods and Procedures In 1997, the Department of Agricultural Education at Texas A&M University, in cooperation with the Texas Education Agency (TEA), conducted Phase I of a descriptive study to assess inservice needs of entry-phase agriculture teachers in Texas. Phase II of the study was conducted in the spring of 1998. The target population for Phase I consisted of entry-phase teachers. “Entry-phase” was defined as teachers who began teaching during the school year 199495, 1995-96, or 1996-97. Those surveyed consisted of “additions” to the Directory: Texas Teachers of Agricultural Science and Technology (Texas Education Agency, 1994; Texas Education Agency, 1995; Texas Education Agency, 1996) for academic years 1994-95, 1995-96, and 199697. In Phase I, 165 teachers were identified as “entry-phase” teachers. Ninety-one (55%) of these teachers responded. These respondents were the target data source for Phase II. Between Phases I and II, the group experienced a mortality of 15 Journal of Agricultural Education 4 1 Vol. 40, No. 3 1999 teachers (i.e., no longer teaching agriculture). The final sample frame for Phase II was 76 teachers. A list of competencies needed by agriculture teachers was developed based on a review of literature (Barrick & Powell, 1986; Birkenholz & Harbstreit, 1987; Claycomb & Petty, 1983; Farrington, 1981; Garton & Chung, 1995; Miller & Scheid, 1984; Norton, 1995; Shippy, 1981; Webb et al., 1977). Content validity of the instrument was established by agricultural educators in Texas, including members of the Texas A&M University Department of Agricultural Education and members of the Texas Education Agency state staff for Agricultural Science and Technology. The conceptual framework for competencies originated from DACUM (Norton, 1995). The final list consisted of 163 different competencies, divided into 14 competency “areas.” Three areas were determined to be “core competency area”: “Facilitating Student Learning in Classroom and Laboratory Settings” (22 competencies), “Facilitating Student Leadership and Personal Growth” (16 competencies), and “Facilitating Student Agricultural Experiences” (13 competencies) (Edwards, Briers, Shinn, & Herring, 1998). To shorten the instrument, the remaining competencies were grouped as follows: “Student Services Competencies” (32 items); “Program Management Competencies” (24 items); “Personal Roles & Relationship Competencies” (33 items); “Planning & Managing Educational Tools & Technologies” (23 items). In Phase I, members of the population were randomly assigned to one of four groups, with each group receiving a different instrument. These same groups were used for Phase II. A matrix sampling technique asked each subject to respond to the 51 core competencies and to approximately one-fourth of the remaining items (23 to 33). In Phase I, teachers were asked to “grade” their level of performance (ability) for the selected competencies. (This rating of their ability to perform selected tasks, that is, their competence, was used in Phase II in calculating discrepancy score.) Next, teachers were asked to rate directly their need for inset-vice training (expressed need), with “5” meaning “highest need”, “4” representing “much need”, “3” was “some need”, “2” being “little need”, and “1” meant “no need”. Finally, subjects responded to items describing themselves and their schools. In Phase I, the 163 competencies were ranked based on the teachers’ expressed need for inservice education. See Briers and Edwards (1998) for specific results/findings of Phase I. In Phase II, teachers were asked to “rate” the level of importance for the selected competencies: “5” was “high importance", "4" was “much importance”, "3 " was “some importance”, “2” was “low importance”, and “1” meant “no importance”. Obtaining an importance rating allowed for calculation of a mean weighted discrepancy score for each competency. First, a discrepancy score for each teacher on each competency was calculated by subtracting the grade (ability) rating from the importance rating. A weighted discrepancy score was then calculated for each teacher on each competency by multiplying the discrepancy score by the mean importance rating for that competency. A mean weighted discrepancy score for each competency was then calculated by dividing the sum of the weighted discrepancy scores by the number of observations for that competency. Finally, the 163 competencies were ranked using the mean weighted discrepancy score (MWDS) (Bar-rick et al., 1983; Borich, 1980; Garton & Chung, 1995; Newman & Johnson, 1994). There were 163 competencies and 63 respondents, and because each teacher responded to only a subset of all competencies, the researchers did not attempt to do any data reduction; factor analysis, for example, was inappropriate. One limitation of this study, then, is that there are no measures of reliability (internal consistency within the competency areas) of the Journal of Agricultural Education 42 Vol. 40, No. 3 1999 participants responses. However, even though the researchers did not examine internal consistency statistically, conceptually the competencies were grouped into competency areas (Norton, 1995). The first mailing of Phase II, in January, 1998, included an instrument, a cover letter explaining the purpose of the survey, and a return envelope coded to determine non-respondents. In February, 1998, a reminder postcard was sent to non-respondents (Gall, Borg, & Gall, 1996). Following the reminder postcard, a second instrument, a slightly altered cover letter, and a second return envelope were mailed to nonrespondents (Gall, Borg, & Gall, 1996). Finally, an attempt was made to contact non-respondents via telephone. Some contacted by telephone requested a third questionnaire; one was mailed to each who requested one. Three mailings, a reminder postcard, and telephone follow-up of non-respondents yielded a return rate of 83% (63 of 76). Results and Findings Of the 163 competencies, four ranked among the top fifteen on both the mean weighted discrepancy score and expressed needs mean (Table 1). They were: “Assisting students in preparing for and succeeding in FFA degree and award programs” MWD S Rank= 1, Expressed Needs Rank=15; “Using Internet as a teaching tool” MWDS Rank=2 “Expressed Needs” Rank=1; “Implementing Tech-Prep and other S-TW initiatives into the program” MWDS Rank=6 Expressed Needs Rank=13; “Integrating CAD into ag mech" MWDS Rank=8 Expressed Needs Rank=2. These findings agree with Garton and Chung (1995). When ordered based on mean weighted discrepancy score, the 20 highest-ranking competencies represented nine of the 14 different competency areas. Fifteen of these 20 competencies came from four areas. “Facilitating Change in Curriculum and Technologies” produced rankings 2, 6, 8, and 18 (Table 1). “Facilitating Balance in Personal and Professional Roles” provided rankings 3, 9, 11, and 13 (Table 1). The area “Facilitating Positive Public Image” yielded rankings 4, 5, 12, and 14 (Table 1). The competencies ranked 1, 10, and 15 came from the area “Facilitating Student Leadership and Personal Growth” (Table 1). Other high ranking competencies were “Actively read professional literature and participate in educational events,” MWDS Rank=7 “Teaching how to keep good record books,” MWDS Rank=16; “Helping gather information about agricultural scholarships,” MWDS=17; “Motivating student learning and improving achievement,” MWDS Rank=1 9; and “Control loss of tools, equipment, supplies, and materials,” MWDS Rank=20 (Table 1). Inservice needs relative to “Facilitating Change in Curriculum and Technologies,” e.g., computer assisted instruction, have been identified by other researchers (Birkenholz & Harbstreit, 1987; Garton & Chung, 1995; Newman & Johnson, 1994). Interestingly, in Phase I of this study, Briers and Edwards (1998) found teachers expressed need for inservice for competencies related to human relations to be low, which was in contrast to earlier research (Claycomb & Petty, 1983). Yet, it appears that when “level of importance” becomes part of the equation, teachers do desire inservice in this area, i.e., “Facilitating Balance in Personal and Professional Roles” (competencies ranked by MWDS 3, 9, 11, and 13) (Table 1). The fact that four competencies related to” Facilitating Positive Public Image” were highly ranked agrees with earlier research (Garton & Chung 1995). Also, the literature supports providing entry-phase teachers with additional training in the area of “Facilitating Student Leadership and Personal Growth,” i.e., the FFA (Birkenholz & Harbstreit , 1987; Garton & Chung, 1995; Shippy, 198 1; Talbert, Camp, & Heath-Camp, 1994; Webb et al., 1977). Journal of Agricultural Education 43 Vol. 40, No. 3 1999 Table 1. Rankings of Inservice Needs of Entry-Phase Agriculture Teachers, MWDS” versus Expressed Needs (N=63) MWDS Ranking Competency Expressed Needs 1 Assisting students in preparing for and succeeding in FFA degree and award programs 1 5