This study investigates the inter-rater reliability of appraisers who award exceptional quality points on the Texas Teacher Appraisal Instrument. Inter-rater reliability was measured when appraisers scored exceptional quality points after viewing a videotaped lesson. Comparisons were made between appraisers when grouped according to elementary or secondary certification, sex, years of administrative experience, and type of training. A total of 707 subjects from 56 school districts participated in the study. Five research hypotheses were formulated with the .05 level of significance for acceptance. All hypotheses were tested by correlation of coefficients, multiple response procedures, frequencies, and percentages.
The data measuring inter-rater reliability of the appraisers in training imply that there is very little reliability in the awarding of exceptional quality points on the Texas Teacher Appraisal Instrument. The findings of this study are that certification, sex, administrative experience, and type of training made no significant differences when scoring the instrument. Therefore, it is concluded that the scoring of exceptional quality points is a subjective, professional judgment made by each appraiser when observing a teacher. Since no significant reliability was found, the scoring of exceptional quality points cannot be supported as a reliable means of determining the quality of teaching in Texas schools. Generally, elementary certified appraisers awarded fewer exceptional quality points than secondary appraisers, males awarded slightly more points than females, appraisers indicated no noticeable trend because of years of administrative experience, and less experienced appraisers had the tendency to award more points than experienced appraisers.
Therefore, inter-rater reliability in awarding exceptional quality points cannot be expected on a consistent basis. Each appraiser, regardless of certification, sex, years of administrative experience or training, will use his or her own professional judgment when scoring the instrument.
Place Name coordinates. (May be approximate.)
Place Name coordinates. (May be approximate.)
