The Accelerated Reader multiple-choice tests are created by specialists who read the books and draft item banks of comprehension questions relating to story grammar and text structure for each title. Correct and incorrect (distracter) answers are carefully constructed. Then, adult reader-evaluators review the draft version of each test on the computer, considering consistency of vocabulary, plot detail, chronological order of events, the extent to which questions cue certain answers, the balance of answer choices presented, and other issues. This leads to the discarding or revision of the less discriminatory, unreliable, or ambiguous items. (In the past, child reader-evaluators were also involved in the editing phases, but they were found to miss too many errors.) Each question is checked against the book at each stage, and tests are designed to ensure that a child could not pass without reading the book. The tests are then reviewed by quality assurance professionals and the program's senior editor (Institute for Academic Excellence, 1998). Thus, the psychometric properties of the AR comprehension tests are likely to be considerably more stable than those developed under much greater time constraints by busy and creative professional teachers.

Post hoc assessment of the reliability of the final version of an AR test presents difficulties, although such assessment is not usual with curriculum-based measures. Accelerated Reader tests are brief, and internal consistency checks are therefore of doubtful applicability. Further, the tests are taken in the context of a highly individualized self-chosen reading curriculum. Traditional test-retest reliability checks are of limited value when motivation at retest is likely to be much lower than at first test; immediate retest is likely to be influenced by practice effects and delayed retest by individual variations in memory.

The AR tests are brief, assess primarily literal comprehension (rather than idiosyncratic reader inferences or other more complex responses which might in any case be culturally specific), and do not pretend to measure all relevant reading behaviors in school or elsewhere. Indeed, AR questions are deliberately restricted to those that demonstrate adequate reliability. Nevertheless, AR points gained are likely to be a fairly accurate measure of the quantity and difficulty of words read and comprehended, and therefore can serve as a useful aggregate measure of successful reading practice.

Note, however, that the points metric has a marked “floor” effect, in that no points are gained for performances of less than 60 percent correct. Thus, if students do not read the book throughout with care and consistency, or the book presents a very high degree of challenge to the student, no points are scored. This implies that the upper extremes of challenge are likely to be missing from any attempt to relate challenge to attainment.

Go to map