Accelerated Reader Evaluation

EPSY 474 – Evaluating Learning Technologies

 Background Information

             The Accelerated Reader (AR) program is a product of Renaissance Learning, Inc. It was developed in 1993-1994. This program is designed to help monitor, manage and motivate reading practice. The STAR Reading Computer Adaptive test is used with Accelerated Reader. This test places students in appropriate reading levels. The AR program allows teachers to monitor and guide students’ reading and assure that all students are reading at an appropriate level (Education Commission of the States, 2000).

            The Accelerated Reader program includes many components. The main features of the program include the STAR test, quizzes, computer-generated reports and teacher assistance. The first component is the STAR test. As mentioned above, this test places students in appropriate reading levels. Students can take the test as many times as required by the teacher throughout the school year. Students receive a color-coded level for easy access to appropriate books. As a student takes the STAR test again, he/she may move to a different color-coded level as he/she progresses.

            The next component of the AR program is the computer quizzes. A student chooses a book according to his/her reading level. After the student reads the book carefully, he/she may take the computer-based comprehension quiz.  The student is then allotted a certain number of points based upon how many questions were answered correctly. Teachers usually use the acquired points with a system of incentives. For example, after a student acquires five AR points, he/she may attend the class party for the first quarter of the school year. Other incentive prizes may include bookmarks, pencils, books, and small toys.

            The last two components of the AR program include the computer-generated reports and teacher assistance. Teachers can access student reports at any time through the AR database. Teachers can view the individual scores of each quiz a student takes. Teachers can also print parent information forms for the results of the STAR testing and for student AR progress. Instructors can see exactly which books a student has struggled with. This allows teachers to directly assist the child with his/her reading comprehension by re-visiting the story and re-taking the quiz. These reports create a way for teachers to assure that a student is reading and succeeding at his/her level.

            As a second grade teacher at North Clay Elementary, I have had direct involvement with the Accelerated Reader program for two years. Students begin the Accelerated Reader program in second grade and continue through the eighth grade. As an internal investigator, I will survey second through fifth grade teachers for input regarding the program at the elementary level.  The Accelerated Reader program was first implemented at North Clay Elementary and Junior High in 1999. This program has been in place for eight years. Throughout this time period, several issues have been brought to the attention of administrators, teachers and parents. One issue, for example, involves teacher implementation of the program to fit the needs of the classroom. Some teachers give grades each quarter for student points earned. If a student does not meet the set goal in point value, he/she may get a lower grade. Other issues involve the validity of the STAR testing, student attitudes toward the program, and overall consistency throughout grade levels second through fifth. The goal of this evaluation is to gather information on the uses of the program and the opinions of the teachers using AR. Hopefully, some consistency among grade levels can be established in order to better fit the needs of the students at North Clay Elementary.

            What are teachers' observations of the use of Accelerated Reader to enhance student reading ability and desire? This is the driving question for the evaluation of the Accelerated Reader program within the North Clay Elementary Unit 25 school district grades second through fifth. According to the Programs and Practices Accelerated Reader/Reading Practices web document published by the Education Commission of the States (2000), the philosophy of the program expresses that the use of a computer-aided reading comprehension and management program can motivate students of all ages to read more and better books. In turn, students will become better readers, enabling students to do better in other subject areas. This evaluation will analyze this philosophy within the North Clay Elementary school. Teachers will respond to the effectiveness of the Accelerated Reader program and the various implementations of the program within second through fifth grade classrooms at North Clay Elementary.

 Evaluation Design

 Participants

            Participants for the Accelerated Reader program evaluation include second through fifth grade teachers employed with North Clay Elementary CUSD 25. Participants will be recruited through an informational email. Each grade level has three teachers. Ten teachers will be expected to participate with four as the minimum number of participants needed. This evaluation will hopefully include at least one participant from each grade level. Although, as a second grade teacher I would qualify for the investigation, I will not include myself as a participant.

Instrument      

            My evaluation instrument will attempt to answer the driving evaluation question - What are teachers' observations of the use of Accelerated Reader to enhance student reading ability and desire? The evaluation instrument will be a survey created with freeonlinesurveys.com. The online survey can be accessed at http://FreeOnlineSurveys.com/rendersurvey.asp?sid=u1n7ksxv3rmal9a337055. You also may refer to the attached survey questions. These survey questions were influenced by my experiences/concerns with the program, those mentioned by fellow teachers and those expressed by the North Clay Elementary Principal. The survey totals sixteen questions. Questions assess teacher feelings toward the program, everyday use of program features, and student attitudes. Questions are grouped by topic. First, each participant will respond to the extent in which Accelerated Reader is meeting its goals and reflect his/her opinions of the program (questions 1-5). These questions will provide general thoughts concerning the program. The second section includes questions regarding the STAR testing and its accuracy (questions 6-11). This section will provide information concerning the validity of the test in comparison to other forms of assessment used in the classroom. The third section asks questions regarding general use of Accelerated Reader in the classroom and teachers’ perceptions of how student respond to or feel about the program (questions 12-16). Information gathered from this section will help to establish general expectations for the program throughout second through fifth grades. It will also help determine the general student response to Accelerated Reader.

 Procedures

             Possible participants will be approached through an informational e-mail. This e-mail will introduce the study by providing the selected teachers with a brief overview of the evaluation. This overview will include a general description of the Accelerated Reader evaluation and the directions for the participation process. Please see the attached sample e-mail.  Teachers will be asked to print the consent form for his/her records and continue with the online survey if he/she agrees to participate. The consent form will include the link to the survey and the password for accessing the survey. Each participant will use the same password to access the survey to ensure confidentiality of results. This password will be made known only to survey participants through the e-mailed consent form.Through the e-mail/consent form, teachers will be directed to the online questionnaire to be completed between the dates of October 22 - October 31. In order to access the survey, teachers will need to enter the provided password. Upon entering the password, the questionnaire will ask those specific questions relating to teacher use of the Accelerated Reader program within his/her classroom (see attachment). As mentioned above, questionnaire results will remain anonymous and will serve as a guide for further program evaluation. The questionnaire will not provide a field to enter identification and participants will be asked to avoid using names while responding to questions. If a name or names are entered in the survey, the name(s) will be omitted when analyzing data and expressing results.

            The participant will complete this survey on his/her school or home computer. The survey will take approximately fifteen to twenty minutes to complete. Upon submission, the survey will be automatically e-mailed to my Regional Office of Education e-mail address (a feature of freeonlinesurveys.com), in which the results will be accessed, printed, deleted with the use of a school computer, and analyzed.

Evaluation Plan

Management Plan

Administrative Procedures

            The Accelerated Reader program is a learning technology used at North Clay Elementary to boost reading level and desire. After considering several learning technologies in place in my district, Accelerated Reader was chosen in result of its frequency of use and importance within the district’s reading program. The Accelerated Reader evaluation process began with input from the North Clay Elementary principal. She expressed many concerns with the program and in turn, assisted with the focus of the evaluation and the survey questions. The next step in the evaluation process involved completing and submitting the IRB form for approval from the University of Illinois. The IRB form included the focused evaluation question, the survey, and the procedures for conducting the evaluation. Upon IRB approval, I will able to continue with my recruiting of participants through an informational e-mail, directing participants to the online, password protected survey (created through freeonlinesurveys.com), the acquisition of results (automatically emailed to my account upon participant submission), and the analysis/reporting of the data.

Analysis

I have included several types of questions in my survey. Teachers are asked to respond to anchored, yes/no and open ended questions. I have included a variety of questions in hopes of allowing teachers to fully explain how they have established this program within their classroom. I have grouped questions within my survey that relate to one another. The first section asks teachers to explain if they think that AR is meeting its goals and to give an overall opinion of the program. I hope to use these results to get a general idea of what participants think.  The second section addresses the STAR testing. On this page of the survey, I will gather information regarding how often teachers allow students to take the STAR test and what times of the year. The third section asks questions as to what other types of reading assessments are present in the classroom and whether these assessments agree with the results of the STAR test. I am looking for comparisons here. How accurate is the STAR test when it is compared to other types of testing like running records? Do teachers use other assessments to determine reading level? Sections four and five ask specific questions relating to how the program is used in the classroom. I will look for consistencies in results from these two pages. For scaled questions, I will determine the mode and average of the data. For the open-ended questions, I will look for patterns, as well as similarities and differences in responses.

 Summary

I will report the results of my survey in a formal paper. This paper will detail teacher opinions of the AR program, the use and accuracy of STAR testing, and consistency of the use of the AR program throughout grade levels 2-5. The report will include an introduction to the Accelerated Reader evaluation, the driving question of the evaluation, an overview of the procedures, and the results of the evaluation including visual representation of data, conclusions, and recommendations for future use of the program. I will look at the similarities and differences in use of the program. Scaled data will be represented in graphs. Open-ended data will be summarized by looking for similarities and differences among participant answers. I will look for practices that could be adapted by all teachers in the program. Also, I will be looking for ways that consistency could be established throughout grade levels. My analysis will be shared with EPSY 474 class, the participating teachers of grades 2-5 and the North Clay administration.

References

Education Commission of the States. (2000). Programs and practices
            accelerated Reader/Reading practices. Retrieved September 22, 2007, from             http://www.ecs.org/clearinghouse/18/79/1879.htm

Survey Questions

1. On a scale of 1 to 4, how strongly do you agree or disagree that the Accelerated Reader program is meeting its goals in helping students succeed in reading and in becoming better readers?                           

                  4 (strongly agree)      3 (agree)        2 (disagree)   1 (strongly disagree)
 

2. Please explain: 

3. On a scale of 1 to 4, how strongly do you agree or disagree that the AR program is positively influencing student desire to read? 

                  4 (strongly agree)      3 (agree)        2 (disagree)   1 (strongly disagree) 

4. Please explain: 

5. What is your overall opinion of the Accelerated Reader program?

6. On a scale of 1 to 4, how strongly do you agree or disagree that the STAR test accurately expresses the reading level of your students? 

                  4 (strongly agree)      3 (agree)        2 (disagree)   1 (strongly disagree) 

7. Why do you feel this way? What evidence do you have to support your opinion? 

8. How often do you administer the STAR test during the school year?

      Once per year

      Twice per year

      Three times per year

      Four or more times per year 

9. What time of the year do you administer the STAR test? Please explain according to your answer above. For example, if you give the test three times per year, please give the approximate dates you administer the test. 

10. Besides the STAR test and AR program, how do you assess student reading level and comprehension in your classroom? 

11. Do these assessment results correlate with the STAR test results? 

                  yes                                          no 

12. How do you administer the AR program for reading level? Do you require every student to read at his/her reading level determined through the STAR testing? 

                        yes                                          no 

13. Do you let your students read books below and/or above their reading level?                

                  yes                                          no 

14. How do you administer the AR program for point values and goals? Do you take a grade? What kinds of incentives/goals are present? 

15. Does this encourage students’ attitudes toward reading?                 

                  yes                                          no 

16. On a scale of 1 to 4, how would you rate students’ attitudes toward the program? 

4 (love it)                    3 (like it)                     2 (dislike it)                1 (strongly dislike it)