EdPsy 490ASA, Fall 2002

Final Project

Kim Baker

Dr. Sandy Levin, Instructor

Proposal

My proposal called for learning more about how portfolios could be used in science assessment. My goal was to use more complex assessments that require higher-level thinking from students. When I began this project, my understanding was that portfolios are more authentic, real world assessments and that they offer opportunities for students to show higher level thinking skills. I wanted to avoid setting up portfolios just to display student work. Rather, I planned to research portfolio assessment in science and learn how to use these assessments in place of some of the fixed choice types of tests I’ve used in the past. For the past 6 years I have relied almost solely on matching, multiple choice, and true/false tests as a means of science assessment. As my goal is to require higher level thinking from my students, it became clear that my assessments needed to match this type of instruction. The tests I relied on were measuring mostly simple knowledge. I wanted students to not only recall facts, but to apply them using scientific reasoning. This required using more open-ended assessments such as short answer assessments. I also wanted students to have the flexibility to create and design experiments, as well as to explain their scientific knowledge. Portfolios appeared to be a more real-world means of assessment than the traditional tests I had been using. Portfolios appealed to me because they allow students to practice self-reflection. Fourth graders are just beginning to evaluate their own work, particularly in writing. The more practice they have at reflecting and evaluating their own work, the more responsibility students will take for their own learning. In addition, I wanted students’ learning to be on going, rather than completed once the unit test was over. I also wanted to be able to see students’ growth in scientific reasoning over the entire school year.

Analysis

Portfolios

The focus of my project began on portfolio assessments. I used the portfolio options from our new science textbook as a starting point for implementing portfolio assessment. I determined that for each large unit, Magnets & Electricity, for example, students would complete 4 portfolio options. During the unit, students completed all of the options. At the end of the unit, students chose which option they wanted me to grade. I created a portfolio rubric that included the rationale for why the student chose that option, as well as the student’s self-evaluation on the project. In addition, the rubric included a "teacher’s score". In the past, I would have simply skipped over the portfolio options in the textbook in order to save instructional time. The objectives that were evaluated through using portfolios would have been addressed in the multiple choice and true/false tests that I had used in the past. The portfolio options, however, gave the students an opportunity to apply their learning by creating, as well as to reflect and evaluate their own learning at the end of the unit.

Experiment format

As I proceeded in creating, revising, and testing my assessments, it became clear to me that additional changes were needed. Traditional tests, short answer questions, and portfolios alone did not provide me with insight into students’ thinking during science experiments. I felt that the process of scientific reasoning is the key to mastering science. Therefore, I needed to be able to assess students’ reasoning in order to better instruct them. This became the central focus of my project. In the past, I gave every student the same number of points for participating during science experiments. The disadvantage of this is that the kids who needed help learning how to reason and think like a scientist, never got the help they needed. I was unaware, based on my "freebie" points, of who needed help. To help with this, I changed the way students conducted experiments. Prior to this year, students all completed science experiments at the same time, in groups of 4. This meant I was trying to assess students’ scientific reasoning, managing supplies, trouble-shooting behaviors, and maintaining a quiet enough environment so that each group could concentrate, all at the same time. I was not able to spend long enough with each team to assess their participation nor to assess their reasoning during the experiments. I found that some other intermediate teachers conducted experiments with only 1 group at a time while the rest of the students worked silently on something independently. I opted to try this new experiment format. I then created a rubric for evaluating students during science experiments, which incorporated working together and following the scientific method. When I tested this rubric using the modified 1-group at a time approach, I found that both the rubric and the approach needed more modifying. The first problem was time. The science experiment took about 40 minutes longer than usual due to only one group completing it at a time. By the end of the 80 minutes, most students were fidgety. To remedy this time issue, I changed the format so that 2 groups (8 students) would be conducting the experiment at once. Then space became an issue. Since 8 students did not fit at any 1 table in the classroom, I created the "portable teacher", a bucket of science supplies needed for the experiment. The students loved being able to stay in their seats in groups while I was "portable" and came to their teams. Also, while I was more successful in evaluating students’ reasoning, I learned that I had too many categories in my original rubric (6 total). For the final rubric, I eliminated "support hypothesis" and "apply prior knowledge" as both of these were difficult to assess by observation alone. One advantage I noticed immediately in changing the format of the experiments is that students took them more seriously. Students listened better during directions, behaved better, relied on their teammates more, and followed experiment procedures more closely. I also began the experiments by reading the directions and answering questions so that my role during the experiments was only to observe. This freed me up to watch how students solved problems using one another for help. I also realized that while students took the experiments more seriously, they needed a better understanding of how they were being graded. I noticed that some groups focused on their productivity at the expense of all group members participating. This is a common problem when students are unaware of which is more important: group productivity or group participation. It became clear that the students needed a better understanding of the rubric I was using to grade them during experiments. To help with this, I modeled an experiment that we conducted earlier in the year on the overhead. I "acted" as a student in 3 scenarios: the student who received all 3s on the rubric, all 2s, and all 1s. Students then came up to the overhead and acted out one of the scores and had the class guess which score they received. A total of 12 points were possible. Another advantage to this format change is that now students had time to complete the portfolio options that I had skipped over in the past due to time constraints. They simply worked on their portfolio options for the previous chapter while waiting their turn for the experiment in the current chapter. For example, during the static electricity experiment, students completed the chapter one portfolio option from the Magnets Electricity portfolio options sheet while waiting for their turn for the experiment. The portfolio options also helped address my goal of assessing students’ scientific reasoning as the portfolio options required that students apply their knowledge by creating and designing. Also, students had to reflect on their own work and evaluate their portfolios at the end of each unit. As I tested the informal observation rubric further, I found that despite the time, space, and practicality improvements I made, it was still impossible for me to effectively evaluate every student during every experiment. To remedy this, I modified my approach further by assessing 3 teams (12 students) during each experiment. I will then alternate assessments, so that students who are not evaluated during experiment A will be evaluated during experiment B. However, I did not inform the students as to which experiment they’ll be evaluated on. Ii this way, they must be consistent in their procedures, cooperation, and scientific reasoning.

Self and group evaluations

Another successful revision that came about during this process was having students evaluate themselves and evaluate their group members following experiments. Now that students understood what it took to get all 12 points (from the overhead demonstration), they were able to determine how many points they thought they earned themselves, and how many points their teammates earned during the experiments. Each student wrote his or her name on the top of an index card. They then wrote the names of the other 3 team members from the experiment. Next to their own name, they wrote a score, and next to each team member’s name they wrote a score. I averaged all the scores and gave each student a science experiment score. I also recorded my score, using the same rubric in the grade book. Prior to this year, I had always given each student the same amount of points for experiments – simply for being there. While this did not assess any scientific processes, I was unsure of how else to get enough grades in science. The new self/group assessments along with my own assessment during science experiments (all using the same rubric) helped solve this problem.

Performance Assessments

In addition to revising the experiment format, creating the portfolio options, creating the group and self-evaluations, and creating the teacher observation rubric, I made several other revisions to my science assessments. One key change was in using performance assessments. In the past, I have used 1 performance assessment per unit. In the electricity unit, for example, I used a performance assessment which required students to create a closed circuit using a light bulb, batteries, and some wire. Students completed the assessment in groups and all group members received the same grade. I did not use a rubric for the grading, I simply gave the group 10 points if they quickly and successfully made the circuit work, and so on. I was unhappy with this random grading. I did not feel it was fair for all group members to receive the same grade. But due to time constraints, I was unclear with how to conduct performance assessments and how to assess students more fairly. I took several steps towards revising my performance assessments. First, I added an additional performance assessment, mystery powders. This was taken from the new science textbook and to be conducted at the end of the matter unit. Since this was the first performance assessment that I used this year, I made little revisions to the existing format. Students worked in their teams of 4, one group at a time (as the supplies took up so much space), while I observed and assessed them using the original rubric. I graded each student individually. During the performance assessment, it became clear that several more revisions were needed to successfully evaluate students. First, the directions were somewhat unclear. I re-wrote them on large chart paper, which I hung up behind the experiment table. Also, the experiment took nearly 2 weeks due to the time required for each group to complete it. Since my role was to observe and assess, the rest of the class had to be working on something silently. I used silent reading time for this. To remedy the time problem in the future, I plan to implement the "portable teacher" format and have 2 groups working on the assessment at one time. I also plan to have students self-evaluate and group-evaluate using the index card format mentioned above. I revised the rubric as well to fit our classroom needs. For example, in the original rubric, there was a category called "written summary". I could not find directions anywhere asking students to write a written summary of the experiment. In order to help save time, I added a column to the student data chart titled, "conclusion" where students wrote the powder name next to the data. I also noticed during the experiment that many students had difficulty organizing themselves to follow the experiment procedure. For instance, many students jumped from powder to powder and forgot which powder reacted with which substance. Therefore, I added this component to the "performing the tests" section. I also noticed that several students correctly identified the powders verbally but wrote them down incorrectly on their charts. I added this component to the "completing the chart and identifying powders" section. Another change I made from past performance assessments is that I graded students individually. I also posted a copy of the revised rubric in the experiment area so that students would know how they were graded. A final change I made to administering the performance assessments involves decision making. In the past students had difficulty sharing supplies and performing the assessments fairly. For example, many times one or two students would take over and the other team members were unable to try the assessment themselves. Another issue occurred when students tried to voice their frustration and arguments and fighting ensued. This resulted in losing sight of the task at hand. In order to help reduce this, I gave the class 5 minutes to make decisions within their team about who would do what tasks in the performance assessment. For example, one team decided that every team member would test all the powders. Most teams opted to split the powders so that 1-2 students worked on one powder at a time. The class has been working on decision making and cooperative learning skills all year long. This performance assessment was an excellent opportunity to practice these skills. The only other change I would make in the future for this performance assessment is adding a category to the rubric called, "working fairly within group". As working with your teammates is a necessity for successfully completing the assessment, it should be evaluated along with the scientific results.

In addition to adding the mystery powders performance assessment to my curriculum, I revised the electricity performance assessment that I used in the past. The first change that I made was that I graded students individually using the new rubric I created. In the past, students were given the same grade as their group members. I also had the students work in pairs (versus teams of 4) so that I could assess their performance more clearly. In testing other assessments this year, I learned that in the past I relied too heavily on students’ auditory processing for directions. I rarely wrote directions for experiments out. This meant that much of my time was spent answering the same questions repeatedly. To remedy this, I wrote out all directions either on the students’ sheet or on large chart paper. I also read the directions and answered questions ahead of time so that my role during experiments and performance assessments was purely to observe students. In addition, I revised the electricity performance assessment to better fit our new science curriculum. In the past, students merely completed a circuit. The revised version of this electricity performance assessment requires students to complete a series circuit and a parallel circuit.

Multiple Choice Tests/Essays

Since I have relied so heavily on multiple choice tests in the past, I chose to revise just 1 multiple choice test for my project. For my original electricity test, I used the textbook "pressed for time" multiple choice test. I revised the test by revising the essay portion of the test. On the revised test, I included specific directions and a scoring rubric. I also indicated how long students should spend on each essay question. When I tested this exam, I found that students were more likely than in past years to score well on the essay portion. I believe this is because I improved the essays by making my expectations clear. Through using words like, "I’ll be looking for" and including the scoring rubric, I increased student’s chances to focus on my goal: their ability to design an experiment and apply their knowledge about how electrons move. I also learned that students need additional instruction in how to design experiments. As one of my goals for this project was to assess scientific reasoning, this small change helped me to do this. In the past, due to the unclear directions and wording, it was difficult to discriminate between student’s inability to express their understanding and their scientific knowledge. The changes that my group helped me make were essential in effectively evaluating each piece separately.

 

Conclusion

According to my original KWL, I wanted to learn how to assign letter grades to portfolio assessments. I learned that rubrics are designed based on my goals for the unit and my expectations. I also wanted to learn the best ways to implement such assessments to incorporate higher level thinking skills. I found that portfolio options require students to apply knowledge in the form of a conclusion or designing an experiment. I also learned that by implementing changes to my assessments slowly, I learned more about what I needed to improve on to better assess students. For example, the first change I made was that I began writing down all directions. I revised my test directions, and clearly wrote out directions for experiments and portfolios. I learned that written directions and visuals help reduce poor completion due to limited reading abilities or misunderstanding directions. In the past I have relied too heavily on student’s auditory processing. This not only made the assessments somewhat unfair, but I was stressed due to the excessive questions students asked during activities. The second change I implemented was breaking down the experiments to 1 group at a time. Due to time constraints, I eventually changed this to 2 groups at a time and this format change impacted my ability to better evaluate students during experiments. Third, students began to self-evaluate and to evaluate their peers during experiments. I also evaluated them using the same rubric. This process of practicing self-evaluation is valuable, and I plan to use it in other content areas. The fourth changed I made was that students began working on portfolio options while waiting their turn for experiments. I learned that these options do not need to be excessive nor time consuming. Yet I am able to assess students’ ability to create and design experiments, and apply scientific knowledge and scientific procedure in using the portfolios. I also modified 2 multiple choice tests. I learned that in the past I relied solely on these type of tests for science assessments. Another issue that I wanted to learn about was how to use portfolios better than simply as a collection of student work. I learned that the reflection process for students is the key. This also helps the learning be continuous, rather than having a finality once the unit is over. I wanted to learn how I could fit portfolios into our existing science curriculum. I found this simple, as many of the portfolio options were listed right in the teacher’s guide. I then created a rubric based on the goals for that unit and my expectations.

The process of evaluating, revising, and creating my science assessments carried over into other subject areas throughout this project. For example, I began writing directions down for every activity. Also, I began to create the rubrics for student projects while I wrote out the directions them. I also began assessing students in a more well-rounded manner in all of the subject areas. For example, students completed an expository on the 3 branches of government in social studies. In the past, I would have assessed students’ government knowledge purely on a multiple-choice test. I used the 6-point rubric provided by the state to grade their expositories. Following a study of economics, students created non-fiction money books. I created a rubric and modeled it on the overhead the same day that I assigned the project. During the electricity unit, students read about electrical safety. Instead of using a multiple-choice test as I might have in the past, I asked students to create safety brochures to share with their first grade buddies. I wrote out the directions for the brochure on the rubric, which I passed out to each student when I assigned the project. In this way, students knew what I was looking for, and they had a real audience – first graders who needed to know more about electrical safety. Another assessment that came about during this project was using extended response questions during our novel study. Students must complete these extended response questions on the ISAT test. However, in the past, I was unaware of how to model extended responses. I found the rubric provided by the state and used it to write an extended response question for the novel we were reading in class, Owls in the Family. I then modeled the extended response on the overhead. Then students wrote one of their own and assessed their own, using the same rubric. My plan is that as we write more and more of these types of responses, I’ll assess one per grading period. Overall, the assessment changes I have made are on-going. As a result of many of the changes I’ve implemented, I am now creating my assessments prior to teaching a unit. In this way, I’ve found myself sticking to my objectives and the state standards more clearly. I’ve also found myself trying new projects and taking risks. I believe this is because I am more aware of how many assessment options there are. As my goals for instruction have moved towards expecting students to do more thinking, it is only logical that my assessments require students to do the same kind of thinking. This process will surely be continuous throughout my teaching career.