Final Stage 4

 Abstract:

 The original purposes for this project, as stated in Stage One, were 1). Design a unit and corresponding assessments in which depth of understanding is the goal, and 2).  Create authentic assessments for this unit that promote maximum performance from students.  Using the concept of Understanding by Design, UBD, stressed by Hilliard City School District, HCSD, the strategy was to 1). define the learning objectives to be mastered by the students, 2). write assessment items designed to assess those objectives, 3), then create the lessons designed to achieve those goals.  The units to be rewritten were for the 8th grade Algebra 1 class: “Systems of Equations and Inequalities” and “Exponents and Exponential Functions”.  Stage 2 of this project outlines the learning objectives.  Stage 3 of this project contains the original assessments and the new assessments which more closely match the leaning objectives.  The assessments previously used for these two units contained a small representative sample of what the students were to have learned and questions in which student expectations were not well defined.  The assessments for both units have been rewritten to contain a larger representative sample of the learning goals, the student expectations and scoring criteria was clearly written for each problem.  To improve scoring reliability, a scoring guide was created which included rubrics for extended response questions.

Analysis and Summary: 

Analyzing the curriculum learning goals and the State of Ohio Academic Content standards, as written in Stage Two, confirmed that the original assessment were inadequate.  Using the stated learning goals to write the assessments made the process more meaningful as the assessment items were directly connected to the learning goals.  I had not planned to write the scoring guide to accompany the assessment, but as I reflected back to the grading process, the reliability of the scores, and the validity of the scores, the scoring guide seemed like a logical approach. 

 The new chapter assessments did contain a higher number of test items to be completed.  To keep each test from being a multi-day event, and ensure test security so that the students would not discuss uncompleted test items outside of class, I divided the chapter 7 unit into two parts: “Systems of Equations” and “Systems of Inequalities”.  The chapter 8 test was also divided into two parts: “Simplifying expressions with Exponents”, and “Exponential Functions”.  The two-part unit, with a test over each part, made the planning easier for me and made the test more manageable for the students.  Each test was easily completed in one period.

 To analyze student performance on the assessment, I compared the final class average for each unit from the 2005-06 school year to the current 2006-07 school year.  Only the final averages were available, and Item analysis was not done for the 2005-06 class. 

Class Year 2005-06 2006-07
Chapter 7 Unit Test 82.4% 82.85%
Chapter 8 Unit Test 80.75% 81.67%

The 2006-07 chapter 7 unit test scores were nearly the same as 2005-06 class.  The test items used for the 2006-07 test were comparable to the 2005-06 test except for three questions in which the students had to describe the relationship between a pair of lines and the solution of the system.

The 2006-07 chapter 8 unit test scores were only slightly higher than the 2005-06 class.  This may seem as if student achievement did not improve by a significant difference, but the test items for the 2006-07 class were markedly different than the 2005-06 test.  The 2005-06 test items were skill and knowledge based questions.  Upon revision, questions designed to assess conceptual understanding were added.  Students in 2006-07 were required to explain the cause of different shapes of exponential functions.  Though the test scores were only slightly better, I hope that the conceptual understanding was much better for the 2006-07 class.  For both of the units described, it was not only the test which was revised.  The instruction was also revised in an effort to improve conceptual understanding. 

 For both unit tests, detailed response expectations were written into the instructions and point values were printed on each test item.  In verbal discussions with students, it was revealed that the detailed response expectations with point values printed on the test was helpful.  Most students provided detailed responses showing clear understanding of extended response questions,  indicating that clearly stated questions with detailed expectations may be one key to getting maximum performance from students on written assessments.

 I would conclude that students will strive to meet the communicated expectations of the instructor.  If the instructor asks only skill and knowledge questions, students will be content to practice skills and memorize facts.  If the instructor requires students to apply knowledge, explain deeper meanings, and demonstrate understanding, then students will rise to the challenge.  They will begin to expect better instruction and more complete explanations from the teacher, and they will begin to demonstrate understanding by applying concepts to new problem situations.

 What will I do differently as a result of what I have learned from this project?

 Before this project, I had not routinely use a pre-test at the beginning of a unit.  I did experiment with using a pre-test at the beginning of the “Exponents and Exponential Functions” unit.  As the students progressed through the unit, they showed a curiosity for finding out how to simplify some of the expressions with exponents that they missed on the pre-test.  This led to more questions about understanding the concepts and more lively class discussions.  I would like to make a pre-test part of the routine procedure for beginning each unit. 

 I will keep records from year to year, with sample tests from former students.  This will help me track changes made and determine if the changes have made a positive difference in student achievement.

 Teacher expectations are the key to getting students to provide desired responses.  I will strive to clearly communicate expectations during instruction and on all future assessments.

 
Questions for feedback: 

 1.  The most difficult part of the project for me was determining how I could fit a larger sample of test items, including some written response items into an assessment that could be finished in one class period.  I did divide the chapter 7 unit and the chapter 8 unit into two parts.  A separate test was given over each part to eliminate the potential for a multi-day test where students could discuss it out side of class.  Do any of the other EPSY590ASA students ever give assessments that unexpectedly last longer than a single class period?  If so, how do you handle it?

 2.  I found the scoring guide for the test to be very helpful, and it actually made the scoring go much faster.  It has taken me 18 years to discover this.  Is a test scoring guide commonly used by other teachers? 


Revised KWL CHART

KWL

What I Know

What I Want to learn.

What I have Learned

I currently do not use a pre-test as a general practice when beginning a new unit.

·    Would a pre-test change the way I approach instruction to make it more effective and benefit my students?

A pre-test used for the unit, “Exponents and Exponential functions”, did not change the way I approached instruction, but it did change the way students reacted to the lessons.  They seemed more curious to find the solutions to questions that they did not know how to answer on the pretest.  There was a benefit to the pre-test, but not the benefit that I expected.

I usually give one or two smaller quizzes as the class progresses through a unit. The smaller quizzes are more formative, helping me to identify areas that need clarification.  A chapter test or unit test is given at the end of each unit.  It is more summative.

·    Is what I am doing, small formative assessments followed by a summative assessment, the most effective method for improving learning and instruction?

In-class experiments with formative assessments have revealed that the formative assessments are effective.  Misconceptions are clarified when analyzing the formative assessments, and as a result, students perform better on the summative assessment.

I spend a lot of time grading tests, looking at each student’s work for each problem.  Multiple choice tests would be easier to grade. Hence, using multiple-choice tests would save me a lot of time.

·    Can multiple-choice tests be effective assessments of student skill, knowledge, understanding, and application?

Multiple choice items can be effective in assessing knowledge, but student-supplied answer questions are best for assessing skill and understanding.  A single assessment should contain some of each type of question.

The “authentic assessments” which were developed by a committee of math teachers appear to me to be simply multi-step word problems.

·    What is an authentic assessment, and what does an authentic assessment of mathematical concepts look like? 

·    Are authentic assessments effective ways to measure student learning? 

·    Are authentic assessments helpful in preparing students to take the type of achievement tests required by the NCLB Act?

Authentic assessments those in which students perform a real task.  Authentic performance assessment can not easily be written into a paper/pencil test unless it is closely tied to some real life goal.  Application problems, “word problems”, are the closest thing to a authentic assessment that can be written into a traditional test. 

Authentic assessment tasks can be effective in measuring student learning if the task is well defined and closely tied to the learning goals.  Authentic assessment can also be a useful learning tool, but just as any other formative assessment, analysis would be the key to achieving this goal.

Some of my 8th grade math students have the preconceived idea that they are not good at math.  This idea causes some students to give up or just be less motivated.  I feel that I am not getting the maximum performance from my students during lessons or during assessments.

·    How can I motivate students to put forth their best effort on the lessons and on the assessments?

·    Is there a way of designing assessments that could help to motivate a student to do their best?

Well defined expectations for any student performance, including test items, is important.  When designing a test to measure maximum student achievement, responses demonstrating that achievement must be defined.  Students may not produce quality work when the expected quality is not defined.
 
LINKS
Lee's Portfolio
Project Main Page
Stage One
Stage Two
Stage Three