Computer Literacy ~ Assessing Assessments
EdPsy590ASA ~ Alternative Student Assessment
Instructor ~ Dr. Katherine Ryan
KRISTY SANDS

 

KWL
Curriculum Map
Assessments

 

Course Description:
Students will conduct an in-depth analysis of current issues and trends using alternative methods of student assessment. We will focus on the assessment and process of measuring complex achievement using various written forms of assess ments, performance- or project-based assessments, portfolios, and other assessment procedures that teachers can use as students engage in their work.

 

Abstract:
This project focuses on using assessments to make informed and meaningful changes in my Computer Literacy curriculum. After evaluating the assessment practices that I use in my middle school Computer Literacy courses I realized that I tend to overuse the use of rubrics instead of using a variety of evaluation tools. In addition to overuse, the rubrics that I used were boring, lacked detail, left too much uncovered territory, and left too much room for misinterpretation. I revised my assessments and evaluation tools by re-evaluating the diversity of students needs, the course learning objectives, and embracing the principles of tests and measurements in order to enhance my curriculum, student achievement, and to assess student learning more effectively. After piloting several of the revised assessments the results were incredible. The students submitted better products, asked questions, and provided feedback to one another. They became experts at evaluating themselves and others and were open to suggestions. Evaluating their learning outcomes was much easier. The results were clear and concise not just to myself, but for the students as well!

 

Project Analysis:
By assessing my own assessments, I recognized many flaws that prevented me from getting an accurate measure of student learning. I learned that spending time to evaluate your curricular goals and objectives reinforces the bond between the standards, goals and objectives, and assessment practices which results not only in solid measurement tools and valid and reliable results. It also amplifies the gaps and weaknesses that need to be filled and strengthened.

Through completing this project it was also brought to my attention that the tools of measurement such as pre-assessments, observations, peer feedback, reflections whether formal or informal are just as valuable as rubrics, checklists, and rating scales in my classroom. Working in a class that promotes the “hands-on, minds-on” philosophy, I strive to make every learning experience authentic and meaningful to all of my students. Therefore I create performance assessments that compliment this philosophy of education. I recognize that the content and skills that I teach cannot be measured adequately by objective assessments.

Fortunately a significant amount of time has already been spent creating, revising, and editing my Computer Literacy curriculum and as a result my learning objectives and assessment tools have been modified regularly. I also recognize it as a work in progress and it can be altered at any time in order to keep it current with our technology savvy society and to prepare my students with the most current information for high school and beyond.

Many of the gaps and weaknesses that I identified multiplied as I became more knowledgeable of appropriate and effective assessment practices. However this disturbing discovery only motivated me to tackle the “don’ts” to turn them into “do’s.” Offering clear performance expectations was the biggest “don’t” that needed to be tackled. So I selected different types of scoring tools such as checklists and rating scales that better clarified the tasks, rating criteria, and learning outcomes. Richard Stiggins embraces this theory by asserting, “If you do not have a clear sense of key dimensions of sound performance – a vision of poor and outstanding performance – you can neither teach students to perform nor evaluate their performance” (pg. 270).

The revisions that I made to the rubrics often involved adding a self-evaluation section to encourage students to be accountable for their own learning and to get a better grasp on how well the students felt they performed. After piloting the assessments that incorporated the self-evaluation section, the students often had a good indication about their strengths and weaknesses. As a result I started offering Q&A sessions on instruction days before the assessments so that students could ask any questions about the concepts they needed more information about. In addition I had the students rely on one another to peer-evaluate sometimes during instruction and other times after the assessments were already completed. This technique often gave students an indication as to what areas they needed to work harder on.

Based on feedback that I received from my own reflections, colleagues, and students I realized that the rubrics I use need some work. One colleague commented, “A suggestion for the future might be to add some scales to your rubric, so you have to put less effort into identifying why this section got a 2, while this section got a 4. You have a good break down of the assignment points; you just might continue to provide a greater breakdown and what those scores mean in a particular section. That will make the self evaluations efforts easier for students which are probably novices to this type of self-reflection.”

Defining clear characteristics expected and the allotment of points is critical to ensuring the reliability of the results. It sounds so easy, but I have struggled with creating this type of tool. I would like to get more information and resources on creating analytic and holistic rubrics. They are very time consuming to create, but make the grading process easier and faster. However they can be confusing to the students, which then makes the objectives and expectations for the task unclear.

According to Linn and Gronlund, “There are (1) pretests at the beginning of a course or unit to determine learning readiness, to aid in instructional planning, and to make advanced placements; (2) tests and assessments during instruction to improve and direct student learning and to identify and remedy learning errors; and (3) end-of-instruction tests and assessments used at the end of a course or unit to assign grades, certify accomplishments, or evaluate teaching” (pg. 166).

With the diverse population of students that I teach and the restricted time frame that I work around, I often overlooked the first two types of assessments. Instead I merely broke up the unit so that the content would not be too overwhelming for the students and assessed the students at the end of instruction. I added a few assessments in order to meet the learning needs of my students, plan and prepare for instruction, and to continually evaluate their progress.

To address the problem with the time restrictions that I encounter I integrated some of the units and I altered some of the assessments so that they would not take as long. These decisions were based on the results the pre-assessments and mid-unit assessments that were added. For one particular assessment I allowed the students to work with a partner because in a real or authentic setting students will often correspond with others when using computers and technology. This also helped limit the time needed to complete the task.

In addition I also initiated an informal observation technique where I selected three to five students from each class each week to observe. This technique allows me to get an adequate idea of how all of my students are performing over time alleviating the burden of observing the progress of thirty students every class period. This also allows me to conference with the students, get to know them better, and identify their individual needs and progress.

After piloting the assessments, the results have made an incredible impact on my teaching. It is easier to identify the areas of strengths and weaknesses in my instruction allowing me to know exactly what needs to be re-taught if needed. The students and I can easily interpret the learning outcomes. Upon the results of their reflections the students can usually decipher exactly the areas they did not master and what they can do in order to ensure mastery of all performance areas.

My students liked the variety of measurement tools that were used. They became experts at providing peer feedback, which was something that I was uncertain if middle school students could achieve. Often they would ask for feedback from one another before submitting their assignments even if it was not required. At the end of the trimester I ask for feedback from the students about the course, but this trimester I asked for input after each unit so that I could make needed revisions. The message that I received most often was that they liked that the assessments didn’t seem like tests and it wasn’t as stressful for them. They also commented that the course content was valuable for other classes and personal use. Dr. Katherine Ryan also commented that, “It is very difficult to distinguish between your assessment activities and instructional activities!”

Of course not all of the students liked the project-based approach and would rather have the traditional paper-and-pencil tests. These students commented that they did not like to work on an assessment over several days that they would rather receive immediate results. Some students felt pressure to meet the deadlines. So I will head back to the drawing board to see where I can incorporate paper-and-pencil tests, which will also fill the gap of my lack of use of multiple choice items. I need to further review the need for more time; perhaps there other issues influencing the need for extended time.

Self-reflections and evaluations have been critical to the success of this project and to the success of attaining the learning outcomes. Getting direct input from the students amplifies the results obtained through other methods and grants the students with the opportunity to recognize how the skills and knowledge achieved can be used in other contexts.

“Student participation need not be limited to the use of assessment instruments. It is also useful to have students help develop the instruments,” suggests Linn and Gronlund. I took this advice in combination with the application of interest surveys. The students enjoyed that they contributed to the creation of the assessments, which gave them an increased sense of motivation, confidence, and pride in their work. This is a wonderful procedure to advocate a learning community within the classroom!

Most importantly this project allowed me to recognize the value of continual assessment and revisions of my own knowledge and practices. Although I have made a tremendous improvement in my classroom teaching as a result of this project, there is always room for more. New information is always emerging and we must be open to any ideas that will help to educate our students regardless of the obstacles that may litter our paths.

 

Questions:
  • How can I create analytic or holistic scoring rubrics that are not too confusing or overwhelming for my students? Any tips, suggestions, strategies, resources, etc.

I used an analytic scoring rubric for one of my projects and it was the only assessment that I did not make any revisions to. It seems to contain everything that Linn and Gronlund suggest, however when I use it with my classes it takes a lot of time to explain how it used and what it means. Perhaps with more exposure the students would get the hang of it.

 

Resources:
  • Linn, R. and Gronlund, N. Measurement and Assessment in Teaching. Prentice-Hall, 8th Edition. 2000.
  • EPSY 590 ASA (Alternative Student Assessment) Course Web site. Spring 2005.

 

Last updated: 04.05