EdPsych 389 Evaluation Proposal
Evaluation
of Mrs. Motsinger's Science Site
Community Consolidated School District 89 (CCSD89) is in its 2nd year of piloting a new science curriculum. This year all 5th grades are included in the pilot. An on-line science site was developed by a teacher to be incorporated as part of the instruction of the first unit covering Geology. http://www.ccsd89.org/teachers/pv/kmotsinger/science/home.htm
Students are required to complete 3 different curricular activities of their choice on 3 different difficulty levels. The levels and activities are set up following Bloom's Taxonomy of Higher Level Thinking Skills. The idea behind this site was to get students to actively participate in Geology on a variety of ability and interest levels while using technology as the tool for acquiring and demonstrating the knowledge. The site was used for the first time for four consecutive weeks in September and October of this year.
The purpose of this management/goals oriented formative evaluation will be to provide information on if the site is meeting the objectives it was designed for; (providing another means of presenting geology and independent learning to enhance classroom material presentation) and if it is, how to improve/enhance the site as well as build onto the site for use with other science themes.
The client is Joan Huchthausen, Superintendent of Curriculum and Instruction for CCSD89. Key stakeholders of this evaluation are the 5th grade Science teachers of CCSD89 as they will have the opportunity to include this site in their curriculum instruction. Secondary stakeholders are 5th graders, who utilize Mrs. Motsinger's Science Site, and non-Science teachers, who may oversee students using Mrs. Motsinger's Science Site.
Is the site functional?
Is there ease of use for teachers instructing students how to use the site? In addition, do students understand posted directions that define what they are supposed to do while at the site?
CRITERIA: % of time students ask
for clarification about the site.
For instance, "How do I get from this page to the worksheet?" or
"What
am I supposed to do for this activity?"
This pertains to assistance needed after all directions for a
particular
activity have been read completely by the students.
STANDARD: 20% or less of total time spent in the computer lab will be spent on clarifying instructions/directions.
Does use of the site increase
student understanding of the topic?
What facilitates
student understanding? What
inhibits student understanding?
CRITERIA:
% of students performing at 70% or higher on end of unit
test.
STANDARD: 80% of students will achieve a 70% or higher on the end of unit test.
Should, or how can, this site be used with future science topics? With what modifications?
CRITERIA: Meeting of the standards of the
previous 2 questions. A satisfaction survey of students
will be administered. Focus groups with
stakeholder teachers regarding the site
will be conducted.
STANDARD: Both standards for the previous 2 questions will be met. Survey results will prove 80% or higher satisfaction with the site. Teachers will voice assent for continuation of site use.
Methods
Subjects:
Two 5th grade non-Science teachers from Park View School in CCSD89 who oversaw student usage of Mrs. Motsinger's Science Site will participate in a focus group determining the ease of use of the site for students and teachers unfamiliar with the content of the site.
Three 5th grade Science teachers from CCSD89 will participate in an interview to determine the appropriateness of the activities found on Mrs. Motsinger's Science Site pertaining to Geology
All 5th grade students who utilized the Science Site for Geology (approximately 75 students) will be given the Student Satisfaction Survey to complete. These same students will take end of unit Geology test as part of their curriculum.
Instrumentation:
Questions for the
focus
group and interview, the
Student Satisfaction Survey, and a copy of the Geology
Science test are linked, as are consent
letters.
Procedures
for Data Collection:
5th grade non-Science teachers will meet with the investigator for 30 minutes on Tuesday, October 21, 2003 from 10:00 a.m. to 10:30 a.m. in Park View Elementary School's teachers' lounge. Questions during the focus group will center on what allowed and prevented students to work independently while using Mrs. Motsinger's Science Site, as well as suggestions for improving student autonomy. The investigator will record responses during the discussion.
5th grade Science teachers will meet individually with the investigator for 30 minutes during the weeks of October 27 and November 3, 2003. Interviews will take place on-line at a Tapped In office space owned by the investigator or via telephone. Questions during the interview will center on the appropriateness of the Geology activities found on Mrs. Motsinger's Science Site, as well as suggestions for improving the site in regards to the content area of Geology and other science topics. The investigator will record responses during the discussions.
5th grade students who utilized Mrs. Motsinger's Science Site for Geology (approximately 75 students) will be given the Student Satisfaction Survey to complete on October 17, 2003. The Science teacher will administer the surveys, as they will be attached to the culminating unit test on Geology. Surveys will be removed prior to turning in the test and collected in a separate location to maintain anonymity. Those completing the survey will receive a Jolly Rancher when the survey is handed in.
Student scores from the end of unit Geology test, taken October 17, 2003, will be collected by October 20, 2003.
Analysis:
A reflective analysis will be performed on the responses given during the teacher focus group and interviews. Patterns among this data and results from the Student Satisfaction Survey will be looked for. Comments will be used in recommending modifications for improvement
Member checks will also be used with a summary of the focus group and interview responses. Focus group members and interviewees will be asked to view summaries of their encounters with the investigator to verify that what has been written accurately reflects what was stated during the conversation.
Questions 1-5 on the Student Satisfaction Survey are Likert-scale items, asking students to rate their opinion of certain statements pertaining to Mrs. Motsinger's Science Site. This data will be analyzed on an item-by-item basis with percentages for each possible opinion choice. A score of 4 and 5 will be considered high satisfaction (to meet the standard listed for the 3rd evaluation question.)
A content analysis will be conducted for answers to questions 6-8 (open-ended). Percentages of respondents making each type of response will be calculated. These will be used in recommending modifications for improvement
Scores from the end of unit Geology test will be calculated into a percentage of students earning a grade of 70% or higher.
Timeline:
September 17 - October 15
5th grade students will have access to Mrs. Motsinger's Science Site 2 hours a week during supervised computer lab time. (1/2 an hour with Science class and 1/2 and hour with homeroom computer lab time per week)
October 15, 2003
Parent consent letter sent home in regards to the Student Satisfaction Survey.
October 17, 2003
5th grade students take end of unit Geology test.
5th grade students complete Student Satisfaction Survey
October 20, 2003
Scores from Geology test collected
Consent letters sent to non-Science and Science teachers in regards to focus group
and interviews
October 21, 2003
Non-Science teacher focus group meets, 10:00 - 10:30 a.m.
Weeks of October 27 and November 3, 2003
Science teacher interviews (specific dates TBD)
November 7, 2003
All data collected
November 10, 2003
All analysis completed
Week of November 17, 2003
Presentation to client and stakeholders
Reporting Plan
During the week of November 17, 2003, a meeting of the Science Committee for CCSD89 will meet at the Board Office. During this meeting a presentation of the results of this evaluation will be presented to the client and key stakeholders. A written summary or data results and suggestions for site improvements and usage will be distributed to coincide with a brief oral presentation.
Evaluation results will be reported to secondary stakeholders (students and non-Science teachers) informally during the week of November 17, 2003.
Budget
|
CATEGORY |
DESCRIPTION |
SUB TOTALS |
|
Salary of
Evaluator |
25 days @ $200/day |
$5000 |
|
|
|
|
|
Communication |
Local Phone (approx. $1 day) |
$25 |
|
|
Internet Connection (estimated) |
$30 |
|
|
Report Publication |
$30 |
|
|
|
|
|
Instrumentation |
Survey materials/copying |
$15 |
|
|
Student incentive |
$5 |
|
|
TOTAL |
$6105 |