Project Evaluation: Student Economic Forecasting
Survey
Regional Economic Review-Building an economic index for
the rest of us!
Mark Emmons, UIUC EPSY590 , August 3, 2006
Contents
Learning Task
Survey
Organization
Participants
Timing/LifeCycle
Results to date
Macro-analysis
Micro-analysis
Technical Difficulties
Engagement
Overview
Often statistics compiled and maintained by various authorities
fail to match the experience of individuals. This is particularly true when
it comes to economic data. Broad measures of the economy, by their very nature,
often lose meaning as they fail to accurately reflect local, regional or even
national outcomes. We frequently ask, “What reality is the authority measuring?” Further,
input into broad economic measures seldom take into account the perspective
of students—this is the student’s chance to have a “say.”
Learning Task
The goal is to generate responses to a survey so that
the results may be used in class as a medium for discussion, analysis and evaluation
of economic trends. The survey asks students to reflect
on the economic well-being in their area—specifically,
to respond to questions regarding their experience with employment, prices,
mobility and immediate economic concerns.
Students are asked to complete the survey based on their knowledge and analysis
of their local economy. As the data is compiled, a snapshot of the economy
will be consolidated that can be used by the student/class to:
The data is summarized by geographic location (states) based on responses received. The data compiled will provide the student with a benchmark to compare their results with other participant experiences;
Further, optional activities will allow the student to evaluate the reliability
and validity of the survey instrument and research additional information resources
relating to economic conditions in their region.
Survey Instrument
The instrument contains 11 questions related to the economy and mobility. The
questions allow a single response and require the participant to choose from
a scale of responses or respond with a simple yes/no. The questions asked
are:
Three additional questions are used for profiling purposes
The survey does not ask for any personal information, although it does provide
for additional comments at completion.
Organization
The enabling structure of the effort is networked collaboration.
Students can participate in the survey from anywhere with an internet connection.
The survey and consolidated results are posted to a centralized web site located
at: http://students.ed.uiuc.edu/memmons2/AboutSurvey.html
Participants
The survey offers students a voice and an additional
tool-their own results, for comparison to data generated by the major governmental
entities. It is targeted for completion by students in grades 9-12. The curriculum
focus is certainly economics, but participation by other disciplines in the
social sciences and business are encouraged to use it.
How the information compiled is up to the participant and participating classroom,
although suggested activities are available. Analyzing the data to "see" differences
and similarities is a good challenge. Students may define “what do we
ask of the data?”
Timing/Life Cycle
The effort is open-ended. There are no time constraints for completion, rather,
data is available real-time and offers the participant the ability to view
change in responses over time.
Results to date
The survey was available on July 20, 2006. Initially,
the project was posted to GlobalSchoolNet in order to promote and advertise
the availability of the Network Learning Project. Timing played a factor in
generating the necessary interest to collaborate on the project—with
school out of session, response to the promotion on the GlobalSchool Network
was negligible.
In order to generate sufficient responses to the survey and gauge the functionality
and efficacy of the effort, I drafted my 15 year-old son to promote the application
to his friends’ network. To that end, as of August 2, 2006, the site
has received 19 responses.
Because the data was artificially obtained (outside of a directed classroom
effort), the results are skewed. All participants are from Illinois and the
majority of respondents are 15 years of age. The
majority of responses reflect the attitudes of of adolescents who have
yet to enter the work force.
Evaluation of the Project
Macro-analysis
Organization and development of the project as a networked
collaborative problem solving activity proceeded smoothly until the call for
collaboration was issued. Developing the promotional materials and extending
the invitation to the broadest possible participant base was initiated
through GlobalSchoolNet. Response to the advertisement was negligible (non-existent)
for participants, although interest was generated for use of the site during
the upcoming school year. The factors that likely impeded a successful response
rate are manifold.
First, collaboration depends on a willingness of external
participants to volunteer for the activity. When there is no requirement
to participate, volunteers do not emerge. My unfamiliarity with GlobalSchoolNet
resulted in a failure to build realistic expectations. Frankly, I did not
know what to expect as far as response, so I expected nothing. The danger
with this level of expectation is that it is difficult to effectively evaluate
the response. Is the message wrong? Is the target too narrow? Is the project
too involved, or not involved enough, or is the project lacking in some element?
Secondly, timing for the activity restricted the pool of available participants—school
is not in session for most high schools during the summer months. Third,
the activity was announced as being restricted to a certain age group and
discipline. This further restricts the potential for a volunteer pool.
The respondent group that was eventually drafted met
the age and discipline requirements, but failed to provide an example of
a directed approach to the activity. Regardless, acquiring the responses
from the drafted group did provide valuable information regarding the activity’s
functioning. To that end, the activity requires further refinement as described
under micro analysis.
Micro-analysis
Technical difficulties
The survey application is an HTML/Java application converted
from an EXCEL spreadsheet. The choice of this initial development platform
was based purely on speed of development. An automated e-mail response mechanism
was used to deliver responses, from submitted participant surveys. Because
the email response server is hosted by a company in Sweden, the first hurdle
was to overcome spam-blocking-a simple correction, but a significant
amount of time was spent on debugging the application when in fact it was the
response receipt that was the problem. The primary difficulty to overcome is
the manual intervention required to compile and summarize the results. Each
response must be parsed individually into the main spreadsheet—this is
an acceptable level of intervention when there are 19 responses, but if there
were 1900, the effort would be too burdensome to complete.
As a result, the survey instrument’s data home
requires modification. A database application that
automatically parses individual responses into the application would alleviate
the manual step and streamline the data acquisition process. This upgrade
would also serve to enhance the usability of the application by others, as
data could be made available on a selection basis, i.e., by date, state,
question item, question response or other variations. This enhancement would
allow for a level of granularity providing a fine degree of flexibility—a
positive selling point in gaining the project’s broader acceptance.
Raw data could be extended on a form basis and data would be open for use
as the participants saw fit.
Migration of the survey instrument to a database would
enhance its ability to include the global community. Currently, all locations
other than the United States are considered “other.” Enabling
the tracking of countries and regions other than the U.S. would further expand
the appeal of the project.
Engagement
Presently, the project hands off data, suggests activities,
but does not allow for two-way communication between the user bases. In order
to enhance collaboration, the project should extend its methods for communication
beyond simple e-mail. Opening the project up by enabling survey editing, data
collection, or user interaction among other efforts, will extend the application’s
reach. Also, making available a simple RSS feed to automatically post updates
to participants would enhance organizational effectiveness of the project.
Finally, a location where participants can post findings, lesson activities
or suggestions would assist in extending ownership to a broader community.
Conclusion
Overall, the application serves its objective well-it
works. However, it is certainly not ready for a broad release without a mechanical
overhaul. The survey instrument needs refinement based on user interaction.
Regardless, the potential for the application is certainly within reach. Further,
testing of the project's broader learning objectives, i.e., the leveraging
of critical thinking skills in the analysis and evaluation of the survey results
compared to other survey metrics, has not been tested and will need to be reviewed.
Mark Emmons, UIUC EPSY590 , August 3, 2006
