Being a unique program with few models to follow, the most difficult aspect of the program was to determine how to assess the success of the program. This assessment was critical to attract participating schools, grant funding, and to measure the overall success of the development of the program. The foundation of the evaluation was based on measuring the proposed objectives; the objectives were divided into two categories: program objectives and student objectives. The program objectives included:
- Develop a working partnership between Ocean Institute, Orange County high schools, Apple, and Best Buy.
- Enhance classroom curriculum by bringing subject matter experts into the classroom via podcasts.
- Integrate technology into the established curriculum for students to develop podcasts based on lesson plans.
- Design an instructional and motivational platform for students to showcase their conceptual understanding of various subject matters.
- Provide professional development for teachers on the advantages of using podcasting and other technology in their classrooms.
The student objectives included:
- Research books and field guides to collect the necessary information.
- Organize the information they have collected in a way that can be developed into an educational lecture.
- Educate their peers on the skates and rays.
- Recognize the differences within aquaria.
- Use field guides to identify various species.
- Complete population counts and assess biodiversity.
- Identify keystone species and the importance of the species.
- Relate the pollution the biological integrity of the aquaria.
- Communicate their findings to their peers.
- Efficiently research on the internet to collect required data.
- Participate in an open discussion regarding problems and solutions to estuaries.
- Collect and organize digital video footage and images depicting their work.
- Edit all digital data.
- Create an mp4 file with digital data.
- Write an instructional script to teach a lesson plan.
- Record a voice file and integrate it into mp4 file to match with visual content.
- Use their creativity to design and produce an instructional podcast to be used by their peers at the Ocean Institute exhibits.
Measuring many of the student objectives were difficult due to the lack of quantitative results. Measuring the student objects were completed by monitoring their milestone accomplishments and assessing their final podcast production; to create a unified measuring tool, an assessment guide was created for teachers to evaluate the students’ progress. Another tool designed to assess the student objectives included a comparison analysis between student pre-tests and post-test results to measure a student’s academic growth. To continue to improve the motivational and content components, students were also distributed student surveys to complete; the surveys that have been completed have been analyzed. Once a greater number of surveys are returned, revisions to the program may be deemed necessary. The final analysis will be to compare state standardized testing results of those students who participated in the program to those who did not; this is a critical assessment in terms of receiving future funding, however due to the time constraints, this analysis has not been completed.
To successfully evaluate the first two objectives, they will need to be redefined with quantitative elements; these original program objectives were broad as a result of an experimental program with no model to follow. As the pilot program comes to an end, the results of the teacher surveys and visitor surveys will assist in providing a foundation to better redefine the objectives. Objectives that will require redefining include grant funding needs; this can be better determined once an accurate assessment of cost per class has been completed. Another objective that requires redefining is the desired number of participating schools both in short term and long term; there are a number of components that will influence this objective including the Ocean Institute’s ability to accommodate schools throughout the program, professional development needs, and budget needs for both schools and the Ocean Institute. Finally, the professional development seminars contain their own set of objectives, but will not be addressed within this report to prevent tangents.
It should be noted that the each of the evaluations noted above will be an ongoing and continuing process to ensure the program is of the highest quality. Continuing assessments are also critical due to the frequent updates and changes in technology which will need to be addressed when it becomes an identified issue; the hope is that the surveys will address such issues.