Assignment #4- Logic Model

So I have been having some troubles posting new posts to my blog, it acts like it should be loading but it all I get is a blank screen. I gave up last night and tried with a refreshed mind this morning, even though it should have been done yesterday- opps. I jumped outside of my knowledge box and took the plunge into using google docs as Bill and J.R had talked about using it to post in class. I have to say that I learned a lot and I will likely start to regularly use it, I didn't know how useful it could be! That being said I'll need to see if my link actually works or if I need to go back and do more research. Here it is, see below for the link- fingers crossed.
Logic Model- Outdoor Education

Now to create my survey :)

Assignment #3- Planning a Program Evaluation

Outdoor Education

Engage Stakeholders

Who should be involved?

The Outdoor education teachers, administrators, participating students at Delisle Composite School will be involved in the assessment and the general community including parents will also be informed of the results.

How might they be engaged?

Staff and students and will reflect on the program through survey questions, interviews, and observations.  Students will also be involved through summative evaluation by comparing final exam answers with non-participants and interviews of selected students.

Focus on the Evaluation

What are you going to evaluate?  Describe program (logic model)

The plan is to evaluate the:

-effectiveness of student learning in Biology 20 and Physical Education 20

-level of student engagement and satisfaction

Delisle Composite School (DCS) serves 230 students from grade 7 to grade 12. The school is part of the Prairie Spirit School Division.  The student population in the school has been decreasing in recent years due to the influx of students choosing to attend Saskatoon high schools. As a way to entice students to keep attending school in Delisle, a number of interesting programs have been created. It is the hope that these programs will increase student engagement and will help DCS compete with the course offerings of bigger city schools.

The outdoor education program combines physical education 20 and biology 20 together which helps to increase the hands on perspective of biology 20. The phys. ed. portion of the course is easily incorporated through biking, snowshoeing, canoeing, hiking and other activities. The biology portion is typically hands on projects and activities such as pond dipping, collecting samples and making observations. This knowledge is supplemented by some coursework and research which helps to occupy any days where weather does not permit students to be outdoors. This is a unique course as it allows students to enter into the community and use the resources that are available around them. Therefore strong community relationships need to be forged allowing students access to resources such as the golf course and continuing funding that the program receives from local businesses.

What is the purpose of this evaluation?

This is a formative and summative assessment with the intent of identifying the effectiveness of the outdoor education program and to identify ways to improve the program by making any adjustments that are needed to make the program more effective.

Who will the use the evaluation? How will they use it?

Who/users	How will they use the information?
Outdoor Education Teacher	Results can be used to ensure that the teacher is delivering adequate knowledge in both subject areas as well as engaging students with a positive educational experience. Results will also help guide the teacher if changes need to be made.
Administrators	Information from the evaluation will be used to improve the program and to gauge budget needs.
Staff	Results and conclusions from the program evaluation will be shared with all staff members to increase staff knowledge of school programs.
Community	Results from the evaluation will be shared with the community, in order to build relationships and ensure contributing businesses that their funding is being well used.
Parents	Results from the evaluation will be shared with parents, who will hopefully endorse the program and spread the word through the community

 What questions will the evaluation answer?

How does Outdoor Education affect student engagement?

How does Outdoor Education affect student learning?

Do parents see benefits to Outdoor Education?

Is the teacher utilizing instructional time efficiently?

Are students performing as well on final exam questions?

Are resources being used to their potential?

Are the existing resources adequate enough?

What information do you need to answer the questions?

What I wish to know	Indicators – How will I know it?
Does Outdoor Ed. affect student engagement? Is the teacher using time efficiently?	Observation and student surveys in both the outdoor education classes and traditional classes
Are students obtaining the same knowledge as those in traditional classes? Are students preforming as well on final exam questions?	Compare marks between the traditional and outdoor education classes.
Do parents see benefits to the program?	Survey given to the parents
Are resources adequate and being well used?	Interview outdoor education teacher and administration

 When is the evaluation needed?

The evaluation would be carried out throughout the semester and would be finished by the end of the year on June 30. This would give the evaluator time to compare final exam results, compile student and parent surveys and complete the necessary interviews.

What evaluation design will I use?

The evaluation will be both formative and summative. It is formative since the research will be on-going during the semester and summative since there will be data collected at the end of the year and any changes or modifications would not be made until the evaluation was complete. The evaluation will be goals-based using a CIPP model.

Collect the information

What sources of information will you use?

People: Administration, teachers, students and parents

Pictorial records and observations: Administrative observations and teacher observations

What data collection method(s) will you use?

·         Surveys (Before and after)

·         Interview (After)

·         Observation (During)

·         Marks on final exams (After)
      Focus Group (During)

                      

Instrumentation: What is needed to record the information?

• Surveys:  design questions; distribute surveys by email to parents, and handouts to students; collect and organize results
• Interviews: choose a random sample of students to answer questions, create questions, schedule interviews at the end of the course
• Observations: general document – to be done throughout term (beginning, middle, end)
• Focus Group: design questions; invite participants; record conversation

 When will you collect data for each method you’ve chosen?

Method	Before program	During program	Immediately after	Later
Surveys	X		X
Interviews			X
Data (marks)			X
Observation Focus Group		X X

Will a sample be used?

No

Analyze and Interpret

How will the data be analyzed and what methods will be used?

·         Surveys: interpret and compile information (evaluator)

·         Interviews and focus group: interpret, compare and compile information (evaluator)

·         Observations: compile and compare 3 sets of observations (teacher, admin., evaluator)

·         Data (marks): compile and compare marks from final exams between two groups of students

How will the information be interpreted-by whom?

The Evaluator will be responsible for analyzing data.  The administration and outdoor ed. teacher will then have the opportunity to be brought into discussion and participate in discussion and allow for their interpretations to be included.

What did you learn?  What are the limitations?

• ·         Since the evaluator is an internal member of staff there will need to be some care taken when analyzing data and making judgements on the program.
• ·         There would also be some time constraints on getting all the data collected between the final exam and the end of the year when marks are distributed.
• ·         The evaluator needs to consider any variations between the traditional classrooms and the outdoor education groups; some years the demand for the program requires that students get ‘accepted’ into the program

Use the information

How will the evaluation be communicated and shared?

The evaluation will be analyzed and complied into a report that will then be shared with administration and the outdoor education teacher.

What are the next steps?

Together the administration, teacher and evaluator will create a brief statement of the results and any changes that may be implemented in the future. This can be shared with other stakeholders whenever it will be possible.

Manage the evaluation

Human subject’s protection

Surveys will be done anonymously by students and parents.  This will allow for participants to answer questions truthfully.  The data collected will have all names and identification removed before the report is made.

Management chart

The evaluator will create a gantt chart (used to schedule complex schedules) and share it with administrators.

Timeline

·         January: complete all preparatory evaluation work, prepare surveys and interview questions

·         February: conduct pre surveys and do observations

·         March/April: analyze pre-survey results, complete another observation

·         May: begin conducting interviews of teacher and administration, distribute parent surveys; conduct student focus group

·         June: distribute, conduct and analyze post surveys; collect and analyze final exam data; prepare report and meet with administrator and teacher to finalize report and add in improvements

Responsibilities

The evaluator is responsible for creating all survey and interview questions. In addition they will need to distribute surveys, set up interviews, analyze and summarize the results and create a report to share with administration and the teacher.  Collaboratively, a list of recommendations will be compiled and included in a brief report that will be shared with the shareholders. The teacher will be responsible for making the final exam and collecting them for the evaluator afterwards.

Budget

All devices and materials required are located internally and therefore, the fees included would be any expenses incurred by the evaluator.

Standards

Utility

Evaluation of the program will be used to make improvements to the program and to evaluate the program effectiveness.   

Feasibility

The outdoor education classes tend to be quite small so the evaluation should be manageable. The most difficult part would be in June when a lot needs to be accomplished and schools are at their busiest, however the interviews and focus group could be conducted in May to help ease this time constraint. If necessary, arrangements could be made to extend the deadline if all the parties involved will agree to it.

Propriety

There would have to be an element of trust from the very onset of the evaluation so all parties are respected in all steps of the evaluation.

Accuracy

The evaluation is looking at specific goals and outcomes and should be accurate due to this concentrated focus of the evaluation.