Assignment #1- Evaluating a Program Evaluation

                After I spent a good deal of time finding a program evaluation that I was interested in and was not too complicated; I came across an Employee Wellness Program Evaluation Report. This was a report put together by the University of Maine for the Maine Department of Transportation. The department had started a wellness program for its employees and they wanted to make sure that the program was worthwhile. The driving factor of Maine Department of Transportation (DOT) was to decrease productivity loss due to sickness, absenteeism and injuries. The DOT was also interested in developing minimum fitness standards for each position and then taking steps to ensure that its employees were able to meet those minimum requirements.  The wellness program was started in 2004 and in 2008 the Maine DOT wanted to create a more detailed evaluation program to better determine the cost/ benefits of the program and to expand the program so it could be used throughout the state.

                The evaluators were looking at both short-term and long-term outcomes. The short-term outcomes focused on increasing the participation level among employees, increasing physical activity and healthy eating habits and decreasing the amount of substance abuse. All of these short-term outcomes would lead to a decrease in illness and injuries. Measuring these short term outcomes will be done by monitoring vital signs (blood pressure, weight, etc), collecting workers compensation and health claims and monitoring health risk scores. In addition to these, employee satisfaction levels and goals would be determined using employee surveys. The evaluators found generally that the program was working well. More money was being saved than was being spent on the program, lost time due to injuries was drastically reduced and participation was increased from 40% to over 60%. The long term outcomes focused on creating a safe and injury free work environment but there had not been enough time to really study the full effect on long term outcomes.

 The evaluation of this program is summative as it is meant to provide feedback of the current program in order to make improvements allowing the program to expand. In my view summative evaluations are an important part of any program as it critically examines the program and is an effective way to bring forth needed changes to improve. The evaluators used an outcome based method of evaluation. First the evaluators determined what the desired outcomes of the program were, analyzed the data collected for each outcome, and finally made recommendations specific to that outcome.

 I think that the evaluation of the employee wellness program was very well done. The outcomes were described thoroughly and it was made very clear where the data to measure it came from and how it was used. The evaluators also provided detailed findings and recommendations specific to that outcome. This made the report very user friendly and ensured that each outcome was covered in detail. I also liked the fact that the evaluators would critique the data used and would provide suggestions as to how to improve the type of data that was collected for that specific outcome in order to improve future evaluations. Some of the weaknesses of the evaluation were that the language was fairly technical in some parts.  Also it was hard for the average person to follow what they were comparing. At times they were comparing the 2008 data to the 2004 data and other times the workers would be compared to the overall state’s data but it didn't specify where that data come from or who was included in it. There are also a few instances where it is unclear how they obtained certain data, such as for vital signs, was it by a doctor or simply part of the employee survey. Overall I believe that the evaluation provides solid recommendations that would easily allow improvements to be made and to allow other regions to implement this program.