Development of a program evaluation plan for the Canary Islands' mental health services

Aspects of Maher's (2000) approach to program evaluation planning were applied to develop a program evaluation plan for administrators of the Canarian Mental Health Services (CMHS) of the Canary Islands of Spain so that they can routinely evaluate their services. Application of this approach involved a series of sequential and interrelated steps, beginning with the initial identification of the client (the CMHS director) and relevant stakeholders (staff, service recipients, residents), followed by an assessment of the evaluation information needs of these constituencies and then development of the plan. Results from the evaluation information needs assessment identified information gaps in the CMHS in several areas. There was insufficient information among residents regarding mental illness and the availability of services. Professionals lacked information on how units functioned, their interactions, guidelines for patient care, continuity and outcome of patient treatment, and opportunities for specialized training. Demographic information on patients and professionals as well as their degree of satisfaction with the services provided was also missing. Finally, there appeared to be less than adequate information about the clinical training of primary care doctors. The findings, coupled with relevant literature, along with other information from the interviews and surveys were used by the consultant in the formulation of a program evaluation plan. The program evaluation plan included: (1) an overview and description of the mental health services provided by the CMHS; (2) specification of the program evaluation questions and protocols, and (3) procedures for communicating the results of the program evaluation and for evaluating the program evaluation plan. It was concluded that Maher's approach was an appropriate framework for the development of a program evaluation plan for the CMHS, due to the flexibility and adaptability of the approach to a variety of cultural and linguistic contexts, along with the specificity of information provided. It was anticipated that results from the program evaluation plan would address the objectives and goals posited by the CMHS and enable its director to make necessary decisions for the ongoing improvement of its services.