20 / 8 / 2012 10.07am
Evaluations allow assessment of the implementation and the final outcome of the project. This will help the organisation to evaluate the options open to them to either continue, rethink and identify future development or stop the project altogether.
Evaluation is used to assess how well the outcomes of a project have met the original objectives and benefits identified in the business case. The evaluation may not be just confined to quantifiable objectives, such as reducing your temporary staffing budget.It should be designed to focus on the areas which will identify the impact of the introduction of a new role in this instance, has had on the trainee, other staff in the team and patients.
There are some core steps which can be used in the evaluation of any project but each time you will need to tailor the approach to your own local needs. Above all it’s essential that you spend some time planning what you are wanting to measure at the start of the project to get the best outcome. To help you through this we have outlined the core steps below:
- Clearly set out the project goal
- Ensure the project objectives are well defined and understandable
- Identify the evaluation stakeholders
- State the expected project outcomes
- Identify the evaluation indicators to measure the project’s success and key achievements – these could be:
- How many people have started the assistant practitioner programme
- How many have completed the programme
Gain feedback from them and the staff they have worked with
- What types of problems have arisen
- What changes in behaviours and skills have been observed by trainees, managers and other staff
- Patients feedback
- Assistant practitioner feedback
- Staff and manager feedback
Choose your questions to reflect the key areas/main issues for the successful delivery of the project.
The first stage of developing the evaluation takes place during the planning of a project – setting the objectives and expected outcomes of the project and the resources that are required.Testing whether the project is delivering the outcome can be done in stages, gathering evidence and intelligence on progress that may help to identify any adjustments that may need to be made and gather data on the individuals learning experience.
There are many ways organisations can evaluate projects but essential to all of them is that they are carefully planned and organised to meet the identified goals and objectives.
Some of the following are tried and tested methods of evaluation:
- Interviews with staff including trainees, managers, mentors, other members of teams and where appropriate patients and carers
- Surveys are a valuable source of information, providing the trust with details such as how individuals perceive a particular issue and help to identify areas of concern, how individual team members view the scope of an assistant practitioner role and the impact of actions
- Use of staged assessments can allow organisations to consider any adjustments that need to be made at key stages, for example, interviews with trainees at the start, halfway point and end of a training programme to test expectations and delivery of aims
- Feedback forms for the different stakeholders involved can be a way trusts can collect data on a variety of areas and topics.
There are a number of reports and evaluations produced by the NHS and there are many generic examples on the internet that will assist trusts with their evaluation process. We have listed below some of the documents organisations can access.
NHS Tayside has produced a report and evaluation the ‘Assistant practitioner project’ which has used many of the tools and resources mentioned in their webpage.
NHS Wirral has produced a factsheet ‘Fact sheet 6: How do I evaluate my project or service?’ which provides a guide to the different types of evaluation organisations can use.
NHS Education South Central has produced a report ‘Evaluation of assistant/associate practitioner roles across NHS South Central’. The main aim of the report was to assess how the assistant practitioner role is used in their area and to examine the impact of the introduction of the role has brought so they can develop the role going forward. The data was gathered by using interviews with managers and assistant practitioners.
NHS Tayside with the Rannoch & Tummel community held a public consultation which brought a number of areas of concern about the provision of emergency services in their area. The report ‘Evaluation of the introduction of a community first responder scheme in Rannoch and Tummel’ has examples of the collection of information from surveys, questionnaires, focus groups and interviews which helped inform their recommendations
NHS Education for Scotland commissioned the Institute for Employment Studies (IES) and the University of Hertfordshire (UH) to carry out an evaluation on the impact their pilot programmes for training assistant practitioners to work in the diagnostic imaging departments within their region. The evaluation ‘Second phase evaluation of the introduction of assistant practitioners in imaging services in Scotland’ was carried out by using various methods including interviews, surveys and case studies and is a good example of using a phased evaluation approach.
You can read more information about assistant practitioners on our website.