2006 OPEN FORUM Abstracts
IMPROVING THE PRECEPTEE EVALUATION PROCESS FOLLOWING PERFORMANCE IMPROVEMENT CHANGES
Baldwin, MBA, RCP, RRT, Alan Alipoon, BS, RCP, RRT, Robert Roy, RCP, RRT
Michael H. Terry, RCP, RRT, Leo Langga, BS, RCP, RRT-NPS, Michael Lum, BS, RCP, RRT.
Loma Linda University Medical Center, Loma Linda, California.
Background: Prior to performance improvement changes, preceptees were evaluated using a Likert rating scale of one through five to indicate the level of targeted performance achieved. A preceptee receiving a score of one was deemed to have failed that particular performance objective. A score of three meant the preceptee had achieved a satisfactory rating and a score of five indicated the preceptee had excelled in meeting the performance objective. There were no performance rating definitions given for a score of two or a score of four. Due to the subjectivity of the descriptors, coupled with definitions for only three of the five scoring options the tool was found to be inadequate in providing pertinent feedback to the preceptee's progress. We also observed inflation was occurring with a disproportionate number of scores of four and five early in the preceptee orientation and felt the current evaluation system was biased towards high scores and hindered the preceptor's ability to provide guidance to the preceptee. Performance improvement changes were implemented that assigned the following descriptors to the Likert numerical ratings:
- Score 1-Novice
- Score 2-Experienced Beginner
- Score 3-Competent
- Score 4-Competent +Efficient
- Score 5-Expert
Attached to every preceptee survey were the definitions of each descriptor score. We hypothesized that with the revised survey tool, preceptee scores would reflect more valid levels of competence in their professional growth and development. The preceptees were rated in the following categories: "Triage of Patient Care", "Assessment of Respiratory Needs", "Response to Stressful Situations", "Communication and Understanding of Respiratory Care Best Practice", "Interpretation and Management of Blood Gases", "Interaction with Patient and Family", and "Interaction with other Allied Health Team members".
Method: Thirty-two Preceptee scores were randomly selected before and after the implementation of the new survey tool. All scores were an overall average of each survey instrument.
Results: See Table 1. The means with standard deviation were calculated for each sample. The sample mean after the performance improvement change decreased from 3.7 to 3.4.
Table 1 - Preceptee Evaluation Scores
|January - March 2006 (pre-revision)||April - May 2006 (post-revision)|
|Mean = 3.7||Mean = 3.4|
|Stdev = 0.77||Stdev = 0.59|
|n = 32||n = 32|
Conclusion: Through performance improvement changes, we
were able to improve the accuracy of our preceptee scoring and provide improved
feedback during their clinical training.