updated Feb. 26, 2014
posted May 23, 2012
In B.C., people who want to practise as full-scope nurse practitioners must pass the Nurse Practitioner Objective Structured Clinical Exam (OSCE, pronounced Os-key). There are other requirements as well, but the focus of this project was the OSCE. In April 2011, the College of Registered Nurses of B.C. commissioned a full review and evaluation of the OSCE.
The review included:
To practise as a nurse practitioner (NP) in B.C., a person must be registered with the College. When registering, NPs choose one stream of practice, depending on their education: family, adult or pediatric. NPs registered elsewhere in Canada, may apply for similar registration in B.C.
The College recently changed its NP exam requirements to comply with the Agreement on Internal Trade and the Labour Mobility Act. In the fall of 2011, the College amended its bylaws and the B.C. government approved the changes. B.C. graduates and certain applicants (those with unrestricted registration as an NP elsewhere in Canada) no longer have to pass the OSCE to register with the College.
Also changed was the Scope of Practice for Nurse Practitioners: Standards, Limits and Conditions. New requirements were added for NPs performing restricted activities and the supervision of NP restricted activities.
Requirements for NPs performing restricted activities independently are in Section A: Standards, Limits and Conditions on Performing Restricted Activities. The process nurses follow when supervising an NP’s restricted activities is in Section B: Standards for Supervision of Nurse Practitioner Restricted Activities. These changes came into effect on October 1, 2011. The OSCE is no longer a registration requirement. Instead, it is a requirement for independently performing restricted activities. To identify who can, NPs who passed the ANCC exam and the OSCE are called “exam qualified.” NPs who are not “exam qualified” can only perform restricted activities under the supervision of an exam qualified NP or authorized physician.
The project time frame spanned eight months (65 project days) from June, 2011 to January, 2012, during which quantitative and qualitative data was collected. There was a literature review, questionnaires, interviews, focus groups, stakeholder consultation and site visits.
The OSCE is a major tool for assessing clinical competence, as shown by the literature review. The review included examining the OSCE in measuring competencies, OSCE strengths and limitations, and key technical and organizational issues. The review also included best practices in OSCE design, scoring and administration, future trends and areas for development.
After studying the standards and best practices in the literature, a framework for review and evaluation of the OSCE was developed. (Vleuten, 2005, Boulet, 2008, Norcini et al., 2011). The framework describes the best practices needed for effective OSCE development, scoring and administration.
The first set of criteria looked at reliability and validity. The questions to be answered were: Can the result be repeated? What is the chance of error?
The second set of criteria looked at the organization and included acceptability, transparency and feasibility. The questions to be answered were: Do stakeholders accept the process? Is communication clear and appropriate? Is it practical, realistic, and sensible, given the circumstances and available resources?
The framework combines best practice with project design. These include data sources, methodology, whether criteria was met/not met and recommendations. The framework provides direction and helps the College interpret findings.
The OSCE meets the technical requirements of reliability and validity. However, it only partially meets the organizational requirements of acceptability, transparency and feasibility. The College needs to work with stakeholders to resolve the limitations identified in the review. It also needs to provide more useful information about the exam.
Although several sources confirmed the reliability and validity of the exam, this view was not shared by all. There are negative views about design, scoring and administration (e.g., lack of exam reliability and validity, transparency, feasibility, etc.). These negative views detract from those of the majority, who endorse the exam. There is less support for the exam today than when it first arrived. Some negative views about the exam may spring from contextual and environmental factors.
Context and the environment
The OSCE developed at the same time as other changes occurred:
Add to the mix:
Some negative views are based on misinformation. These can be addressed through better communications and information sharing. Other problems need a focused and collaborative effort to lay the foundation for the future success of the OSCE.
We will explore the idea with the B.C. Nurse Practitioner Association and NP educators in the coming months.
We will assess our communications and make improvements as needed.
We will consider this recommendation as we work with other stakeholders on the national project.
Three components were evaluated:
The consultation period took longer than anticipated. There was also a larger than expected amount of qualitative data to analyze.