J Acupunct Res Search


J Acupunct Res > Volume 39(2); 2022 > Article
Cho, Han, Kang, Kim, Shin, Oh, Jung, Jeon, Cho, and Leem: Development of an Objective Structured Clinical Examination Checklist and a Post-Education Questionnaire for Musculoskeletal Ultrasound Training Focusing on Volar Wrist and Carpal Tunnel Syndrome



The objective structured clinical examination (OSCE) is used in the colleges of Korean Medicine, but few studies have validated the OSCE evaluation criteria or post-education questionnaires. Diagnostic ultrasound is used in Korean medicine treatment including acupuncture, acupotomy, and pharmacopuncture to increase the safety and efficacy of treatment. We aimed to develop and validate a OSCE checklist and questionnaire for diagnostic musculoskeletal ultrasound training.


A OSCE checklist and rubric for diagnostic ultrasound training, and questionnaire was developed using literature research. Eight expert panelists verified each draft item in a single-round survey. Items with a content validity ratio (CVR) < 0.75 were excluded or modified to reflect the experts’ opinions.


The OSCE checklist and rubric for diagnostic ultrasound training focusing on volar wrist and carpal tunnel syndrome included: 15 items revised according to CVR and expert opinions, the pre-examination procedure, structures to be identified by ultrasound, scans with 2 diagnostic criteria for carpal tunnel syndrome, an explanation of the exam results, and the post exam procedure. The questionnaire consisted of 15 items, including the overall evaluation of training, the effect of the OSCE, and the perception of the ultrasound. All 6 self-evaluation items were not revised, as they had a CVR of ≥ 0.75.


An ultrasound OSCE for scanning the volar wrist and diagnosing carpal tunnel syndrome was developed using 15 validated tasks, 15 survey questions about ultrasound training, and 6 questions for self-evaluation. These results may be used in the future for education in diagnostic ultrasound.


The objective structured clinical examination (OSCE) was developed in 1975 in the UK, and is widely used in the assessment of healthcare students to evaluate clinical performance using a valid, reliable approach [1,2]. Previously, the students’ competence in taking the medical history, performing a physical examination, diagnosis, and providing a management plan was evaluated mainly at the bedside using a single, lengthy case. However, concerns were raised that the number of items was too small, not sufficiently objective, and in terms of reliability and validity it was deemed low. To resolve these problems, Harden et al developed the OSCE to control the characteristics for the patient, and examiner, and variables of the clinical examination. Students examined using the OSCE are evaluated over the diagnosis of various conditions and several patients, allowing many skills to be evaluated, and the evaluator bias is minimized due to the checklist items or rating scale assigned in advance (practical and flexible approaches are also provided for the use in other situations) [1,2]. As the OSCE became widely adopted around the world, versions of the OSCEs were developed and implemented in various areas of healthcare education including medicine nursing, and dentistry [1]. In Korea, since 2009, with the launch of the national practical examination of doctors, the OSCE has been implemented in all medical schools [3]. The implementation of the OSCE in Korean medicine education is increasing, however, there have been no discussions to increase its reliability and validity. The Institute of Korean Medicine Education and Evaluation require that the Clinical Performance Examination or the OSCE is conducted at least once a year in Korean medicine clinical education as an eligibility criteria of the 2nd cycle of evaluation and accreditation of Korean medicine [4]. It is likely that most colleges of Korean medicine are implementing the OSCE, as all colleges of Korean medicine (school of Korean medicine) have completed the 2nd cycle of evaluation and accreditation of Korean medicine [5]. Furthermore, each college of Korean medicine should carry out at least 10 OSCE (including at least 1 OSCE for each clinical subject), for the next evaluation and accreditation of Korean medicine [6]. However, research on the development and validation of a checklist for OSCE evaluation in Korean medicine education is still in an early stage, and there are few studies that have developed an OSCE checklist for Korean medicine. In a previous study on the experience and students’ perception of acupuncture and moxibustion medicine OSCE, the existing OSCE checklist was partially modified to suit the situation without a validation process [7]. OSCE checklists or rating scales are going to be required for introducing the practical examination in the 2030 national examination for Korean medicine, as discussed by the Deans Association of colleges of Korean medicine in 2019, which will require sufficient discussion by experts [8]. Referring to the introduction in 2009 of the medical license examination practical exam in Korea, after selecting the evaluation items during the preparation for the practical questions, the experts in each item were asked for a draft of a rating scale of detailed items that the student needs to perform, which the Questions Review Committee completed after deliberation [9]. Pilot studies related to the production of such checklists and rubrics are needed in Korean medicine, so, although the list of critical skills that students majoring in Korean medicine must be able to perform by graduation has not yet been established, the need for education on ultrasound diagnostic devices is increasing as reported in a 2013 survey of Korean medicine graduate school students [10]. In 2017 a survey of full-time faculty members at the colleges of Korean medicine revealed that competency was important to Korean medicine doctors (just after obtaining their license and in the next 10 years), specifically in the use of diagnostic devices (e.g., ultrasound, pulsimeter) [11]. In addition, studies using diagnostic ultrasound devices have been reported to increase the safety and efficacy of Korean medicine treatment [1215]. Diagnostic ultrasound training is required for undergraduate courses because the safety of a procedure can be improved if the structure and depth of the treatment site are identifiable [16,17]. Evidence shows that ultrasound-guided interventions enable the delivery of more effective treatment for relieving pain and improving function compared with procedures where diagnostic ultrasound has not been used [18]. Instruction on scanning using ultrasound and scan analysis are included in the textbook for practice in diagnostics of Korean medicine [19], and ultrasound images of all acupoints are included in the textbook for the practice of acupuncture [20]. There has been a report of teaching how to perform a pelvic examination using ultrasound in clinical practice education in Gynecology of Korean medicine [21]; however, to our knowledge, there has been no report of implementing the OSCE or developing an OSCE checklist in the College of Korean Medicine. Therefore, we aim to report the process and results of developing an OSCE checklist for diagnostic musculoskeletal ultrasound and a questionnaire for student feedback using expert consensus. The goal of this study was to develop an evaluation sheet for ultrasound training in the future, thus improving the quality of training, and contributing to the competent use of ultrasound in the clinical practice of Korean medicine.

Materials and Methods

In this study, after drafting the OSCE checklist and questionnaires through a literature review, the validity of content was verified by an expert group to evaluate each item through a questionnaire survey.

Ethical consideration and informed consent

We acquired prior approval from the institutional review board of Wonkwang University for validation by survey (no.: WKIRB-202106-SB-039). The voluntary consent of the panelists was obtained before their participation in the survey.


The OSCE checklist of this study was developed and validated for diagnostic musculoskeletal ultrasound training for students of the College of Korean Medicine (Fig. 1).

Setting learning objectives and contents

Education in ultrasound is provided to the senior students of the College of Korean Medicine during the clinical skills practice course. The contents of this course were drafted by a Korean Medicine doctor who was licensed to practice musculoskeletal sonography [22,23] and a doctor of Korean Medicine with experience in clinical research in ultrasound who had more than 2 years of implementing the OSCE. During the drafting of the learning objectives and training required in musculoskeletal ultrasound, advice was sought from a professor of Korean medicine diagnostics. The volar wrist was selected as the area to be scanned by ultrasound and the competency of the student to diagnose carpal tunnel syndrome was included as a learning objective. The anatomical structures in the volar wrist are located superficially, thus, it is fairly easy to perform an ultrasound scan in this location. In addition, scanning the wrist can be performed without concern about body exposure. Furthermore, testing the competency of the student to diagnose carpal tunnel syndrome is essential because the number of people diagnosed with, and receiving treatment for carpal tunnel syndrome at Korean medical institutions is increasing [24]. Table 1 presents the learning objectives of the clinical skills practice in ultrasound.

Literature review

Two researchers drafted the OSCE checklist and questionnaire on ultrasound training. Firstly, the overall ultrasound diagnosis procedure was checked in the Practice in Diagnostics of Korean medicine [19], and Practice of Acupuncture; A Guidebook for College Students [20], which are the recommended textbooks for students at the College of Korean Medicine, and contain ultrasound-related content. In addition, structures to be scanned in the anterior part of the wrist and methods for diagnosing carpal tunnel syndrome were extracted and selected by referring to musculoskeletal ultrasound books [2529] and a previous study [30]. The OSCE was designed with a checklist and rubric for each item, and post-education survey items by referring to previous studies [3135] and books [36,37] on the ultrasound section in the OSCE. The draft OSCE checklist consisted of 11 tasks to be performed before, during, and after the ultrasound examination. The total potential score was 23 points. The questionnaire consisted of items about ultrasound training and the OSCE, and questions for self-evaluation after training. The draft was drawn through agreement by 2 researchers and was passed on to the next round after review by a specialist in internal Korean medicine with more than 10 years of clinical research experience.

Expert panels

A group of 8 panelists was formed for content validation of the OSCE checklist and draft questionnaire. Information on the experts is presented in Table 2.

Content validation

The survey was distributed using e-mail or messenger with an online survey tool (SurveyMonkey, San Mateo, CA, USA) between June 23, 2021, and June 27, 2021. The learning objectives of the ultrasound training, structures that can be observed with ultrasound from the volar wrist, and methods for diagnosing carpal tunnel syndrome with ultrasound were presented to the experts’ before the survey.
The content validity ratio (CVR) that was used in this study is calculated using the following formula to judge content validity where Ne is the number of panels that answered “essential,” and N is the total number of panels [38,39].
According to the minimum CVR values, 0.75 (“essential” from 7 out of 8 panelists) is required to satisfy a 0.05 significance level. Therefore, 8 experts were asked to rate the draft on a 4-point scale (not at all essential, slightly not essential, slightly essential, strongly essential), and 7 or more panelists responded that “slightly essential” or “strongly essential” were acceptable. The validity of all the OSCE checklist items was evaluated, and then the validity of the scoring criteria (rubric) of each item was evaluated. Panelists were asked to present specific opinions, including revising the score on each item, if necessary.


The OSCE checklist and rubric

Content validation of the draft OSCE checklist and rubric are shown in Table 3. Item 6, with a CVR of less than 0.75 was excluded from the final version of the checklist. For Item 3, the expression “with hand sanitizers” was removed and modified to only evaluate whether hand hygiene was performed. Although the CVR of Item 5 was 1, selecting the probe as “linear,” the area to be scanned as “musculoskeletal” and “wrist,” and adjusting the depth to scan were added to the rubric because the expert opinion was that the capability to select or adjust the setting of the device for musculoskeletal ultrasound should be added. Item 6 was included to evaluate probe handling, but the CVR was 0.5. There was an opinion that the evaluation criteria for proficiency were vague, and it would be better to consider the pressure and angle of the probe handling, and holding the probe in such a way as to prevent slipping, in addition to adjusting the direction and position of the probe. Thus, the item was modified to evaluate both adjusting the direction and position of the probe, and whether the probe was held stably. Item 7 had a CVR of 1 but was modified to “apply sufficient gel to the lens surface of the probe or the patient’s body to be scanned” to reflect the opinion that gel can be applied to the patient’s body rather than only the probe. Item 8 was to scan and explain the structures that can be observed from the volar wrist, but there was an opinion that using a log scale instead of a linear scale would be better because if the student could not scan, there was no point using a linear scale. Therefore, 10 structures of volar wrist were listed, and the rubric was changed to identifying 9–10 structures for 3 points, 4–8 structures for 2 points, 1–3 structures for 1 point, and 0 points if the student failed to scan any structures. Hand hygiene after the ultrasound examination was added following patient contact. Cleaning the device was added following expert opinion. In addition, self-introduction to the patient was added to the 1st part of the checklist, according to expert opinion that it was necessary. The final OSCE checklist consisted of 15 tasks with a total score of 28 points (Supplementary A).

Questionnaire on ultrasound training

The draft questionnaire for ultrasound training and the OSCE consisted of 19 items, including 17 multiple-choice, and 2 subjective questions. Panelists were informed that a 5-point scale would be used to answer multiple-choice questions before the survey. The results of the expert validation of the draft questionnaire were analyzed (Table 4).
Question E-1 was excluded because it was similar to Question 1, and ultrasound was taught in the previous curriculum, in practice education in acupoint and meridians. For Question 1, the opinion that ultrasound is not only a diagnostic device and is not only used in the medical field was accepted, so it was specified as “diagnostic” ultrasound, and the expression “before this training” was added. Questions E-2 and E-3 in the draft were excluded because they had a CVR of 0.5. Question E-4 was excluded because it was related to Question E-3. By reflecting on the opinions of the expert panel Question 8 which had a CVR of 0.75, was modified to “Did the ultrasound OSCE help motivate independent practical training?” to improve the understanding of the question. For Question 9, the expression “difficulty high” was modified to “difficult” reflecting the opinion that expression was not appropriate. Through this modification, the final version of the questionnaire for ultrasound training which consisted of 15 questions, was derived (Supplementary B).

Questionnaire for self-evaluation

The draft questionnaire for self-evaluation was finalized without revision since all self-evaluation questions had a CVR of 0.75 or higher and there were no additional opinions offered by the expert panel (Table 5). The Korean version of the final OSCE checklist and questionnaire on training in performing ultrasound are available online at http://dx.doi.org/10.13140/RG.2.2.35088.89609.


In this study, after setting the learning objectives that senior students of the College of Korean Medicine can use ultrasound devices, examine the structures of the body, and evaluate clinical implications, we developed and validated an OSCE checklist for scanning volar wrist with ultrasound and examined it according to the diagnostic criteria for carpal tunnel syndrome, a questionnaire concerning ultrasound training, and a questionnaire for self-evaluation. The OSCE checklist was modified from 11 items (23 points) in the draft to 15 items (28 points) in the final version. As for the questionnaire on ultrasound training, the draft consisted of 17 multiple-choice and 2 subjective questions, and the final draft, derived after review by 8 expert panels, included 14 multiple-choice and 1 subjective question. The draft questionnaire for self-evaluation was verified as valid and was decided as the final draft without modification. The results of this study will be used for musculoskeletal ultrasound training and OSCE for future doctors in Korean medicine.
Ultrasound is increasingly being used in education and clinical fields. Abdominal ultrasound OSCE was performed in postgraduate medical education in the 1990s, and the national ultrasound curriculum was developed in the United States in the 2010s [31]. Point-of-care ultrasound, taking ultrasound to the bedside, and examining the patient’s symptoms and signs in real time has been rapidly spreading in patient care and education [40]. Point-of-care ultrasound is being used in various fields, including anatomy, physiology, physical examination, pathology and clinical skills, and the importance of ultrasound training in medical schools [40, 41]. In Korea, ultrasound has been included in the training course of internal medicine residents since 2017, and a resident training staff certification system for ultrasound education was introduced [42].
The development and validation of the ultrasound OSCE checklist have been reported in several studies, but it has not yet actively been used in the educational field. One such study evaluated the importance of the rheumatoid OSCE assessment checklist through a single-round survey of experts in 2015 [43]. In that study, musculoskeletal ultrasound, especially “demonstrate/describe how to inject carpal tunnel syndrome” was included in the final rehabilitation station, but there were no items related to scanning the volar wrist or diagnosing carpal tunnel syndrome [43]. In addition, even though assessments for critical care ultrasound [44] and thoracic sonography [45] have been developed, validated checklists in previous studies mainly include procedures for using ultrasound and identification, rather than pre-examination procedures such as explaining the examination, hand hygiene, and post-examination procedures such as explaining the final findings. There have been several feasibility studies of training musculoskeletal ultrasound for undergraduate students [46], internal medicine residents [35], and rheumatology fellows [47], but specific tasks or checklists to be performed by the student have not been described. There are no previous studies on the development of an ultrasound OSCE checklist in traditional medicine.
According to a scoping review of ultrasound curricula in undergraduate medical education, musculoskeletal ultrasound is included in various topics of integrated point-of-care ultrasound curricula. The objectives/skills of musculoskeletal ultrasound are identifying the ultrasound appearance of muscles, tendons, bones, and nerves. The median nerve and carpal tunnel are presented as examples of the nerve [41]. Although ultrasound scanning of the median nerve and carpal tunnel has been included in medical education, cases of ultrasound education or specific OSCE evaluation checklist have not been reported so far. The OSCE checklist of this study could be applied to various fields in the future since proper OSCE rating scales are lacking, although the frequency of ultrasound use in clinical and educational fields is increasing.
In this study, most of the items had a CVR of 0.75 or higher, so content validity could be considered high, but we considered the average, median, mode, and opinions of expert panels to further adapt to the educational field. According to previous OSCE experience, there were cases in which evaluation items were not clear among checklists published as books, or it was difficult to grade because 2 or more tasks were included in 1 item [7]. Therefore, we aimed to include 1 task per item when developing the OSCE checklist. In addition, expert consensus could reduce the number of variables that can occur in OSCE as much as possible.
We aimed to increase validity through content validation from 8 expert panelists. To date, several studies have reported surveys of education. Some studies transculturally translated validated English questionnaires into Korean [48], but most existing studies published in journals of Korean medicine produced questionnaires themselves and did not mention how to validate the questions or conducted the survey after being reviewed by 4 or 5 experts [7,10,21,4959]. Researchers planning future surveys for Korean medical students must consider how to determine the reliability and validity of the survey questions.
Eight panelists were requested for content validation, and professors from the College of Korean Medicine, where the OSCE checklist and survey questions on ultrasound training will be used, were targeted in particular. Considering the previous study [60] that suggested recruiting 5 to 20 panelists in the Delphi technique, this may be a relatively small number. It is desirable for key stakeholders to discuss the OSCE rating scale, but it is still difficult for several professors to discuss OSCE within a single college of Korean medicine. The panels in this study are not directly related to the OSCE. However, we intended to include at least 2 experts in each ultrasound, practice, and education. Panelists who majored in basic Korean medicine, clinical Korean medicine, and pedagogy were included. The composition of this expert group allowed a review of various viewpoints and backgrounds.
This study had several strengths. To our knowledge, this study is the first to develop an ultrasound OSCE checklist in the Korean medicine or integrative medicine field. We comprehensively reviewed textbooks and guidelines, as well as related previous research, to develop a practical checklist. In addition, we adopted an expert survey method for content validity. The validated questionnaire, not only for the ultrasound OSCE but also for overall training, could be used for further assessment and improvement of ultrasound training models.
However, this study has some limitations. One limitation is that the content validation procedure was conducted only once. A more detailed discussion would have been possible if a focus group interview was conducted after the survey. In addition, if the Delphi technique is used in the future, it would be possible to develop a more valid OSCE by collecting expert opinions several times and drawing consensus. A scoping review is needed to understand the current status of Korean medicine education, especially on the OSCE. Future ultrasound training might be supplemented by receiving feedback from students using the results of this study and post-education questionnaire. Moreover, a before-after study of self-efficacy on musculoskeletal ultrasound could be conducted by a self-evaluation survey before and after ultrasound training. A feasibility study on training ultrasound, implementing OSCE, and improving education by students’ satisfaction survey using the results of this study is needed.


As practical examination is scheduled to be introduced in the national examination for Korean medicine doctors and clinical skills examinations are beginning at the College of Korean Medicine, studies on educating various clinical skills and designing and validating OSCE are needed. This study suggested a validated musculoskeletal ultrasound OSCE checklist and rubric to train senior students of Korean medicine for their future use of musculoskeletal ultrasound after graduation. In addition, we described the process and results of developing and validating questionnaires about training and self-evaluation questions to investigate the opinions of students. The quality of education could be improved by using the results of this study for ultrasound training and OSCE for various diseases and anatomical structures in the future.

Supplementary Materials

Supplementary is available at doi:https://doi.org/10.13045/jar.2022.00038.


EC thanks Jung-Han Lee, O Sang Kwon, Jiseong Hong for advice on OSCE checklist and questionnaire.


Conflicts of Interest

The authors declare that they have no competing interests.


This work was supported by a National Research Foundation of Korea grant funded by the Korea government (no.: 2022R1C1C200873811). This research was also supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute, funded by the Ministry of Health and Welfare, Republic of Korea (grant no.: HF20C0173)

Ethical Statement

This research did not involve any human or animal experiments.

Data Availability

All relevant data are included in this manuscript.

Fig. 1
Students of the College of Korean medicine training ultrasound in clinical skills center, and a preceptor observing the students’ performances and providing feedback.
Table 1
Learning Objectives of Musculoskeletal Ultrasound Training, Focusing on Scanning the Volar Wrist Area, and Diagnosis of Carpal Tunnel Syndrome.
Learning outcomes
1. I can use an ultrasonic device.
 1) I can guide the patient into the correct posture before conducting the ultrasound examination and explain thoroughly to them about the upcoming examinations.
 2) I can select the probe suitable for the anatomical structure during the examination and also use it appropriately.
 3) I understand the principles of ultrasonic devices and can use each button appropriately while conducting the examination.
 4) I can make quantitative evaluation and diagnosis of a patient’s condition using various functions of ultrasound.
2. I can examine structures of the body with ultrasound and evaluate their clinical implications.
 1) I can explain the anatomical structure of the wrist anterior aspect.
 2) I can examine the wrist anterior aspect through an ultrasonic device and describe each structure in the clip.
 3)I can understand the diagnosis of carpal tunnel syndrome using ultrasound and explain the test results to the patient.
Table 2
Characteristics of the 8 Experts on the Panel.
Expert panel Age Expertise Experience with ultrasound Experience with education
1 50’s Korean medicine Used ultrasound devices in clinic > 25 y Lectured on the musculoskeletal system and abdominal clinical ultrasound education for > 5 y
2 40’s Acupuncture and moxibustion medicine Published ultrasound-related clinical research papers Lectured on acupuncture and moxibustion medicine at the College of Korean medicine for > 15 y
3 40’s Rehabilitation of Korean medicine Published ultrasound-related clinical research papers Lectured on rehabilitation of Korean medicine, Chuna manual medicine, and clinical practice at the College of Korean medicine for > 10 y
4 40’s Korean medicine diagnostics Published papers on ultrasound Lectured on diagnostic practice at the College of Korean medicine
5 50’s Meridian and acupoint Used ultrasound in teaching for > 5 y Lectured on meridian and acupoint practice at the College of Koran medicine for > 20 y
6 40’s Meridian and acupoint Developed an ultrasonic moxibustion device and illustrated an ultrasonic diagram Lectured on meridian and acupoint at the College of Korean medicine for > 5 y
7 40’s Education policy None Investigated various educational policies (national examination, evaluation and accreditation of Korean medicine education, license system, education policy)
8 40’s Pedagogy None Investigated various studies on competencies and education of Korean medicine
Table 3
Analysis of the Expert Opinions on the OSCE Draft Checklist and Rubric for Musculoskeletal Ultrasound Focused on Volar Wrist and Carpal Tunnel Syndrome.
No. OSCE checklist, rubric (score) Average Median Mode CVR
1 Was patient identification (name, date of birth) performed? (1) 4 4 4 1
1-R 1 point: Performed
0 points: Not performed
3.625 4 4 0.75
2 Did the student explain the ultrasound examination to the patient? (1) 4 4 4 1
2-R 1 point: The student explained the purpose, necessity, time required, and possible side effects of the ultrasound examination
0 points: The student didn’t explain about the ultrasound examination
4 4 4 1
3 Was hand hygiene performed with hand sanitizers before the examination? (2) 3.875 4 4 1
3-R 2 points: Hand hygiene was performed according to the 6 steps of hand hygiene
1 point: Hand hygiene was performed, but only some steps out of the 6 hand hygiene steps were followed
0 points: Was not performed
3.75 4 4 1
4 Was guidance into the correct posture for the examination provided to the patient? (2) 4 4 4 1
4-R 2 points: The student guided the patient to supinate the forearm and extend the wrist at about 45 degrees
1 point: The student encouraged the patient into the correct posture, but it was not suitable for the examination
0 points: Not performed
3.75 4 4 1
5 Did the student choose an appropriate probe for scanning the body part? (1) 3.875 4 4 1
5-R 1 point: The student set an appropriate probe for the wrist examination
0 points: Not performed
3.5 3.5 4 1
6 Was the student proficient in using the probe? (2) 3.125 3 4 0.5
6-R 2 points: The student could adjust the direction and position of the probe to identify the required anatomical structures
1 point: The student could adjust the direction and position of the probe, but was not proficient in using it
0 points: The student failed to use the probe in a proper way
3.5 3.5 4 1
7 Was the examination conducted after the gel was sufficiently applied to the lens surface of the probe? (1) 3.625 4 4 1
7-R 1 point: Enough gel was applied
0 points: Gel was not applied
3.375 3 3 1
8 Were the following structures identified and described? (5) 3.75 4 4 1
8-1 Scaphoid, Pisiform (1) 3.875 4 4 1
8-2 Median nerve (1) 3.625 4 4 0.75
8-3 Ulnar artery (1) 3.625 4 4 0.75
8-4 Flexor retinaculum (1) 3.875 4 4 1
8-5 Flexor pollicis longus (1) 3.875 4 4 1
8-R 1 point: Both identification and description were correct
0 points: Identification or description was incorrect (2 structures under 1 item have to be performed both accurately for the point)
3.5 4 4 0.75
9 Measuring the structure of the proximal carpal tunnel and explaining the result correctly (3) 3.625 4 4 0.75
9-1 Did the student successfully connect the top of the scaphoid and the pisiform with a line? (1) 3.875 4 4 1
9-2 Did the student measure the vertical distance between the line connecting the top of the scaphoid and the pisiform and the flexor retinaculum? (1) 3.75 4 4 1
9-3 Did the student accurately explain the examination results? (1) 3.5 4 4 0.75
10 Measuring the median nerve’s cross-sectional area proximal to the Flexor retinaculum and explaining the results (3) 3.375 3.5 4 0.75
10-1 Did the student properly measure the cross-sectional area of the median nerve proximal to the flexor retinaculum? (2) 3.625 4 4 1
10-R 2 points: The student measured the cross-sectional area of the median nerve proximal to the flexor retinaculum
1 point: The student measured the cross-sectional area of the median nerve proximal to the flexor retinaculum, but the measurement was inaccurate
1 point: The student measured the cross-sectional area of the median nerve, but the area to be scanned was incorrect
0 points: Both the area to be scanned and the cross-sectional area of the median nerve were inaccurate
3.5 4 4 0.75
10-2 Did the student accurately explain the examination results? (1) 3.625 4 4 0.75
11 Did the student inform the patient about the examination’s completion and explain the final findings after the examination? (2) 4 4 4 1
11-R 2 points: Both tasks were completed
1 point: Only 1 task was completed
0 points: Both were not performed
3.625 4 4 1

CVR, content validity ratio; OSCE, objective structured clinical examination; R, rubric.

Table 4
Analysis of Expert Opinions on the Draft for the Questionnaire on Ultrasound Training.
No. Survey questions Average Median Mode CVR
E-1 Have you been trained on ultrasound before? 3.5 3.5 4 1
1 Do you have first-hand experience of using diagnostic ultrasound before this training? 3.75 4 4 1

E-2 Do you think that the topic of ultrasound education (identifying anterior wrist structures and diagnosis of carpal tunnel syndrome) was appropriate? 3.5 4 4 1
2 Were the learning objectives of ultrasound training appropriate? 3.625 4 4 0.75
3 Were the learning objectives and training contents well related? 3.75 4 4 0.75
4 Was it easy to understand the contents of ultrasound training? 3.625 4 4 0.75
5 Has ultrasound training improved your knowledge about the anatomy and physiology of carpal tunnels? 3.875 4 4 1

E-3 Do you want more time for ultrasound practice than this one? 3.375 4 4 0.5

E-4 (only for those who answered positively on E-3 to be surveyed) Please fill out the desired training time. (Subjective question)
Example: ○ times per week, ○ weeks, or ○ semesters
3.25 3.5 4 0.75
6 Did the ultrasound practice help you develop ultrasound skills? 3.625 4 4 0.75
7 Did the ultrasound OSCE checklist reflect the training? 3.625 4 4 0.75
8 Was OSCE useful in evaluating your skills? 3.625 4 4 0.75
9 Was the ultrasound OSCE’s difficulty high? 3.5 4 4 0.75
10 Was the OSCE time limit (5 min) enough? 3.5 4 4 0.75
11 Are you generally satisfied with this ultrasound training? 4 4 4 1
12 Do you have any intentions of studying ultrasound more in the future? 3.875 4 4 1
13 (only for those who answered positively on 12 to be surveyed) Please fill out the human body parts that you want to learn more about, through ultrasound education, in addition to wrist. (Subjective question) 3.5 4 4 0.75
14 Are you willing to use ultrasound in clinical practice after graduation? 3.375 4 4 0.75
15 Has your opinion about ultrasound usage changed positively through this education? 3.625 4 4 0.75

CVR, content validity ratio; E, excluded; OSCE, objective structured clinical examination.

Table 5
Analysis of the Expert Opinions on the Draft Questionnaire for Self-Evaluation After Ultrasound Training.
No. Survey questions Average Median Mode CVR
1 I actively participated in ultrasound training. 3.875 4 4 1
2 I was considerate of the subject during the ultrasound examination. 3.625 4 4 0.75
3 I communicated appropriately with the subject during the ultrasound examination. 3.875 4 4 1
4 I can use basic functions of ultrasound such as ultrasonic probe selection, depth and focus control, and freeze. 3.75 4 4 1
5 I can identify the anterior structure of the wrist through ultrasound. 3.75 4 4 1
6 I can do ultrasound scans according to the criteria for diagnosing carpal tunnel syndrome. 3.375 4 4 0.75

The draft version and the final version are the same.

CVR, content validity ratio.


1. Harden RM, Lilley P, Patricio M. The definitive guide to the OSCE: The Objective Structured Clinical Examination as a performance assessment. Edinburgh (UK): Elsevier. 2016:13–16, 130–134, 213–322.

2. Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. BMJ. 1975;1:447–451.
crossref pmid pmc
3. Park H-K. The impact of introducing the Korean Medical Licensing Examination clinical skills assessment on medical education. J Korean Med Assoc. 2012;55:116–123. [in Korean].
4. Institute of Korean Medicine Educationand Evaluation. The manual of 2nd cycle Korean medicine education evaluation and certification for College of Korean Medicine and Graduate School of Korean Medicine. Seoul (Korea): Karammunhwasa. 2017:9.

5. Institute of Korean Medicine Education and Evaluation [Internet]. Status of evaluation and certification by year. [cited 2022 Jan 31]. Available from: https://www.ikmee.or.kr/html_2016/02/03.php [in Korean].

6. Institute of Korean Medicine Education and Evaluation. KAS2021 Amendment Announcement Public Hearing. Seoul (Korea): Institute of Korean Medicine Education Clinical Performance Examination Evaluation. 2021:14–15.

7. Cho E, Lee J-H, Kwon OS, Hong J, Cho NG. Assessment of Acupuncture and Moxibustion Medicine Clinical Practice Using the Objective Structured Clinical Examination. J Acupunct Res. 2021;38:219–226.
8. Akom news [Internet]. Preparing a ‘place of communication’ on current issues of Korean medicine education such as unification and reorganization of licensing examination. 2019. [cited 2021 Dec 6]. Available from: http://www.akomnews.com/bbs/board.php?bo_table=news&wr_id=36073[in Korean].

9. Kim JH. The effects and challenges of clinical skills assessment in the Korean medical license examination. Korean Med Educ Rev. 2013;15:136–143. [in Korean].

10. Sim SB, Kweon JH, Kim HW, Hong JW, Shin SW. Student Satisfaction Study of Clinical Skills Training in Korean Medical Education. J Korean Med. 2013;34:37–53. [in Korean].
11. Kang YS, Chae YB, Go HY, Kim HH, Kim HS, Hong JS et al. A study for improvement of the national examination system for Korean medicine doctors. Seoul (Korea): Korea Health Personnel Licensing Examination Institute. 2017:80–81. Report No.: RE02-1802-1803 [in Korean].

12. Go HI, Cho E, Jung H-J, Lee S, Kim J-H, Kwon OS. What is the source of pressure pain during abdominal examination in Korean medicine? Integr Med Res. 2020;9:100406.
crossref pmid pmc
13. Chu H, Kim J, Ha W, Cho E, Kang G, Park S, et al. Change of Safe Needling Depth at Acupoint GB21 according to Posture and Breathing. Evid Based Complement Alternat Med. 2018;2018:2308102.
crossref pmid pmc
14. Chu H, Park K, Jang J, Shin H-R, Yang M, Kim DR, et al. Comparison of Cervical Hyeopcheok Acupoint’s Depth Change with Cervical Rotation in a Prone Position: Protocol and Pilot Study Result for Accurate Observation of Cervical Hyeopcheok Acupoint by Ultrasonography. J Korean Med Rehabil. 2019;29:81–88. [in Korean].
15. Ahn T-S, Moon J-H, Park C-Y, Oh M-J, Choi Y-M. The Effectiveness of Ultrasound-Guided Essential Bee Venom Pharmacopuncture Combined with Integrative Korean Medical Treatment for Rib Fracture: A Case Study. J Korean Med Rehabil. 2019;29:157–163. [in Korean].
16. Kim S, Lee S, Ha W-B, Lee J-H, Jung H-J, Chu H-M, et al. Development of an ultrasound-imaging procedure and acquisition of ultrasound images of acupuncture points for safety and accuracy of needle insertion. Integr Med Res. 2017;6:427–433.
crossref pmid pmc
17. Yoon S-H, Kim S-A, Lee G-Y, Kim H, Lee J-H, Leem J. Using magnetic resonance imaging to measure the depth of acupotomy points in the lumbar spine: A retrospective study. Integr Med Res. 2021;10:100679.
crossref pmid pmc
18. Diep D, Chen KJQ, Kumbhare D. Ultrasound-guided interventional procedures for myofascial trigger points: a systematic review. Reg Anesth Pain Med. 2021;46:73–80.
crossref pmid
19. Korean Medicine Diagnostic Practice Compilation Committee. Practice in diagnostics of Korean medicine. 2nd ed. Seoul (Korea): Koonja Publish Co. 2017:51–54. [in Korean].

20. Editing Textbook of the Acupuncture. Practice of Acupuncture; A Guidebook for College Students. 7th ed. Daejeon (Korea): Jongryeonamoo. 2015:5–610. [in Korean].

21. Yang S-J, Park K-M, Cho S-H. The analysis of satisfaction with clinical training and the related factors in the Korean medicine obstetrics and gynecology. J Korean Obstet Gynecol. 2020;33:116–127. [in Korean].

22. Musculoskeletal sonography examination - MSKS / ARDMS [Internet]. ARDMS, 2021. [cited 2021 Dec 23]. Available from: https://www.ardms.org/get-certified/rmsks/musculoskeletal-sonographer-msks/ .

23. Chung SH, Lee HJ, Kim HS, Oh J-Y. Health Insurance Benefit Criteria and Quality Assurance Policies of Diagnostic Ultrasound Services in Other Countries. Health Policy Manag. 2014;24:109–119. [in Korean].
24. Lee S-H, Park H-J, Kim H-T, Park S-Y, Heo I, Hwang M-S, et al. Clinical Practice Patterns for Carpal Tunnel Syndrome in Korean Medicine: An Online Survey. Korean Soc Chuna Man Med Spine Nerves. 2021;16:73–89. [in Korean].
25. Bianchi S, Martinoli C. Ultrasound of the musculoskeletal system. Seoul (Korea): Epublic. 2009:21–27, 46485598100448[in Korean].

26. Association of Pain and Function Analysis Research group of ultrasound. Muscular ultrasound for pain physician. Seoul (Korea): Medianbook. 2019:6–9. [in Korean].

27. Han JH, Choi SY, Jung SY, Jang YI, Lim CS, Lee HS. Ultrasonography Imaging. 2nd ed. Seoul (Korea): MDWorld. 2013:11, 15, [in Korean].

28. Korean Society of Musculoskeletal Radiology. Musculoskeletal ultrasound. 2nd ed. Paju (Korea): Koonja publish Co. 2018:5–10, 14–15, 494591[in Korean].

29. Korean Orthopedics Ultrasound Society. Orthopaedic ultrasound. Paju (Korea): Koonja Publish Co. 2019:78, [in Korean].

30. Wong SM, Griffith JF, Hui ACF, Lo SK, Fu M, Wong KS. Carpal Tunnel Syndrome: Diagnostic Usefulness of Sonography. Radiology. 2004;232:93–99.
crossref pmid
31. Baltarowich OH, Di Salvo DN, Scoutt LM, Brown DL, Cox CW, DiPietro MA, et al. National Ultrasound Curriculum for Medical Students. Ultrasound Q. 2014;30:13–19.
crossref pmid
32. Knobe M, Münker R, Sellei RM, Holschen M, Mooij SC, Schmidt-Rohlfing B, et al. Peer teaching: a randomised controlled trial using student-teachers to teach musculoskeletal ultrasound. Med Educ. 2010;44:148–155.
crossref pmid
33. Kessler C, Bhandarkar S. Ultrasound training for medical students and internal medicine residents-A needs assessment. J Clin Ultrasound. 2010;38:401–408.
crossref pmid
34. Ma I, Wishart I, Kaminska M, McLaughlin K, Weeks S, Lautner D, et al. Medical educators’ perspectives of teaching physical examinations using ultrasonography at the undergraduate level. Can Med Educ J. 2013;4:e59–68.
crossref pmid pmc
35. Gulati G, Alweis R, George D. Musculoskeletal ultrasound in internal medicine residency - a feasibility study. J Community Hosp Intern Med Perspect. 2015;5:27481.
crossref pmid pmc
36. Kyunghee University College of Korean Medicine Clinical Skills Compilation Committee. Korean medicine clinical skills OSCE. Seoul (Korea): Korea Medical Book Publishing Company. 2019:12–20. [in Korean].

37. Pacific Academy of Sciences 5 minutes OSCE. 5th ed. Seoul (Korea): Pacific Books. 2018:303–309. [in Korean].

38. Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28:563–575.
39. Lee E-H. Psychometric property of an instrument 1: content validity. Korean J Women Health Nurs. 2021;27:10–13. [in Korean].
crossref pmid pmc
40. Moore CL, Copel JA. Point-of-Care Ultrasonography. N Engl J Med. 2011;364:749–757.
crossref pmid
41. Tarique U, Tang B, Singh M, Kulasegaram KM, Ailon J. Ultrasound Curricula in Undergraduate Medical Education: A Scoping Review: Ultrasound Curricula in Undergraduate Medical Education. J Ultrasound Med. 2018;37:69–82.
crossref pmid
42. Kim BK, Cha JM, Song DS. How to Revitalize the Abdominal Ultrasonography Education Program. Korean J Gastroenterol. 2019;73:66–69. [in Korean].
43. Criscione-Schreiber LG, Sloane RJ, Hawley J, Jonas BL, O’Rourke KS, Bolster MB. Expert Panel Consensus on Assessment Checklists for a Rheumatology Objective Structured Clinical Examination: Consensus on Rheumatology OSCE Assessment Checklists. Arthritis Care Res. 2015;67:898–904.
44. Patrawalla P, Eisen LA, Shiloh A, Shah BJ, Savenkov O, Wise W, et al. Development and Validation of an Assessment Tool for Competency in Critical Care Ultrasound. J Grad Med Educ. 2015;7:567–573.
crossref pmid pmc
45. Millington SJ, Arntfield RT, Guo RJ, Koenig S, Kory P, Noble V, et al. The Assessment of Competency in Thoracic Sonography (ACTS) scale: validation of a tool for point-of-care ultrasound. Crit Ultrasound J. 2017;9:25.
crossref pmid pmc
46. Wright SA, Bell AL. Enhancement of undergraduate rheumatology teaching through the use of musculoskeletal ultrasound. Rheumatology. 2008;47:1564–1566.
crossref pmid
47. Kissin EY, Niu J, Balint P, Bong D, Evangelisto A, Goyal J, et al. Musculoskeletal Ultrasound Training and Competency Assessment Program for Rheumatology Fellows. J Ultrasound Med. 2013;32:1735–1743.
crossref pmid
48. Kim H, Jeon P, Kim S, Hong J, Kang Y. Cross-Cultural Adaptation and Validation of the Korean Version of the Dundee Ready Education Environment Measure (DREEM). Edited by Adsuar JC. In: Evid Based Complement Alternat Med. 2021:2021:5591911.
crossref pmid pmc
49. Joo S, Eom D, Song J. A research for recognitions of Korean medical student toward traditional Korean medicine and education. J Soc Prev Korean Med. 2016;20:1–8. [in Korean].

50. Park S, Choi E, Kim S-Y. A Study on the Korean Medical Students’ Perception of Medical Ethics Education Using Flipped Learning. J Soc Prev Korean Med. 2018;22:1–12. [in Korean].

51. Kwon SW, Shin SW, Lim B. A survey of students’ satisfaction with education in traditional Korean medicine. J Korean Orient Med. 2012;33:1–11. [in Korean].

52. Jo H-J, Park J-S, Roh JD, Sung HK. A Survey on Students’ Perception of Clinical Performance Examination (CPX) in College of Korean Medicine Using Student Standardized Patients. J Soc Prev Korean Med. 2020;24:1–13. [in Korean].

53. Kweon JH, Sim SB, Kim EJ, Hong JW, Shin SW. Evaluation of the Implementation of Clinical Performance Examination in Korean Medicine Education. J Physiol Pathol Korean Med. 2018;32:51–61. [in Korean].
54. Cha HY, Kim NH, Hong JW, Shin SW. Evaluation of the implementation of problem-based learning in Korean medical education. Korean J Orient Physiol Pathol. 2012;26:351–359. [in Korean].

55. Cho CS. Student Satisfaction Study and Interrater Comparative Study on Patient-Physician Interaction Score of Clinical Performance Examination in Korean Medical Education. Korean J Orient Physiol Pathol. 2015;29:152, [in Korean].
56. Yang S-B. Survey research for evaluation of real-time non-face-to-face medical education in clinical Korean medicine. J Intern Korean Med. 2020;41:984–992. [in Korean].
57. Cho HW, Hwang EH, Shin BC, Sul JU, Hong JW, Shin SW, et al. The analysis of satisfaction with clinical training and the related factors-especially in oriental rehabilitation medicine-. Korea J Orient Prev Med Soc. 2012;16:1–15. [in Korean].

58. Chang Y-J, Hong S-U. The Efficiency of Objective Structured Clinical Examination (OSCE) as a Method of Clinical Clerkship in the Korean Ophthalmology and Otolaryngology Department. J Korean Med Ophthalmol Otolaryngol Dermatol. 2014;27:121–130.
59. Lim KS, Lee JC, Park YG. The result of oriental medical ophthalmology and otolaryngology class applying student-oriented teaching and learning method. J Korean Med Ophthalmol Otolaryngol Dermatol. 2010;23:163–173. [in Korean].

60. Rowe G, Wright G. [Internet]. Expert Opinions in Forecasting: The Role of the Delphi Technique. 2001. [cited 2021 Dec 16]. Available from: http://link.springer.com/10.1007/978-0-306-47630-3_7 .

Share :
Facebook Twitter Linked In Google+
METRICS Graph View
  • 2 Crossref
  • 1,732 View
  • 41 Download
Related articles in JAR

Article and Issues
For this Journal
For Authors
Submit Manuscript
Editorial Office
Gil Korean Medical Hospital, Gachon University
Keunumul-Ro, Chung-Ku, Inchoen 22138, Korea
Tel: +82-70-7606-6353,4    Fax: +82-32-232-3334    E-mail: jared@e-jar.org                

Copyright © 2022 by Korean Acupuncture & Moxibustion Medicine Society.

Developed in M2PI

Close layer
prev next