Informator uniwersytecki
numer 019
Luty 2023
★
2
A few words about ACEM - how flagship exams are held
Magdalena Mielnik from the Department of Didactics and Medical Simulation of the Medical University of Lublin told about how ACEM works and how flagship exams are held.
What exactly is ACEM? How was it created and what does the abbreviation mean?
Created in 2019, the system was initially called CMET (Centre for Medical Test Exams) and was used only for examining students using computer tests modeled on LEK exams. Over time, the system was expanded with additional modules related to checking students' knowledge or supporting teaching, as a result of which, from September 2021, the CMET platform, referring only to tests, was renamed to: ACEM (Academic Center for Medical Examinations).
Created in 2019, the system was initially called CMET (Centre for Medical Test Exams) and was used only for examining students using computer tests modeled on LEK exams. Over time, the system was expanded with additional modules related to checking students' knowledge or supporting teaching, as a result of which, from September 2021, the CMET platform, referring only to tests, was renamed to: ACEM (Academic Center for Medical Examinations).
The ACEM computer-based testing module allows you to conduct test exams based on closed questions with one correct answer option. The tests are created from questions parameterized in several levels, stored in gradually created databases dedicated to individual units of the Medical University and subjects. Each question entered into the system undergoes a one- or two-stage verification before it can be allowed to create tests. Test sets are generated automatically (randomly) from the questions available in the database based on the given guidelines (i.e. thematic category, degree of difficulty, frequency of occurrence of the case).
What kind of test exams are conducted?
Currently, all exams indicated as "flagship" for a given field of study are carried out in the computer-based testing system. The word "flagship" should be understood here as high-profile exams, final exams in a given subject or module, important from the point of view of the student's future professional professionalism.
How is ACEM developing?
The activities of ACEM are supervised by the Coordinator of the University Team for the Academic Center for Medical Examinations, prof. dr hab. n. med. Renata Chałas, holding this position since 2020. During these two years, the number of exams in the ACEM system has almost doubled.
Due to the great interest in conducting computer-based exams, efforts have been made to increase the resources of electronic equipment necessary for this type of exams. From 2020, 150 tablets were made available to ACEM, which significantly increased the existing capacity of ACEM, which at that time comprised 114 computer places. In the current academic year 2022/23, this number has been increased by the purchase of another 260 tablets, which will soon be available to ACEM.
How many majors use the ACEM module - computer tests?
Each Faculty of the Medical University of Lublin that educates students has flag exams planned in the ACEM system.
At the Faculty of Medicine, these are exams for students of the 6th year of the Faculty of Medicine from 7 modules of Practical Clinical Teaching (internal medicine, paediatrics, psychiatry, family medicine, emergency medicine, gynecology, surgery) for both Polish and English-speaking students. In addition, from 2021, an exam in oncology was introduced to the group of subjects ending with the flag exam.
There are currently eight of them at the Faculty of Medicine and Dentistry: maxillofacial surgery, dental surgery, conservative dentistry with endodontics, pediatric dentistry, dental prosthetics, orthodontics, periodontology and diseases of the oral mucosa. From 2020 ACEM exams are also conducted for English-speaking students.
At the Faculty of Health Sciences, flag exams are held in 4 fields: Obstetrics I degree, Nursing I and II degree, Physiotherapy and Public Health II degree.
At the Faculty of Pharmacy, the following faculties use the ACEM system: Pharmacy and Medical Analytics.
The Faculty of Medical Sciences has one flag exam in the field of First degree Medical Rescue.
On the other hand, the Faculty of Biomedical, established in 2021, conducts flag exams in the field of Dietetics 1st and 2nd degree.
How many exams are held under ACEM?
Since its inception, the number of flag exams conducted on the ACEM system has been constantly growing with each subsequent academic year:
- academic year 2019/2020: 30 flag exams in 9 subjects/specialties,
- academic year 2020/2021: 46 flag exams in 28 subjects/specialties,
- academic year 2021/2022: 62 flag exams in 48 subjects/specialties,
- academic year 2022/2023: 83 flag exams in 58 subjects/specialties.
What is standardization in ACEM?
First of all, it is a database of standardized (parameterized) tasks for a given exam/examination subject, checking the fulfillment of the requirements resulting from the relevant core curriculum. Tasks from the task database can be used to create any exam tests according to a predetermined algorithm. The rule is that all candidates in a given examination round receive the same set of questions.
The standardization of the methods of examining students in the ACEM system also includes uniform and general rules for conducting the test included in the ACEM Regulations.
How are exam statistics analyzed and what do they reveal?
The advantage of conducting test exams in the ACEM system is the possibility of analyzing the results, which is a condition for improving the quality and an indicator of the effectiveness of teaching.
In addition to the basic parameters determining the average grade, we have the ability to verify statistical parameters indicating what are the measures of dispersion of student results, whether a given group is uniform in terms of skills. The assessment of the reliability of the test, using the Cronbach's alpha index, shows the connections between individual questions within the set of questions. Knowing the reliability of the test is important because it is the basis for evaluating the quality of our measurement tool. Data on students' answers, depending on the degree of classification of questions in terms of difficulty, give a feedback on mastering knowledge at a given level
Taking care of the consistency and high reliability of the exam is the key to ensuring an objective and fair assessment of students' achievements. Thanks to this, we can also diagnose didactic weaknesses. We have a chance to improve our workshop, as the author of test questions should be particularly interested in how the test tasks that we have created work in practice. In addition, an important advantage of test exams is the possibility of improving the tool in subsequent exam sessions.
Each Faculty of the Medical University of Lublin that educates students has flag exams planned in the ACEM system.
At the Faculty of Medicine, these are exams for students of the 6th year of the Faculty of Medicine from 7 modules of Practical Clinical Teaching (internal medicine, paediatrics, psychiatry, family medicine, emergency medicine, gynecology, surgery) for both Polish and English-speaking students. In addition, from 2021, an exam in oncology was introduced to the group of subjects ending with the flag exam.
There are currently eight of them at the Faculty of Medicine and Dentistry: maxillofacial surgery, dental surgery, conservative dentistry with endodontics, pediatric dentistry, dental prosthetics, orthodontics, periodontology and diseases of the oral mucosa. From 2020 ACEM exams are also conducted for English-speaking students.
At the Faculty of Health Sciences, flag exams are held in 4 fields: Obstetrics I degree, Nursing I and II degree, Physiotherapy and Public Health II degree.
At the Faculty of Pharmacy, the following faculties use the ACEM system: Pharmacy and Medical Analytics.
The Faculty of Medical Sciences has one flag exam in the field of First degree Medical Rescue.
On the other hand, the Faculty of Biomedical, established in 2021, conducts flag exams in the field of Dietetics 1st and 2nd degree.
How many exams are held under ACEM?
Since its inception, the number of flag exams conducted on the ACEM system has been constantly growing with each subsequent academic year:
- academic year 2019/2020: 30 flag exams in 9 subjects/specialties,
- academic year 2020/2021: 46 flag exams in 28 subjects/specialties,
- academic year 2021/2022: 62 flag exams in 48 subjects/specialties,
- academic year 2022/2023: 83 flag exams in 58 subjects/specialties.
What is standardization in ACEM?
First of all, it is a database of standardized (parameterized) tasks for a given exam/examination subject, checking the fulfillment of the requirements resulting from the relevant core curriculum. Tasks from the task database can be used to create any exam tests according to a predetermined algorithm. The rule is that all candidates in a given examination round receive the same set of questions.
The standardization of the methods of examining students in the ACEM system also includes uniform and general rules for conducting the test included in the ACEM Regulations.
How are exam statistics analyzed and what do they reveal?
The advantage of conducting test exams in the ACEM system is the possibility of analyzing the results, which is a condition for improving the quality and an indicator of the effectiveness of teaching.
In addition to the basic parameters determining the average grade, we have the ability to verify statistical parameters indicating what are the measures of dispersion of student results, whether a given group is uniform in terms of skills. The assessment of the reliability of the test, using the Cronbach's alpha index, shows the connections between individual questions within the set of questions. Knowing the reliability of the test is important because it is the basis for evaluating the quality of our measurement tool. Data on students' answers, depending on the degree of classification of questions in terms of difficulty, give a feedback on mastering knowledge at a given level
Taking care of the consistency and high reliability of the exam is the key to ensuring an objective and fair assessment of students' achievements. Thanks to this, we can also diagnose didactic weaknesses. We have a chance to improve our workshop, as the author of test questions should be particularly interested in how the test tasks that we have created work in practice. In addition, an important advantage of test exams is the possibility of improving the tool in subsequent exam sessions.
Magdalena Mielnik,
Chair and Department of Didactics and Medical Simulation of the Medical University of Lublin
Interviewer: Aleksandra Kaźmierczak
Interviewer: Aleksandra Kaźmierczak
Redakcja zajmuje się opracowaniem dostarczonych materiałów do artykułów i nie odpowiada za ich wartość merytoryczną. Za wartość merytoryczną odpowiadają ich twórcy.
© 2022 Centrum Symulacji Medycznej UM w Lublinie