Assessment methods

ASSESSMENT METHODS

Assessment methods are the means to evaluate the knowledge, understanding and performance of a learner at the end of a learning process. One or more methods can be used to assess the achievement of a single LO in a curriculum. Combining several methods ensures a comprehensive evaluation of gained competence.

The choice of assessment methods follows three basic rules:

• Assessment methods need to be aligned to the LO by taking into account the verb in the learning outcome statement. For example, the verb “explain” indicates the need to construct rather than select the answer.
• Assessment methods must be appropriate to the purpose of the curriculum and the LO at curriculum level. This is important if the LO is designed to develop general competences, for example critical thinking, problem solving, cooperation, etc.
• Assessment methods must be linked to educational strategy of the curriculum.

A transparent, reliable and fair process of LO assessment motivates students to learn and develop. There’s not a single and universal catalogue for LO assessment methods. In the following sections a list of the most frequently used methods is presented for each category identified.

Assessment of Interprofessional Competency

Assessment of interprofessional competencies in healthcare has received considerable attention in recent years due to the need for evidence that health and social care practitioners are able to work well together. Interprofessional academic scholars have authored a consensus statement to guide decisions about the selection of a valid and reliable assessment process and tool (Rogers et al, 2027)¹ The principles to be considered when making assessment decisions about interprofessional competence include:

• Assessment of interprofessional competence should include: role understanding; interprofessional communication; interprofessional values; coordination, and collaborative decision making; reflexivity; and teamwork
• Assessment of individual, rather than assessment of the performance of team is recommended for undergraduate learners
• The provision of accurate, timely, feedback to learners on their progress toward achievement of interprofessional competence is critical
• The use of self-assessment tools and scales to measure attitude or perceived confidence is not recommended in summative assessment of interprofessional competency, though some such tools may have a place as part of a portfolio assessment process.

Multiple valid and reliable interprofessional assessment tools have subsequently been proposed, and are summarised by the NEXUS network: https://nexusipe.org/advancing/assessment-evaluation.

One example of a useful tool is the Interprofessional Collaboration Assessment Rubric (ICAR).² The ICAR measures communication, collaboration, roles and responsibilities, collaborative patient-centered care, conflict management and team functioning. There are 31 IPCP competency statements, each of which are rated by an observer, rather than via self-assessment. The ICAR is therefore able to provide learners with detailed feedback on each competency area, to understand areas for further development. As a summative assessment, ICAR may be used to assess learners’ achievement and reliability and validity studies support the usefulness of this tool.

Written exam/essays/assignments

Multiple-choice question test (MCQ)

MCQ test is a reliable method for effectively assessing knowledge and reasoning. This type of test is less susceptible to guessing and easier to score. The questions can be written to evaluate various levels of learning outcome, from basic recall to application and analysis.

A multiple-choice item is composed of the problem called ‘stem’ and a list of suggested solutions, known as alternatives. Alternatives are formed of one correct answer, which is the key, and incorrect options, called distractors. The stem can be formed as a question, short scenario or a case study and expressed as simple as possible. The preferable form is a question. The stem should present a definite problem, without containing irrelevant material which can decrease the reliability and the validity of the test scores. It is better to avoid negative form in the stem because negative phrasing could be a source of misunderstanding unless significant learning outcomes require negative phrasing. In this case the negative form in the stem should be highlighted. Alternatives should be plausible, stated clearly and narrowly, mutually exclusive, homogenous in content and presented in a logical order (e.g., alphabetical or numerical). The alternatives should not contain any clues to the correct answer. The options “all of the above” and “none of the above” should not be used. The number of alternatives can vary among items as long as all alternatives are plausible. Language must be similar in all items, and the placement of the correct answer must vary. Only one best answer should be in the alternatives. Easy and non-functioning distractors should be avoided. MCQ test items should be piloted by someone who can find mistakes, clues, grammar and punctuation problems before implementation.

Essays

Essays have traditionally been a useful method to measure higher-order learning such as abilities to reason, create, analyse, synthesize and evaluate. Essay questions provide the opportunity for more systematic and deeper thinking, including problem-solving and decision-making rather than selecting a solution from limited options. However, with the recent deluge of Generative AI, there is the potential for learners to outsource most long form writing tasks if not completed under exam conditions. Another disadvantage of the essay is that it is difficult to objectively assess, which is time-consuming. As a result, assessment by essay requires careful preparation and scoring.

Essay questions can be divided into restricted or extended response essays. The restricted response essays usually cover a limited part of content of the curriculum. These questions can be designed as problem solving exercises, case studies or scenario-based activities involving clinical data and situations. Extended response essays assess complex learning outcomes that allow the demonstration of reasoning and thinking skills, creativity, integration and evaluation of ideas, construction of arguments, etc.

Reports can be used as extended response essays which provide information about a result of research and analysis of data. The structure of reports reflects the information seeking process and has 6 subsections: summary of the contents, introduction or background, methods, results, discussion and conclusion and/or recommendations. The formulation of essay questions, especially those using case studies, is time-consuming. The essay questions should clearly state the involved learning outcome, define the task and not to allow for various or broad interpretations. The task can be written as a statement or a question. Several relatively short questions are better than one long question. It is advisable to indicate a time limit for answering the question. A limited essay question requires 10 to 15 minutes to answer. A broader question that requires more than a page or two to answer should be given half an hour to an hour.

The essay exams can be assessed using a holistic or analytic approach. In the holistic approach, all answers to a given essay question are first reviewed and then a score is given based on the overall quality of the answer. The responses can be grouped into best, average and poor answers before a score is assigned. A holistic approach helps to avoid predisposition in scoring. Reading all the answers of one person at the same time may lead to a positive bias in the assessment if the first answers are very good or a negative bias if the first answers are poor. A holistic approach is recommended for questions with a variety of acceptable answers. The analytical approach starts by drawing up a list of key elements that should be included in the answer. The essay question is scored according to the number of response items that fit the response model. Regardless of the approach used, narrative feedback should be provided by the assessor to promote learning.

Assignments and projects

Assignments and projects can be an excellent mechanism to assessment teamwork tasks. It is possible to combine a range of modalities to demonstrate student learning. Again, the verbs in the learning outcome direct the type of written assignments or projects that should be created. The verbs that best describe the skill or skills the teacher intends to assess are sometimes called directive verbs. In Figure 2 appropriate testing methods are associated with Bloom’s taxonomy. The verbs under each domain show the kind of activities that a test might assess. These verbs can be used when constructing assignment and project tasks. In addition, some complex learning outcomes can be more effectively assessed with other types of assessment, for example performance assessment, simulation, etc. In most situations written exam is a solitary activity and typically does not assess teamwork skills.

Oral exam

Identifying effective exam questions

The acquisition of a qualification and the assessment of its acquisition is a particularly important factor increasing human competition in the labour market and in a changing world of work. For this reason, much attention is paid to the organization of the assessment of competencies acquired by a person, the design and quality of their tasks.

This methodological process requires clinical tutors to plan the training: the training focuses on the fact that the student understands the importance of a real case discussion, related to an emotionally involved person, is able to identify the elements of a care plan (problems to be identified, goals to be achieved, interventions to be implemented, assessments to be carried out), and is able to apply them to a patient. It is important for the student to gain a significant training experience since the construction phase of the text as it requires the ability to:

• focus on a typical healthcare problem related to the students’ educational goals;
• design a case including a description and additional data;
• contextualize the problems included in the case, directing them to the students’ educational goals.

Example of oral exam forms

Simulation/skills demonstration

Practical application of tasks requires demonstration, either in a simulated setting or in the workplace. When designing tasks for the assessment of the application of acquired competencies, simulation situations can be used to provide opportunity for feedback on learning or summative assessment.

A practical exam assesses whether a person has acquired skills that can be demonstrated in a real or simulated work environment. It should be noted that the same methodological principles apply to the planning of the skills assessment as to the theoretical part of the assessment, except that the skills assessment requires a real, practical task, not just a mental one. For this reason, the practical part of the assessment task requires more than just a theoretical evaluation, identification and calculation, as it also implies performance. The person should also provide an evaluation of their work and explain and justify the decisions they made (why did you do the task one way or another, what would you do differently, etc.). Working groups are important in the preparation of tasks for the members of the assessment of acquired competencies, since they help them have a common understanding of what needs to be evaluated (what is the object of evaluation?) and what are the components of the object of assessment.

Practical skills can be assessed in a simulated or authentic work environment. The quality of the assessment of practical skills therefore depends on the context and authenticity of the professional activity. The context of the activity is not only the technological and technical environment (materials, equipment, devices and tools used) but also the professional organizational and social environment of the activity (functions assigned to the workplace in the general production scheme, the channels of presentation of material and information resources to the workplace, the transfer of work results to another workplace, the authentic professional communication and collaboration within and between workplaces, and the workplace's informational environment in terms of the standards, rules and requirements for the performance of the various activities, etc). The more these elements are present in the assessment process, the more accurately the results of the practical skills assessment will indicate whether the practical skills acquired meet the requirements of the professional activity.

Below some hints to design effective assessment tasks in the practical part of the assessment of the competencies acquired by a person are provided:

Assessment of Work-Based Learning

A variety of methods can be used to assess work-based learning (WBL) achievements, depending on the specific learning outcomes and whether the focus is on the content, the learning process or both. Most often, the competences/skills of learners are assessed by workplace mentors and their ability to link theory and practice. In the following paragraphs are presented some assessment methods for WBL.

Mini-clinical evaluation exercise [mCEX]

Mini-clinical evaluation exercise (mCEX) is an observational tool that assess the leaner’s clinical performance, interaction with a real patient, ability to collect a focused medical history, perform a physical examination, make a diagnosis and propose a treatment plan. This tool takes approximately 15 minutes. A structured evaluation form is used to assess the performance. Constructive feedback must be given to the learner in order to improve the performance. Usually, four to eight mCEX sessions are needed to obtain a reliable image of the trainee’s clinical competence. The assessment should be performed by different assessors using the mini-CEX tool.

Direct observation of procedural skills [DOPS]

Direct observation of procedural skills [DOPS] is as an adaptation of the mCEX to assess procedural skills related to a real patient. The assessment is based on a common rating scale.

Case-based discussion [CbD]

Case based discussion [CbD] is designed to assess the knowledge, diagnostic reasoning, rationale for choosing certain actions and understanding of differential diagnosis. Usually, the learner describes the case and the assessor prepares questions for discussion. The discussion focuses on the case, identifying the learner's clinical reasoning and management skills. Medical records should be available during the discussion. The assessor examines the learner's professional judgement and provides constructive feedback.

Multisource feedback [MSF] and other tools

Multisource feedback [MSF] tools represent a way of systematically collecting and gathering the perspectives of colleagues and patients, so that they can be used both to evaluate performance and to provide feedback to learners at the same time. The test can objectively assess competences such as communication skills, interpersonal skills, collegiality and professional competence.

The most used MSF tool is the mini-Peer Assessment Technique (mini-PAT). Usually, the learner chooses multiple assessors representing a mix of senior supervisors, colleagues, clinic staff and etc. Each assessor completes a mini-PAT questionnaire. The learner also performs a self-assessment using the mini-PAT questionnaire. The combined data from the questionnaire is presented in a way that allows the learner to see his/her own evaluation and compare it with that of peer assessors.

Team assessment of behaviour is used as a formative and summative tool to improve performance. This assessment tool requires a minimum of 10 responses to make a valid and reliable assessment. Patient satisfaction questionnaire is a structured questionnaire designed to obtain patients’ feedback on the trainee’s performance.

Learning journal/learning log

Learning journal/learning log is a useful tool for monitoring and evaluating progress towards learning outcomes. It allows both formative and summative self-assessment.

Assessment based on other data

This category includes all the methods that do not fall under the previous categories, such as the ones related to the use of ICTs (e.g. collection of tracking data and learning analytics) or innovative tools such as e-portfolios or specific functionalities of Virtual Environments.

If the course has an e-learning component, students’ assessment can rely on learning analytics, i.e. data collected by the e-learning platform about learners and their activities; such data can be collected, analysed and reported by a student (student history), by groups of students, by activity/resource, by assessment method, etc. The collection of these data is fundamental above all for course monitoring and formative assessment

An e-portfolio is a collection of digital course-related artefacts, collected and managed by the learner in different forms (essays, posters, photographs, videos); e-Portfolios can also capture other aspects of the learner's life, such as volunteer experiences, employment history, extracurricular activities, and more. It can also be a source of reflection for the learner about the learning process and include also data about the learners’ participation in collaborative activities carried out through chat, forums, etc.

¹ Rogers, G. D., Thistlethwaite, J. E., Anderson, E. S., Abrandt Dahlgren, M., Grymonpre, R. E., Moran, M., & Samarasekera, D. D. (2017). International consensus statement on the assessment of interprofessional learning outcomes. Medical teacher, 39(4), 347-359.

² Curran V, Hollett A, Casimiro LM, Mccarthy P, Banfield V, Hall P, Lackie K, Oandasan I, Simmons B, Wagner S. (2011) Development and validation of the interprofessional collaborator assessment rubric (ICAR). J Interprof Care, 25(5):339-44.