Book/Open-ended Questions

A short form answer item

These are also referred to as items with a short answer, short-answer question, SAQ. The examinee is most often required to answer the item with a short form answer consisting of one word or a phrase. Often the answer is also the result of a calculation, a sketched graph or picture, a chemical formula, a mathematical equation, etc. Depending on the construction, these items are sometimes divided into production and supplementary items:

• Production item:

  What is the name of the capital of Great Britain?

• Supplementary item:

  The capital of Great Britain is __________________.

Short answer questions are an excellent part of formative tests. Their benefit lies in the fact that they provide the teacher with information not only about which parts of the curriculum the students have mastered and to what extent, but at the same time they can learn about possible mistakes, misunderstandings and erroneous concepts from which the students start out. Thanks to this, it is possible to react in a more targeted way when preparing the next lesson and better guide students.

Short-answer questions are also useful even for summative tests. In some areas, they are quite common – for example, in the learning of languages, mathematics and geometry, etc. However, preparing short answer questions for a summative test must be done very carefully in order to avoid ambiguities.

Example:

A properly constructed SAQ:

What is the name of the capital city of Great Britain?

Correct answer: London.

Even in this simple case, the evaluators must agree on how they will grade an examinee’s answer of, for example, “Londra”.

A poorly constructed SAQ:

The capital of Great Britain is __________________ .

In this case, it is not clear what the author of the item is asking: is the capital of Great Britain London, big, in England, historical, on the Thames...

In general, it can be said that there is more than one correct answer to most SAQ items and the answers must usually be evaluated by a qualified evaluator – an expert in the relevant field. Assessors must either agree on which answers should receive full points, which ones should get partial scores, and which answers to consider incorrect, or they must be given detailed instructions.

What is a unit of mass?

Answers can include, for example, kilogram, gram, ton, metric cent, pound, ounce, carat, quart...

What animal is in the coat of arms of Moravia?

Red and silver checkered eagle with golden crown

How to evaluate the omission of a color, or the description of a color combination as red and white? How to evaluate the answer “single-headed eagle”? Is it a mistake to leave out the information about the crown?

Due to the number of possible answers, it may even happen that it is not possible to reliably assess whether the answer is correct.

Give the name of at least one painter.

How many points will you award for a name you don't know at all? Can it be reliably verified that the answer is correct? How much time and effort will such verification cost?

Outside of summative testing, this feature of SAQ can be used. Different answers to the same question can stimulate discussion during the lesson and activate the students. They can also help us create closed-ended SBA items: if you need to find distractors for this kind of a item, ask the same using a SAQ. The answers obtained can provide valuable inspiration.

Fill in the blanks test (cloze deletion test)

In a fill-in-the-blanks test (cloze test), the examinee receives a text with portions left out, which the examinee must complete. Most often, he or she has to fill in the missing words in several different places. In some cases, the examinee is supposed to complete some part of a word.

Example:

Complete the missing text

A plane body that has three sides and three vertices is called a t_____angle. A special category is made up of right _________ le t________les. Their longest side, the so-called hy__________use, compared with the right __________, which is created by the_________.

Supplementary items are widely used in language learning, for example to test vocabulary or to test the ability to understand the spoken word.

Example:

Based on listening, fill in the missing words and data

Earth's only natural satellite is called ________. Its average distance from the center of the earth is _________ km. It orbits the Earth about once every ________. Man stepped on it for the first time in _________.

In this form, supplementary items are actually a bundle of items with a short-form answer. Sometimes similarly constructed items, in which the examinee chooses their answers from a predetermined list of words, or the number of possible answers is limited (e.g. items of the type Complete -its-/-it’s-) are sometimes referred to as fill-in-the-blanks (cloze). In that case, this actually involves only a differently indicated matching assignment, i.e. closed-ended items.

When scoring completion items, a partial score is most often assigned for each correct completion, or a partial PS₅₀ score is used.

Modified essay

A Modified-Essay Question (MEQ), is different type of short-answer problem set. The introductory text is followed by the first question, then supplementary information and other questions alternate.

Example:

A 78-year-old man, a widower who lives alone, came to the in-patient clinic with a complaint of fatigue and weight loss. He was admitted to the general internal medicine department, where you work, for further evaluation.

Question 1: What are the three most likely diagnoses?

Question 2: Write five questions you would ask the patient that would best help you distinguish between the three diagnoses.

Laboratory tests showed mild anemia with a hemoglobin concentration of 104 g/l. The red blood cell count is lower than the reference range. You therefore conclude that the patient suffers from microcytic anemia.

Question 3: Give two typical clinical signs that you will look for when examining the patient.

Question 4: Briefly write how the given result changes your initial diagnosis.

The modified essay is somewhere between short-answer and wide-open questions. They are especially valuable in formative tests, in which they can to some extent simulate a problem-solving dialogue between the student and the teacher. However, preparing this format for summative tests is challenging. In addition to all the requirements and limitations mentioned for short creative items, there is the fact that a mistake at the beginning of drafting a modified essay can also affect the following sub-questions. This would not happen in an orally conducted dialogue, as the teacher would correct the initial mistake in an appropriate way. Of course, it also affects whether the student can go back while completing the test or not.

Note that in a modified essay, individual questions tend to be approached in a much broader perspective than in typical short-answer questions. This time we no longer expect the test taker to answer with a word or phrase. Rather, their answer will consist of several separate statements. The student must not only master the material he or she is being tested on, but must also be able to formulate their answer concisely and precisely in a short time.

Essay

An essay is a item with a long, formulated answer. The examinee writes a text of extended length, ranging from one paragraph to several pages.

The essay usually forms a separate part of the exam, and its evaluation is not combined with other items. The grading may be subject to the subjectivity of the evaluator. Therefore, if an essay is used for summative assessment, it is usually assessed by multiple assessors and, ideally, the essays are anonymized. In order to make the assessment more objective, it is usually assessed according to a predetermined outline, i.e. the extent to which the examinee fulfilled certain assessed aspects (“rubrics”) in the essay is scored. Apart from the knowledge, i.e. the essay content side, the ability to give a well-organized interpretation, analyze and describe contexts, express oneself in a clear, structured and comprehensible manner and use professional terminology correctly, adhere to conventions customary in the field, etc., typically have a great influence on the grade as well.

Using the essay as an assessment tool varies throughout different parts of the world. To some extent, it can be said that the alternative to the essay is the oral exam, during which the examiner conducts a conversation with the examinee. An advantage of the oral exam can be the dialogue, which makes it possible to more accurately identify the examinee’s strengths and weaknesses. However, the disadvantage is the non-reproducibility of the assessment. While the essay can be re-graded at any time by another assessor, the oral exam cannot be repeated. Even if it is audio-visually recorded, repeated assessment can be difficult, especially if the examiner conducted the interview inappropriately or incorrectly at some point. Examining in front of a committee can contribute to increased objectivity, but in practice a larger number of examiners does not automatically mean that they can be considered mutually independent evaluators who do not influence each other during grading. Despite all these reservations, the oral exam has its pedagogical value, thanks to the possibility of basing the act of examination on professional dialogue and interaction. However, it is conditional upon the high erudition and professionalism of the examiner.