Book (2022)/Hofstee Method

In principle, both absolute and relative standardization have their limitations. Hofstee, for example, drew attention to this some time ago, when prof. Wijnen from Maastricht University proposed an interesting method of relative standardization (Wijnen's method): Wijnen assumed that the average student would try his best and should pass, so we can take the average test score as a starting parameter. As a cutoff score, we can then use an arbitrarily chosen value between this average score and its value reduced by two standard deviations. The advantage of this solution is that it corrects the effect of the unreliability of the test because it uses the standard deviation as a measure. Prof. Hofstee was not satisfied with this method, arguing that the procedure does not take into account the absolute results of the test and, like any relative standardization, condemns a more or less fixed percentage of students to failure in advance, regardless of their performance.

Hofstee Method

Hofstee therefore proposed mixed standardization offering a compromise between absolute and relative standardization. In this method, the cut-off score for passing the test is set using expert estimates.

It is assumed that each of the experts is thoroughly familiar with:

• The test
• The nature of the tested group
• The expected level of knowledge of the examinees

Experts must answer two questions:

• In what range should the number of students who fail a given test be (e.g.: 10-30% of students from a given group should not pass this test)?
• In what range should the minimum score for passing the given test be (eg: The minimum for passing this test should be somewhere between 50 and 60%)?

Analogously to absolute standardization, experts are therefore asked about the threshold of success and at the same time, similar to relative standardization, they are asked about the desired percentage of success. After discussing the suggested values, where the experts can still modify their suggestions, we get 4 values:

• the minimum and maximum permissible proportion of failures, fmin and fmax,
• the minimum and maximum allowable limits of success, kmin and kmax.

All four values are determined as medians of individual experts' suggestions.

Fig. 5.5.1 The Hofstee method of determining the cut-off score for passing the test

The pass threshold is determined after scoring the test as follows: Based on the test performed, a distribution curve of the test scores is created. k_min and k_max are plotted on the horizontal axis, f_min and f_max are plotted on the vertical axis. A straight line connecting the intersection of f_max with k_min and the intersection of fmin with k_max is made. The intersection of this straight line with the distribution curve is used as the pass threshold for the test.

The Hofstee method is often classified as a compromise method that tries to resolve the differences between absolute standardization (assessing the percentage of correctly answered items) and relative standardization (assessing the percentage of examinees who passed the test).

The Hofstee method is in many aspects similar to the Beuk method. Both require evaluators to establish cut-off scores directly, without examining individual items, and involve judges' estimates of the performance of the entire group being tested. Both methods need to know the actual distribution of test scores, so they cannot be performed until the test is completed and scored. For both, the necessary expert recommendations can be gathered before taking the exam. Beuk's method, in contrast to Hofstee's method, can propose a cut-off score even if the experts' estimates are higher or lower than the points on the distribution curve of the achieved scores.