Book (2022)/Contrasting Groups Method

Another procedure for determining a defensible threshold score (cutscore) is the method of contrasting groups. Unlike the previous methods, it does not assess individual items of the test, but only works with total point gains.

The assumption of this method is that we have two contrasting groups available. For example, we present the test to a group of beginners (i.e., students) and to an advanced group (for example, people from the field). A curve is created for each group that shows the distribution of the scores obtained. The cut-off point for passing the test is determined as the intersection of the curves of both contrasting groups (Fig. 5.4.3).

Fig. 5.4.3 Finding the cut-off score using the contrasting groups method. (The figure illustrates the use of the contrasting groups method. The dashed normal curve represents the performance of the novice group. The dotted normal curve represents the advanced group. The vertical black line through the intersection of the two curves represents the pass/fail cutoff scores.)

In a real case, especially with small groups of individuals, the curves will not be so smooth and “normal”. Commonly, therefore, the real collected points are interpolated with a smooth curve and the intersection of these interpolated curves is sought.

Implicit assumptions must be met in every cutoff scoring method. While in the Angoff method we implicitly assume that expert judgments are correlated with item difficulty, in the contrast group method we assume that test performance is correlated with another available assessment method. The objective of the contrasting groups method is actually to evaluate how the test results predict some “gold standard” evaluation of the test takers, i.e., the division of students into contrasting groups. This gold standard might be a teacher evaluation or some recognized performance metric. If a suitable "gold standard" cannot be found, other methods must be used - e.g. the bookmark method or the modified Angoff method.