Book/Testing Security from a Risk Management Perspective

When assessing the importance (and the rationality of the costs incurred) of individual aspects of testing security, we can use a methodology familiar in the field of risk management. Within the institution and in the context of the upcoming testing, we should think about:

• Identifying assets,
• Asset valuation,
• Identifying threats,
• By estimating the probability of the occurrence of individual threats,
• By estimating the vulnerability of the asset to the threat,
• By estimating the total risk resulting from individual threats to assets.

Asset Identification

Assets are anything that represents value to the organization. Although it may not be obvious from a narrow view of testing, the credibility of the entire institution is among the main assets in education. There are cases where gross errors in the security of the testing process (during the admission procedure) have led to fatal consequences in the form of withdrawal of accreditation.

Asset valuation

When valuing assets, we can estimate, for example, the price of an item bank or its contents. The item bank of the First Faculty of Medicine of the Charles University contains about 10,000 items. The cost of their acquisition was about CZK 1,500 per item. From there we get a price of CZK 15 million for the entire content of the item bank. We can also estimate the cost of the established credibility of the institution. These are values ​​built up over the long term with large costs. If, for example, there were a scandal surrounding the admissions process, then this could mean that PR costs for a period of, say, 5 years were devalued. If the loss of confidence in the regularity of the procedure leads to the withdrawal of accreditation, the faculty will lose income for teaching students, i.e. one of the biggest sources of income, for several years. For a larger faculty, the loss thus reaches hundreds of millions of CZK. However, protected assets also include, for example, the personal data of test participants, which are protected under the GDPR under the threat of draconian fines.

Threat identification

Threats are scenarios in which an organization's assets may be at risk. In testing, this mainly concerns external and internal threats. Among the threats, a special place is occupied by attempts to intentionally influence the results by illegal procedures. There are a number of types of unethical and fraudulent behavior that can compromise the test's informativeness:

Leaking items, or unauthorized acquisition of prior knowledge, can occur if participants in previous runs bring up the wording of questions. Either they memorize the content of the exam, or they copy the question with their mobile phone, or they write it down. The objective is to achieve an advantage in the test over other test takers, thanks to knowledge of specific test questions. Unauthorized access to test questions gives cheaters an unfair advantage over honest test takers.

Unauthorized cooperation. Two or more test takers may attempt to work together on completing a test. For example, copy answers, or share answers during the test via text messages and the like.

Identity confusion. The test's informativeness can be impaired if someone other than the actual candidate takes the test. This “test proxy” can be prevented by maintaining high standards for identity verification. This issue needs to be paid a lot of attention, especially in distance tests, where the options for identity verification are limited.

Unauthorized assistance. The test result can also be distorted by collusion, if the test taker receives help from the staff that organizes or evaluates the tests. Cheating means that the test proctor or test administrator provided unauthorized assistance to the examinee or tampered with the test data or test session in some way. An example of collusion could be when an invigilator allows a test taker to deviate from approved test procedures, gain access to unauthorized resources, or allow the test taker to exceed the approved test completion time. Collusion may also involve tampering with exam records, such as changing an examinee's answers from wrong to right, or adding missing answers.

Prohibited aids and resources. According to a survey among 15-year-old pupils in the Czech Republic, in 2013, the most widespread method of cheating was still the use of cheat sheets, while other, technical, means such as mobile phones trailed behind.

Security threats vary for different types of testing. Paper-based exams may be more prone to copying answers than computer-based exams (especially in the case of adaptive testing), while computer-based testing may be more prone to the use of unauthorized resources. Security policies and procedures should be tailored to suit the type of test.

Every type of security threat must be prepared for:

• An estimate of the probability and potential consequences of each of the possible cases
• Preventive measures to reduce threats
• Follow-up procedures to minimize the impact of extraordinary events.

Although risk assessment is laborious and the reasons for dealing with it may not be apparent at first glance, it serves a fundamental purpose – to help ensure the protection of important values with means that are reasonably commensurate with the protected values.

It is obvious that the prevention and elimination of security threats requires a systematic approach. Therefore, in summative testing of great importance, a test security plan is created, which specifies who should deal with what and when to achieve the necessary level of security. Let's go through such a security plan step by step.