Book/Proctored Testing

There are situations when it is necessary to test candidates who are not all in the same place. In most cases, everyone is required to come to the exam venue. But this may not always be effective, or even feasible.

The classic solution to such a situation is for the examinees to gather in several centers, where the examination board will come to them. The exam can take place simultaneously or at different times in all places. Therefore, parallel test sessions are created, or parallel versions of the test are also used.

A fully qualified examiner may be represented by an invigilator, a proctor. The proctor is not an expert in the area being tested, so he or she cannot be a test evaluator, they cannot provide feedback on the test, and they cannot replace an expert in other roles either. However, they can ensure the environment and conditions for the proper execution of the test.

Proctored testing first gained popularity during the verification of the competence of army officers, when it was necessary to test people scattered around the world, or with international language tests. In a typical arrangement, the examinee arrives at an equipped testing center with trained personnel who verify the identity of the examinee, administer the test, and ensure that the examination is conducted according to the declared rules.

Even the invigilator does not have to be physically present at the testing site. He or she can supervise test takers remotely (remote proctoring). This route makes it possible to use the existing type of tests and complements these with consistent online supervision. The described methodology of online supervision of distance testing developed into a separate discipline about ten years ago – “online proctoring”. Proctoring strives to use technical means to eliminate the risks of unwanted behavior of test participants. It covers the whole process from verifying the identities of the participants, checking the space in which the test is administered, positioning the camera(s), checking the programs running on the computer, to sending and evaluating the results.

Online proctored testing has two main modalities. On the one hand, it concerns testing on a large scale, where economies of scale appear to be an advantage. The second modality is distance exams of great importance, which have higher security standards and are used, for example, for entrance and graduation exams, for certifications, etc. Both types are also offered commercially as a service.

Large-scale proctored testing is usually organized for hundreds to thousands of participants. To deliver the test, it most often uses slightly modified tools for common electronic testing (Moodle, BlackBoard, ...), possibly extended by modules limiting, for example, the opening of other applications. Artificial intelligence is often used for surveillance in such large-capacity systems, which detects non-standard behavior of participants. When choosing software for online testing, you need to take into account how it behaves when the connection is lost. If, for example, the connection goes down during testing in Moodle, all data filled in by the testee is lost. If the connection goes down while testing in Rogō, only the last (currently being answered) item is lost.

Prevention and detection of cheating during distance testing

Proctored test cheating prevention follows the same rules as test cheating prevention, which we discuss in a separate chapter on test security.

A specific feature of proctored testing is the absence of a teacher on site, which may tempt some students to find unethical methods of influencing the result. Therefore, in remote testing, great attention is paid to technical solutions that replace personal supervision and limit the possibilities of cheating. The implementation of surveillance can take several forms, differing in the level of security, the number of people tested and the price.

Live proctoring in real time

Real-time streaming video from a web camera and mobile phone is used for control. This kind of remote surveillance creates a difficult confluence of image transfer requirements. Unlike video conferences, where a high-resolution transmission from the presenter is sufficient, surveillance systems require a high-resolution image transmission from all tested persons. This would lead to the rapid exhaustion of the transmission bandwidth when sharing the full video and thus limit the maximum number of people in one test run. Therefore, powerful video compressions are used, the image is switched between tested ones, or only individual photos taken at random times are transferred.

Proctoring Using Recording and Subsequent Review

The course of testing is recorded and the recording is then evaluated or kept for later evaluation. The advantage is the ability to test larger groups with fewer supervisors. Webcam proctoring has been shown to have beneficial effects in reducing academic dishonesty in online tests. However, the bandwidth issues are the same as with live proctoring.

Proctoring Using Artificial Intelligence

Supervision in this case is two-tiered. In the first level, the tested students are monitored by artificial intelligence, which evaluates the student's behavior in real time. In the event of an incident, it will alert live supervision, which will resolve the situation. The advantage is again the ability to serve a large number of test takers with a small number of supervisors. If the artificial intelligence application runs on the test taker's computer, bandwidth requirements are reduced, opening the possibility of serving really large numbers of test takers.

In addition to supervision, technical solutions to limit the possibilities of illegally obtaining information from the Internet are usually deployed during testing. This involves the use of specially developed internet browsers, such as Safe Exam Browser and others. Programs of this type usually work well on standardized classroom computers, but when used in students' home environments, problems may arise when working with different operating systems and incompatible combinations of other programs.

A special browser is not the only solution to the abovementioned problem. One alternative tested was the JavaScript program PageFocus, which very sensitively and selectively monitors the attempt to open another window and warns the examinee of his or her illegal actions.

Recommended Preventive Measures

• If you test all test participants at once, by testing in a “time window”, you reduce the amount of shared information.
• Measure the duration of the test in advance and set the time allowance very tightly.
• Allow test takers to take the test only once.
• Arrange the items in the test assignment of each participant randomly so that the order of the items is not the same.
• Do not allow test takers to change answers or go back to previously answered items.
• During the test, monitor or limit the activities of test takers using a specific browser.

Proctored Examination Controversy

The much-debated pedagogical problem of proctored testing is that it a priori views the test taker with distrust and with its technical measures it anticipates unfair competition – “head to head…”. There are several disadvantages of proctored online testing and objections to it:

• There is an undesirable intrusion into the student's privacy, for example by keeping video footage from the student’s home. Problems arise with the storage of personal and biometric data.
• An environment of mutual mistrust and suspicion is created.
• Test anxiety increases.
• A number of technical tools are put between the examinee and the examiner, each of which can fail (loss of connection, system freeze...). Some testing steps become more complex (e.g. identity verification, workplace verification). Therefore, extensive instructions and scenarios are created for both proctors and examinees. Training everyone involved can undesirably divert attention from the tested field itself, which is what students should focus on prior to the test.
• Online testing is influenced by the extent to which the examinee is familiar with the technical means used. It depends on the speed at which the student is able to send the answers, but also the confidence with which they answer and their test anxiety. The transition to remote testing can therefore introduce a significant bias into the assessment.

Currently, the debate whether the benefits of online proctored testing and proctored testing using artificial intelligence can even balance their disadvantages and risks is far from settled. However, sometimes it is unavoidable and proctored testing should be used, even with awareness of these issues.

Alternative Approaches

A large part of the problems associated with proctored testing in higher education is related to the knowledge-based concept of assessment. Distance testing is threatened to a greater extent than face-to-face testing by a handful of threats: identity confusion of the examinee (i.e. someone other than the designated examinee answers the questions), illegal cooperation (i.e. someone helps or provides hints to the examinee) and, in particular, the use of unauthorized sources and tools.

Although current technical means make it possible to verify the identity of the examinee relatively reliably under ideal conditions (e.g. also according to biometric features), when time stress, a large number of test takers and poor connection quality come into play, identity can be easily forged. Especially if the cheater is aware of the technical means used and the limitations faced by the examiner. In such a case, it is appropriate to consider uploading and saving the identification procedure, which gives an additional possibility to clear up any potential doubts.

A major threat to the validity of the results is unauthorized collaboration and the use of unauthorized resources and aids during the exam. If we exhaust the technical and organizational means to exclude them, these risks can be further reduced by appropriate design of the test and test items. Hints (mutual cooperation) are easy for closed-ended itemks. The effect of illegal aids is significant especially when answering knowledge questions.

Therefore, the appropriate approach to take seems to be to construct remote exams in such a way that they above all verify the achievement of higher educational objectives, not just memorization and recall of factual knowledge. When taking the test, specific factual information can most easily be found using a search engine, in notes or in the literature, and quickly be applied. On the other hand, when solving tasks aimed at a deeper understanding or even certain skills, neither searching on the Internet nor copying from books or textbooks will help a less prepared candidate – isolated facts alone are not enough to answer the question. Hinting and cooperation with another person is also more difficult.

A frequent recommendation is that remote exams should be designed as “open book exams” to the greatest extent possible. Currently, there are still not enough studies to support this recommendation unequivocally, but it is well-founded theoretically.

The relationship between the test taker and those administering the test must also be considered. An atmosphere of trust and fairness largely suppresses attempts at cheating. Less strict supervision therefore appears to be a suitable solution especially if the teacher/examiner has been working with the students over the long-term, solid relationships have been established between teacher and students and among the students themselves, and if higher levels of educational objectives are being tested. In contrast, a typical situation where a “hard” approach and strict supervision cannot be dispensed with are highly competitive exams where candidates and examiners do not know each other, such as entrance or certification exams.