Book (2022)/Administration of Tests

Administering tests – in-person or remotely?

Thanks to the development of computer technology, it is possible to choose how the test will be administered. It can be a written (paper-based test, PBT) or an online (computer-based test, CBT) test. Each of these approaches has its advantages and limitations.

Paper Testing

Paper tests consisting of multiple-choice questions saw massive expansion as early as World War I in response to the personnel needs of the US military. It was then necessary to quickly and efficiently classify a large number of recruits, and this could not be achieved in the short time available by the usual individual-basis work of psychologists at that time.

Thanks to its efficiency, preprinted testing quickly spread to other fields that previously relied on individually administered tests – education, intelligence testing, and other areas.

In the English-language professional literature, two terms are differentiated: purely computer-based testing and computer-supported testing. In the second case, the collection of answers can also take place using paper questionnaires (this is paper-based testing), but the tests are then evaluated and further analyzed using computer technology.

Computer-based testing is certainly the direction in which the entire field is moving. Nevertheless, paper testing is important, not only when there is a lack of computer equipment, but also as an easy entry into the world of testing and the use of related methodologies. Appropriately chosen programs and technologies can make our work significantly easier.

In the simplest form of a paper test, freely printed questions with proposed answers are enough. The traditional evaluation of answer sheets using transparencies with a template of correct answers showed a large error rate due to the human factor, often comparable to the number of errors made by the respondent in the answers. With the advent of optical scanners and optical mark recognition (OMR) technology, reading the answer forms is no longer a problem. Corrections and changes to the answers made by the examinee can also be analyzed relatively easily. For automated evaluation, however, the forms must be designed so that they are easily machine-readable, i.e. they meet the requirements for optical mark recognition. Examples of machine-processable questionnaire sheets can be found on the Internet under the terms “bubble answer sheet”, “OMR answer sheet”, or “scantron test sheets”.

Tests can also be generated and printed directly from testing support programs such as the specialized Rogō test program. It supports the printing of machine-readable forms, including the creation of several versions of the test with differently ordered items. Printing test forms is also possible with LMS Moodle, which has the Quiz OMR extension for creating machine-readable forms.

While the printing of test forms is often included in testing programs, the reading and recognition of completed forms is not addressed in the test systems mentioned. It is necessary to use an external solution, such as the proven commercial software Remark Office.

Advantages

• Paper testing usually uses pre-printed forms on which the test taker marks their answers. The advantage is that a large number of tests can be administered simultaneously.
• Answering on paper is more intuitive and comfortable for some test takers, since it doesn’t raise concerns whether they can manage the technology.

Disadvantages

• One disadvantage is the inflexibility of the entire process due to the technologies used.
• For example, is not possible to obtain certain information, such as regarding the speed with which the testee responded.

Computer Testing

Electronic evaluation has largely evolved from conventional forms of evaluation. The original paper tests and answer sheets were converted into digital form and delivered to the test taker either by an application running on a local computer, or, with the development of technology, now more frequently online, via the Internet. A massive increase in electronic testing can be seen especially in the last ten years . Added to this now is testing using mobile platforms. A number of software tools are available for online testing. On the one hand, there are specialized programs that deal only with testing (e.g. Rogō) or test modules that are part of various comprehensive tools (e.g. LMS Moodle).

Advantages

Computer testing is incomparably more flexible than paper testing. It allows you to use multimedia in items, and there is no loss of quality in images. Computer testing also brings a number of advantages for the administering of the test and controlling its progress (e.g. it is possible to set one-way passage through the test, finished itemks can be locked, etc.). A huge advantage for test security is the ability to track how the test taker answered over time and how long each item took. In addition, computer testing opens up entirely new possibilities for adaptive testing. Direct filling and processing of the test on a computer significantly speeds up its evaluation, which is greatly appreciated by students expecting feedback. Electronic testing is generally less error-prone and leads to higher quality assessments. Thanks to computerized testing, new formats of test questions are entering the assessment, using, for example, the possibility to mark the answer in a picture. And finally, computer-based testing is more cost-effective and environmentally friendly than its paper counterpart.

Disadvantages

Compared with paper testing, computer testing is limited by the computing technology available to the operator. A computer system can be infected by a virus or attacked by hackers, or fail due to a loss of power or connectivity. Prior to electronic testing, test takers and staff must be trained in the use of the electronic testing system. Potential dispute resolution with disgruntled test takers can be more complicated because there is no “paper proof” of what the item was and how the student answered. When testing many clients at once, there may be increased demands on transmission capacity, especially in the case of mobile devices with wireless connections. High initial costs (HW + SW) may discourage the deployment of electronic forms of assessment, but this disadvantage is offset by low subsequent operating costs.