Book (2022)/Rogo

Among the tools for electronic testing, the Rogō program occupies a prominent place. The extraordinary position of this system lies in the fact that it is a high-quality, secure and easy-to-use test tool that is also freely distributable.

Advantages (quality)

• Each item has its own record of changes and use in tests.
• Invited external collaborators have easy and secure access to reviews.
• During remote testing, it is possible to set the end of the test after the time allowance has been exhausted.
• The items used in the test will be locked.
• Items cannot be added or removed from a test that is currently in progress.
• Possibility to set the time limit of the test.
• Student responses can be analyzed and item properties can be easily evaluated.
• The grading can be adjusted according to the performance of the cohort after the end of the test using the Hofstee method.
• Modified Angoff and Ebel methods are also available for setting test scores.

Security

• Only HTTPS protocol is used for connection. Data is encrypted using 256-bit SSL.
• Seamlessly and securely share materials across staff teams to work together on assessments.
• Customizable checks and weights to ensure fair grading for all users.
• They can use established local authentication systems for student and teacher access.
• In the event of an Internet connection failure, only the answer on the currently open test page is lost, not the entire test.
• Access to the test can be limited to selected IP addresses.

Applicability

• It can run on both Windows and Linux servers.
• LDAP compatibility – no need to create additional credentials for users.
• Support for language mutations.
• Tailored help systems - separate help for users by role.
• Rogō is a web application that runs on all major browsers - Chrome, Edge, Firefox, Safari, Internet Explorer.
• Adepts with special needs can adjust both the appearance of the test and the time allowance.

License

As far as license is concerned, the Rogō online testing web application is a freely distributable open source program, released under the GPL version 3.0. It is therefore possible to change the code, extend it and thus contribute to the project. In practice, it works in such a way that requests for code modifications, whether related to reporting problems or proposals for new functionality, are written into the request queue and are gradually addressed by the community.

History

The Rogō test system has been developed since 2003 at the University of Nottingham Medical School under the name “TouchStone”. The Rogō system originated at the medical school and is very well adapted especially for learning medicine. Among other things, it allows the use of interactive images in items, on which the student marks the desired object with the mouse. The system then evaluates whether the student marked the desired structure with the required accuracy.

After its success in its home faculty, it was expanded to the entire university, converted to open source software, and on that occasion renamed to avoid confusion with other systems. Rogō means “I ask” in Latin. The Rogō development community received financial support from JISC, which enabled further development of the system, including the ability to translate into national languages. In the Czech Republic, the Rogō system is installed (on the servers of the First Faculty of Medicine of the Charles University) at the address https://www.rogo.cz/ and, in addition to the First Faculty of Medicine of the Charles University, it also serves other faculties of the Charles University. The First Faculty of Medicine of the Charles University has prepared a Czech translation of the environment and is continuously working on the localization of the help section. Thanks to LDAP support, students are automatically imported into the system from SIS (Study Information System of the Charles University) and can authenticate with their CAS account (Central Authentication System of the Charles University).

Specific Properties

Unlike other programs, Rogō covers and supports many steps of the test development cycle, from collaborating on the preparation of test items, to challenging them in terms of difficulty and relevance, creating a test plan, standardization, and evaluating the quality of questions. This kind of comprehensive solution brings significant advantages. For example, it is advisable to invite a number of in-house and external experts to supervise test questions, which is usually time- and organizationally demanding. At the same time, if item proposals circulate among a large number of people, it is very difficult to ensure their secrecy. In Rogō, on the other hand, opponents are prompted to join the system, so the test items never leave the system. Again, comments and suggestions are entered directly into Rogō and item authors can respond to them immediately. After the test has taken place, it is possible to display the descriptive characteristics, a histogram of the total scores of all students, or the difficulty of the items. Rogō automatically calculates discrimination indices for each item of the test, which makes it possible to identify poorly constructed items and exclude them from further use. From the point of view of applying modern procedures in testing, the system is completely unique and its introduction supports the extension of good test practices into the field.

The system makes it possible to distribute both paper and online tests, both for self-assessment and for secure summative testing.

Rogō includes tools to automatically import students and courses they have enrolled into the system. Thanks to the support of the LDAP directory service, which is used by the entire Charles University, students can log in directly with their CAS account and are assigned to all their courses in Rogō.

The system allows educators to create a many types of tests and surveys:

• Formative assessment
• Summative assessment
• Progress tests
• Surveys (questionnaires)
• e-OSCE (clinical trial)
• Offline tests
• Peer assessment (student)

Each of these types of tests and surveys can use a variety of item forms:

• Area delimitation
• Dichotomous items
• Multiple choice questions
• Extended Matching
• Multiple true false
• Complete the text
• Mark points in the image
• Likert scale
• Scenario matching test
• Text fields

Advantages

Table. X.X Advantages of the Rogō system

Low cost of acquisition Support for the entire testing process Support for teamwork High level of security A large selection of types of test items, including multimedia ones

Disadvantages

Table X.X Disadvantages of the Rogō System

A smaller community maintaining and developing the system Need for local support and administration Not quite intuitive operation Time consuming when learning the program