Authoring Tools for Electronic Examinations

I'm going to put a little more detail into each of the steps in electronic examinations workflow I wrote about last week. So this week I'm discussing exam authoring.

The authoring tool should allow examiners, at a minimum, to create questions, edit existing questions, store their questions in a question bank. Also create an exam, add questions to an exam, and then schedule the exam.

Creating and Editing Questions

To create or edit a question you’ll provide a question title, the question stimulus e.g. Which of the following statements about aspirin antiplatelet effect is true? The correct answer or answers and how they are scored really depends on the question type. One of the more common question types in medical and veterinary education is the multiple choice question (MCQ), so I’ll use that for all my examples in this post. So for MCQs you’ll also provide the distractors, then flag and score the correct answer. The exact scoring is up to the examiner.

Question Banks

Questions can be saved to a question bank or repository, question banks can be shared or private. Shared obviously allows for team collaboration on examinations. They can also be stored on the local hard drive or in a shared online database. Some tools give the option of creating a repository on both, some don’t. It’s really depends on the authoring client I suppose. Web based tools won’t have access to the local hardware, for example.

Exam Creation

To create an exam the authoring tool should allow the following. The exam needs a title, I’d suggest using the subject and the type of exam and the semester: Gross Anatomy Spring Midterm 2011 for example. Obviously you’ll need to add questions, and they can either come from your question bank or by creating the questions as a sub-task of exam creation. You’ll need to set a passing grade for the exam, as well as scheduling a time and date for the exam to become available.

Extra Features

Some nice additionals to electronic exams can include setting a time limit, once the student has reached the time limit their exam is automatically submitted for them. Randomizing question order, so the same exam is presented to students in a different order, this can reduce academic dishonesty by making it difficult to copy somebody else’s answers. Some tools allow for question distractors to be presented in a random order as well, giving the same advantages. Messages can be set to be displayed at the start and end of the exam, such as instructions for the exam at the start and potential feedback at the end. Exam scheduling may also be set when authoring or may be managed by an administration tool once the exam has been created.

Electronic Examinations - A Simple Primer

Electronic examination, in a nutshell, refers to exams taken using software. That software at a minimum should allow instructors to create and edit exams, present those exams, record students’ answers, and then grade the exam. In this blog I’ll present the general workflow of creating, administrating, and grading exams. This flow will need to be tweaked for your specific needs.

The (very) basic workflow has the following steps:

The faculty member(s) create the exam – this can be done as an individual but some systems provide collaboration. The questions are then added from a question bank or created from scratch. Scores are set for each question and a passing grade is set for the exam. The exam can then be scheduled to run at from certain time and date. The scheduling can be done either by faculty members or by support staff.

The exam is then delivered to the students, either through a web browser or other client. Some solutions provide a security application that will lock down a pc or mac, depending on the vendor, preventing network access, hardware access such as the USB port or the hard drive, and even disable right-clicking on the mouse or screen-grabbing. This behavior can be quite invasive however, in my experience this may activate anti-virus software which recognizes it as an attack. In the case of windows at least, windows groups may be a better option for securing the computer.

For the traditional exam environment students come to a computer lab to sit their exam, they can also do open book exams over the web.

Grading is automatic, most solutions give the option of providing a grade immediately or at a later date. Immediately means that students’ final scores, and whether they are passing grades, are presented on screen once the exam is completed.

It should be noted that when running an electronic exam in the tradition environment it is the recommended practice that you always have a backup plan. ICT equipment can suffer failures and being able to roll-back to “Plan B” is generally better than cancelling an exam outright (more on this in a later post).

First Post

One of my responsibilities is examinations and importantly for this blog, electronic testing of veterinary students.

Electronic testing has advantages and pitfalls, which I'll discuss here, with the occasional tangent into other topics of interest to myself. In my current roll we have two types of *computer based* testing. First is what I'm guessing you'd imagine electronic testing to be: the exams are authored, taken, and scored using an electronic testing tool, I won't name names just yet.

The second are what we call computer aided / media associated exams which, in our case, are answered on a bubble sheet but do have an *E* element to them. That can be either images such as radiographs displayed on a computer, video, audio or simply displaying the exam in pdf format on screen.

I'll be exploring both in the coming weeks and months.