Laurence Fierens (20, political studies and Middle Eastern studies) got a ‘pretty weird feeling’ when he was taking a practice exam last week, using ProctorExam, purchased by Leiden University in order to administer tests online. The surveillance software has access to the computer screens of the participants, to keep an eye on them.
‘At first I had to use my webcam to take a picture of my face next to my student card to identify myself, like in real-life exams’, says Fierens about his participation in the pilot. Then, he took the exam in a secured environment. ‘Everything you do is constantly monitored; you get the feeling that you are being watched the entire time. I thought that was pretty distracting.’
Fierens wonders if this way of taking exams is going to work in practice. He also worries about his privacy. ‘I have no idea what will happen to those images. It also wasn’t clear if someone was watching live during the test or afterwards.’
Destroyed
Leiden University doesn’t want to comment on the results of the pilot yet.
Erasmus University College in Rotterdam already uses proctoring. ‘Only the examinations committee and ProctorExam are allowed to watch the recorded images, and afterwards they are destroyed’, says Gera Noordzij, head of Social and Behavioural Sciences.
Data from students who, according to ProctorExam, displayed unusual behaviour, are sent to the examinations committee. That could mean the student opened another website, or was staring at something else than the computer screen. The committee then decides if the student has cheated or not.
‘Our experience is positive. We expected unusual activity in about ten percent of the exams’, Noordzij says. ‘But in the end it was only four percent. And those irregularities happened mostly because of bad wifi. The internet connection has to be strong enough to follow the screen and to let the webcam film. It’s more about things like that, than about students doing anything weird.’
Xander Dangerman (19, technical physics) also took an exam with proctoring, at the TU Eindhoven. He used Proctorio, an American programme comparable to the Dutch ProctorExam.
‘I got an overview of what they are watching and what they have access to: your camera, microphone, location, identity, clipboard, mouse location, browser, Windows, your head, eye and mouth movements, your screen, all websites you visit and all other active applications and screens that are connected to your computer’, he sums up.
‘I thought that was invasive. I noticed I was being filmed the entire time, because the light next to my webcam lights up. It’s a very uncomfortable feeling, sitting in your own familiar environment with someone watching you.’
He scored a 2,5. ‘I wasn’t comfortable and couldn’t concentrate. I could see myself on the screen the entire time. I just wanted to get through it as fast as possible.’
Afterwards, he requested an alternative exam at the examinations committee because of ‘insurmountable difficulties’. It was granted. ‘Which means I don’t have to take my exams until we’re able to do them physically again. So it’s not a very useful alternative.'
It’s not clear if Leiden is going to use ProctorExam. It’s one of the options, Koen Caminada, vice-dean of the Faculty of Governance and Global Affairs and member of the remote teaching and testing project team, says. ‘We first want to have a good idea of the possibilities and limitations of the software. Things like privacy of the students, and the privacy law AVG, play a role. That should all be in order, of course.’
Problematic data
Leiden professor in Law and the Information Society, Gerrit-Jan Zwenne, says the AVG doesn’t prohibit proctoring. ‘But it needs good stipulations to protect the students’ privacy.’
For example, registering and storing ‘bijzondere gegevens’ - sensitive personal data- is problematic, Zwenne thinks. ‘Those relate to religion, political preference, health and ethnicity of a person. You see the skin colour of the student taking the test, a muslim might wear a head cover, or there might be a poster of a political party on the wall behind the student. The rule prohibits storing that kind of information.’
Except when the student gives permission. But students who don’t give permission, are excluded from the exam: ‘So it’s not real permission.’
Zwenne has one footnote. ‘If the data is exclusively used to analyse something that has nothing to do with ethnicity, health or religion, it’s defendable. So it’s important for ProctorExam to be able to prove that.’
Right to access
Another important issue is the right to access. By law, students are allowed to see their exam and how it’s graded, but proctoring complicates that. Students would also have the right to see the analysis the software made based on their data, like eye movements, type speed and keystrokes. ‘The company may not want to share how their algorithms work, because it could facilitate fraud, for example.’
Zwenne thinks it’s important students are properly informed in advance. ‘The regulations on education and exams (OER) should state that a programme like ProctorExam can be used.’
Leiden University comments that is has put ‘adjusted’ versions of the regulations online, concerning online examinations. Usually these adjustments need to go through the faculty and university councils, but ‘in times of crisis it’s a little different’, university spokesperson Caroline van Overbeeke says. ‘We include the councils wherever we can up front, but in most cases they will look at it afterwards.’
UPDATE:
In response to this article, the Socialist Party (SP) addressed proctoring in the House of Representatives on Thursday. Representative Frank Futselaar asked the minister of Education, Ingrid van Engelshoven, if the ‘privacy of students is guaranteed’.
Futselaar also wants to hear from the minister if it’s true that the surveillance software monitors, among others, webcam, screen, microphone and eye movements, as is mentioned in the article. He wants to know if that is legal, and what happens with the images and other information that is stored after the exam is finished.
Leiden University changed the regulations (OER) to include online examinations, without going through the university council and faculty council, which is the rule. A questionable course of events, Futselaar thinks: ‘Is an OER that hasn’t been approved by the councils legally tenable?’
He wants to know if the councils agreed to the use of proctoring. ‘If not, are you going to address this issue with Leiden University?’
If proctoring is not possible, there are enough other options that ‘protect the quality of the curriculum as much as possible, and prevents fraud’, Caminada says. Open book exams and essay questions, for example. ‘With essays, it’s easier to check afterwards. Ans is a good programme, that the Law faculty already uses. It can quickly detect suspicious behaviour through tekst analysis.’ Plagiarism scanner Turnitin also works well to detect fraud, he says.
There are more ways. ‘But I’m not going to tell you how we’re going to use them, of course. Teachers are ahead of the students now in this cat-and-mouse game, and we want to keep that advantage.'
Technology is only part of the story. ‘We are thinking about having students sign a statement that they finished the exam in good faith. That will be an obstacle for some students who are thinking of cheating.’ It would also help against students conspiring together: ‘It’s harder to find a classmate who is also willing to cross that line.’
Hannah Borst recently resat her criminal law exam, without proctoring. ‘Usually it’s a written exam in the university sports center. But now it was an open book exam from home.’ The exam was an hour shorter. ‘Otherwise it would have been too easy. There were definitely questions that you couldn’t answer if you didn’t know the material.'
She doesn’t know what her other exams will look like. ‘We haven’t been informed yet, and it’s a little frustrating. I can imagine they will have us write papers, then you can prevent all fraud.’
Caminada has a lot of faith in the students and teachers, no matter how exams will be taken. ‘There has to be trust from both sides. It’s important for everyone that the quality of education and the diploma is maintained. No one wants corona-sixes.’