Remote Online Proctoring: Will Exams Go The Way Of The Dodo In DL?

Critical Observations On Remote Online Proctoring

• Quality Assurance Agencies for Higher Education in the UK and EU at the time of writing this article do not appear to have taken an official clear-cut position in their documentation regarding the legitimate use of third party proctoring services. However, when approached, they acknowledge and allow their use with broadly conceptualised recommendations (see section 7 for more details).
• A growing number of universities in the UK at both undergraduate and Master’s degree levels use a variety of assessment methods in DL programmes without any examinations; moreover, using a variety of 'student-centred' methods in assessment is in line with UK/EU QAA recommendations.
• Scaling third party proctoring services, depending on the company, can be very expensive and potentially problematic (see section 5 for more details).

1. Executive Overview Of Online Proctoring

• Online proctoring, or remote surveillance of exams, has been around for approximately 10 years now.
• Currently millions of students in thousands of higher education institutions have likely used it worldwide for exams or professional testing.
• The market is predicted to reach 10 billion dollars by 2026; however, this prediction might prove overly optimistic, if a more rounded formative ‘exam-less’ student-centred approach to assessment becomes the preferred approach in online higher education.
• Robust, scalable, and credible technologies broaden student study options and make education more open, more accessible, and more flexible for students.
• It provides a competitive advantage in the increasingly internationalised world of higher education.

2. Who Uses Remote Proctoring/Professional Testing Services?

Examples of universities that have used such services include:

3. Some Of The Well-Known Platforms Used Today

4. Proctoring Services: An Overview

• Authentication processes leverage technology to help validate the identity card process e.g. through keystroke biometrics, facial recognition, answers to knowledge-based challenge questions.
• Prevention of cheating with use of live proctors monitoring, recording, and intervening real-time into cheating or suspicious behaviour.
• Test-taker data is collected and stored for review purposes.
• Helpdesk services check hardware and system prior to scheduled exam. There are also often live 24/7 technical support services.
• Incident Report Centre where instructors and administrators can view details of exam incidents.

5. Any Teething Issues? And What Are The Costs?

• Costs should usually vary from approximately 6 to 30 dollars per examinee per hour depending on the company and its service options [e.g. human, automated proctored exam etc]. Therefore, scaling could be expensive depending on service and number of exams.
• Some of the teething problems, depending on the company, may include: (1) the authentication process taking longer than it should; (2) some students not liking or not trusting an ‘unknown person’ proctor using remote access software to access their computers or at times using it to change some settings on their computers; (3) problems associated with inadequately trained/motivated proctors; (4) administrative concerns regarding scaling services to large numbers of students; (5) concerns regarding the management of, and control over, institutional data; (6) mistakes made by the institution or proctoring company in setting up exams, or by the proctoring company in interpreting dashboard exam instructions.

6. Broader Questions About Assessment In DL (i.e. Should Exams Even Be Used?)

• Analysis of trends in the UK [1] indicates that at both undergraduate and Master’s degree levels, a variety of assessment methods in DL programmes are used without any examinations [e.g. written essays, reports, career development portfolios, critical writing assignments, and a dissertation research project].
• The UK trend mentioned above appears to resonate with the position of the European Quality Assurance Register for Higher Education (EQAR) and the European Association for Quality Assurance in Higher Education (ENQA), which is reflected in the concise 28-content-page Standards and Guidelines for Quality Assurance in the European Higher Education Area [ESG].

In the ESG in section 1.3 on student-centred learning, teaching, and assessment, the guiding standard indicates that ‘Institutions should ensure that the programmes are delivered in a way that encourages students to take an active role in creating the learning process, and that the assessment of students reflects this approach’.

Moreover, the UK Quality Assurance Agency for Higher Education recommends using a variety of assessment methods that actively engage students in the learning and assessment processes; this therefore also resonates with ESG.

For instance, Indicator 5 of the UK QAA Code B6 [Assessment of Students and the Recognition of Prior Learning] mentions ‘the direct involvement of students in assessment, for example through self-assessment, and by negotiating the form of assessment that might be used’. Furthermore, essays, reflective journals, peer assessment, portfolios, assessment of performance, and creative work are all recommended.

7. Can UK/EU Institutions Of Higher Education Decide By Themselves On Whether To Use Third-Party Proctoring Services?

‘YES, [UK]’

‘YES, provided in line with ESG [EU - ENQA/EQAR]

The UK Quality Assurance Agency for Higher Education position [see excerpt below of email response on 11 January 2018] regarding the use of third-party proctoring companies [and not physical invigilation] in higher education is:

‘UK degree awarding bodies have considerable scope to operate assessment processes provided that they meet the Expectation in Chapter B6.  The use of third parties is not expressly allowed nor prohibited in the Quality Code. As you point out, identity is one of the areas of risk providers need to be mindful of when conducting assessment, but ultimately it is up to the provider to decide on the most appropriate and secure examination procedures, and the extent to which those procedures are operated by third parties’.

Designed by Vladislav Kolev

However, with regard to actually undertaking exams, indicator 11 of the UK QAA Code B6 [Assessment of Students and the Recognition of Prior Learning] emphasizes the importance of checking the identity of students taking tests in an examination room or online.

EU QAA Position Regarding The Use Of Third-Party Proctoring Companies

The position of the European Quality Assurance Register for Higher Education (EQAR) regarding the use of third-party proctoring companies is only that they be compatible with the succinct requirements of the Standards and Guidelines for Quality Assurance in the European Higher Education Area [ESG].  See key excerpts below of email response 23 January 2018 from EQAR:

EQAR does not have a specific policy or position on the matter. [NB third party proctoring]

As long as the institution ensures and it can demonstrate that external proctoring companies operating student assessment/examination do so in line with the ESG requirements, it is compatible with the ESG.

The position of the European Association for Quality Assurance in Higher Education (ENQA) regarding the use of third-party proctoring companies is similar to those of EQAR: See key excerpts below in email response on 16 January 2018.

For the time being, ENQA has not taken a formal position as regards examination procedures as such.

It [i.e. everything concerning the examination or identification of students at an institution] would need to be in line with the requirements of the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)[2]: http://www.enqa.eu/index.php/home/esg/

8. Key Takeaway Questions

1. Will most institutions of higher education offering fully online DL programmes at undergraduate and post-graduate level move away from incorporating any summative exams in their programmes [whether proctored remotely or not]?
2.  Will such a decision be justified by preferring to aim at ensuring that the delivery of their fully online DL programmes should encourage students to take an active role in creating the learning process and that the assessment of students will be reflected within this approach? In addition, will such institutions choose the ‘pragmatic route’ of avoiding: (1) potentially high proctoring costs, (2) possible bottleneck vender-lock-in dependency, (3) problematic implementation issues [depending on the company], and, (4) possible lower student intakes brought about by students preferring to study elsewhere in programmes that implement an exam-less student-centred approach to assessment?
3.  And if so, how will existing and future learning technologies be utilized in DL to validate a more rounded formative exam-less student-centric assessment of a student’s abilities to master course concepts?

References/Endnotes:

[1] This analysis was undertaken in 2016: (1) 23 fully-online UK university BA/BSc DL programmes that had no exams in their courses were analysed in terms of modes of assessment; (2) 68 fully-online UK university MA/MSc DL programmes that had no exams in their courses were analysed in terms of modes of assessment.

[2] NB [Email response from EQNA 16 January 2018 also stated] --The ESG is not only common for ENQA and EQAR but to all higher education institutions and agencies in the European Higher Education Area (EHEA). Such institutions should all build their QA systems in alignment with the ESG. The Ministers have made a formal commitment to this by adopting the ESG (revised version) in 2015 and by establishing EQAR in 2007 to oversee the compliance of QA agencies with the ESG’.