Sustainable Upskilling Solutions: Proctoring Done Right

Sustainable Upskilling Solutions: Proctoring Done Right
Summary: Proctoring is the method of validating a learner’s identity with technology, which may include monitoring the learner through a particular phase of the course, typically during assessments.

What Is Proctoring?

Proctoring is in essence a validation and monitoring service whereby the monitoring of learners is either effected live or via the assistance of technology. Live proctoring (i.e., the monitoring of learners' writing assessments on computers or tablets by invigilators) has been around for many years and predates proctoring that is underpinned by technology. To illustrate—and for example—live proctoring that is still commonplace today can be found within the aviation sector. Student pilots typically “write” assessments on a computer in an assessment center, whereby each computer is monitored live and in situ by an invigilator to deter cheating. Conversely, and with the maturation and enhanced capabilities of technology, technological proctoring has become a real and mainstream alternative to live invigilation proctoring.

Technological proctoring has become prevalent within schools, colleges, and universities given that students have been thrust into online teaching and learning, chiefly due to the COVID-19 pandemic. Similarly, organizations are relying more and more on proctoring to ensure that the integrity of (inter alia) critical and risk-bearing assessments are reliably maintained within a remote setting. For the purposes of this article, “technological proctoring” will be simply referred to as “proctoring.”

Proctoring is supported by Artificial Intelligence (AI), however, proctoring does not automatically equate to AI. To quote Dr. Ashley Norris:

AI is an information gathering and assessment engine. In that way, it’s just like an exam itself, collecting information and scoring it along a set scale. What makes AI different is that it can "learn" from what it gets wrong and what it gets right. AI systems get more and more accurate with use. That gets us to what AI does not do during remote testing. It does not—and I cannot underscore this enough—decide who is cheating. It does not "flag" some behaviors as cheating and fail a student. It does not have a set score of cheating versus not cheating—look off your screen twice in a minute and you’re fine, do it three times and you fail. It simply does not do that.

Proctoring Benefits And Drawbacks

The advantages and value of proctoring can be crystallized into 4 quintessential benefits:

  1. Assessment centers can in essence be done away with, which in turn also extends to physical invigilators.
  2. Time, as well as traveling expenses, are in turn vastly reduced.
  3. High levels of assessment integrity are reproducibly maintained.
  4. Reliable and transparent records of assessment takers may be retained in perpetuity.

Per the above, the benefits appear sound and the case for proctoring seems obvious. However, there is still significant pushback against proctoring, often not without merit:

  • Proctoring is not regulated, meaning each proctoring service is free to record and apply what they deem fit and suitable, from face recording to keystroke logging, from ambient noise capturing to screenshot recording—the list is harrowingly endless and often overshoots the line of what is ethical.
  • User data, such as face recordings and screenshots, for example, are usually shipped from the Learning Management System to a third-party proctoring platform that deploys the technology to validate and verify a user. A key concern and risk is the management of, and control over, this institutional data by a third party. This also extends to the reliability and speed at which the proctoring is completed—the latter issues (reliability and speed) are often found wanting.
  • Similarly, many users are simply not comfortable having an external, unknown, and possibly inadequately trained or equipped company being the jurors of their assessment.
  • Lastly, scalability. Administrative concerns are not uncommon, especially from large corporate customers such as those customers who operate in the financial services sectors.

These concerns are far from baseless and can quickly materialize into a proctoring showstopper. Luckily Sustainable Upskilling Solutions or SUSs comprise an alternative to traditional proctoring.

Proctoring Done Right

SUSs have a somewhat different approach to Learning and Development, including assessing, compared to a traditional Learning Management System. A traditional Learning Management System predominately focuses on consolidating and disseminating content electronically to learners together with top-line assessing (and in turn reporting)—and that is really it. On the other hand, SUSs provide organizations with an end-to-end mechanism to easily and sustainably train and assess their entire workforce. More about the differences between these two types of systems can be read in this article.

A key proctoring differentiator is that proctoring is built into, or integrated into, an SUS, rather than being merely bolted on and performed by any third party. This means that the organization is in complete control of who has access to proctored data and what ultimately shall become of it. Users are also assured that this data does not depart from the organization’s SUS, which—aside from the mitigated security risks—also means lightning-fast proctoring action. This is furthermore substantiated by the powerful and resilient architecture on which SUSs are built, meaning that organizations are assured of a scalable proctoring experience.

One key benefit of SUS proctoring is the sheer simplicity of how proctoring is accomplished, which includes the philosophy on which it is based. Face recording, keystroke logging, ambient noise capturing, screenshot recording, disabling of tabs and other software is technically not a challenging feat to accomplish, but one that is contentious and questionable. Worth noting is that the “disabling of tabs and other software” is more of a salesman’s promise rather than a reliable reality, given the different operating systems, browsers including deployed versions, and management software that corporate users make use of. Should an organization truly record the user's screen or log each keystroke? What were to happen when a user must urgently attend to a bank wire transfer in order to top up data during the assessment process? Should the organization be privy to seeing the user's bank balance? Better still, should the organization be able to log the user's bank profile details including username and password?

With key logging, this could easily become a reality. While just an example, where does this conduct start, and where does it end? What should happen if an organization deploys two types of proctored assessments: one that permits references, and one that does not. This would mean that the third-party proctoring software must be able to reliably cater for both scenarios, allowing for tabs or other programs to be open during one assessment and firmly preventing them from being opened in another. The concerns cited above are just a few of many that pose endless questions to a rather simple challenge, preventing users from cheating during an assessment.

The first step in solving this problem is essentially the easiest, the assessment itself. If a user is adequately prepared to write an assessment, then they should be able to do so within a given time frame (after all, there is a reason why most material assessments are time-limited). If a user is given ample time to research the question in order to plagiarise or research the answer then one has to question the adequacy and quality of the assessment envelope itself.

The second step in solving the problem, albeit more in parallel, is how the assessment is delivered. This article explains SUSs assessing capabilities in more detail. In short, an SUS through their question banks and question selectors ensures that no assessment is alike, thereby mitigating cheating further.

Lastly, proctoring. Ethical proctoring has a firm limit, and SUSs have drawn a line in the sand and sided with face recognition only, given that it is adequate in proving that a user, who is authenticated against an identity document, sat in front of their computer for the duration of the assessment. Going beyond this level of proctoring is unnecessary, unethical, and potentially pointless.

If a user really wants to cheat, they will, if so inclined. If a user cannot open another tab to search for an answer on their computer, then they will use their smartphone to achieve this. If a user cannot talk to someone else to obtain an answer, then they will use the chat functionality on their tablet. The list goes on.

Proctoring should ensure that a high degree of fidelity and reproducibility is warranted and that learners are made aware of that. Going beyond what is socially acceptable may only serve to push proctoring into the waste bin of failed technology whereby assessment centers will, again, become the preferred order of the day. One can certainly use a cannon to kill a fly, however, one often achieves more by doing less, which is generally an opinion echoed by the users who are subjected to the various proctoring options.