Tech Alone Can't Stop Cheats
If you had to give someone an important test—say, to be sure they really know how to fix your roof or care for your child—you’d probably put serious thought into the questions you’d ask and you'd want to be sure the person you’re testing isn't cheating, bluffing or just lying their way through the test. After all, what’s the point in testing someone if you can’t trust that their answers are really their own?
In fact, the more serious the task or skills—life and death skills like engineering or medicine or flying a plane for example—the more emphasis you’ll put on not only finding the right questions but also on ensuring how you test so that the outcomes are real.
Why Proctoring Is Important
As a general rule, the more significant the test, the more time and money test-writers, and test-givers spend in getting it right. And yet, it surprises people that organizations invest so much in writing test questions for high-stakes exams like nursing or graduate school entrance exams. Just one single question on a high-profile licensure exam, like the bar exam for would-be lawyers or the test of the medical licensing board, can take years to develop and cost tens of thousands of dollars. For just one question.
Not only are these high-stakes professional exams usually expensive to create, but they are also always proctored, meaning the test process itself is observed to ensure a fair and standardized experience for every examinee. Proctoring occurs either in a physical testing center with someone watching or, when given online, with technology-aided security features and a webcam.
Furthermore, if the exam is taken to verify that the examinees actually earned the license to become a nurse or a lawyer or to be admitted to medical school, and if you have invested $2,000,000 in developing and proving the validity of this test, of course, you'd make sure that the test session is proctored.
How Do People Cheat?
Given the consequences of not passing an exam after spending, say, three years and $200,000 on law school, it’s not difficult to expect that some people will try to cheat the system or steal the test content.
Not too long ago, these cheating attempts were dramatically unsophisticated. It was pretty common, for example, for someone to sign up for a high-stakes test, go to the test center, get signed in, and then grab the test booklet and sprint for the door. Or they’d try to quietly rip a page or two from the test book to slip into their pants pockets and sell afterward to the highest bidder. They didn't care about the consequences. Their sole intent was to grab content that somebody else would spend big bucks to have.
I can also tell you stories of test-takers who made temporary tattoos out of exam notes, spinning words into funky patterns or translating to obscure languages that look like poems or memorials. We’ve seen twins switch places during bathroom breaks, having stashed books and notes in restroom garbage cans.
Even with trained proctors watching a test session in real time, people try to cheat and it’s getting more sophisticated over time. About a year ago, someone took a high-stakes certification exam online while a drone with a camera hovered overhead snapping pictures of the exam questions. Our technology picked up the buzzing sound and when we reviewed the recorded test session, our fraud team saw the drone reflected in the window. That’s a perfect example of humans and technology working together to solve a problem.
There’s more. We’ve had people run cables under rugs and floorboards to different parts of their homes or offices so they could record their online tests on different computers. We’ve had test-takers hide a literal team of people in the next room, cabled to the examinee’s computer. Camera-rings or those “elf on shelf” video cameras? We see those all the time and nobody gets away with it. Not all the attempts are very sophisticated though. This year, we caught a guy taking the same test for a half-dozen different people, including a few women. His blond wig and scarf, it turned out, were not convincing enough to fool expert proctors.
Catching Cheaters: Technology And The Human Factor
As expected, the more the stakes and investments in these tests increase, so do the efforts to cheat them. It’s not uncommon for us to catch a test-taker in California signing in to take their test through a server in another country using a fake web address. Now, not only do thieves try to sell stolen test questions, but they also peddle software that can supposedly mask a laptop’s hard drive or web browser, theoretically allowing a test-taker to look up answers or review prohibited notes without tripping our security. We know because we buy these purported cheating solutions and with a little reverse-engineering, catch the people who try to use them as well as the creative “geniuses” who created the software.
Increasingly, we use metadata to catch cheats. Metadata is like the fingerprints of technology: not what you do but how and where and when you do it. Our systems, for example, can easily spot a test-taker who’s answering questions too quickly or too slowly. It also catches someone typing way too fast, like 700 characters of text being dumped into an answer all at once.
What’s interesting is that these bits of information by themselves don’t necessarily mean a person is cheating; it just means a test session has an outlier metadata point warranting a more thorough review by the proctor. In every case, actual accusations of cheating are made by people who review the sessions and then send their impressions to the testing provider. Computers alone don’t make these determinations, but they are significant parts of the overall equation.
This is important because even though technology is getting more complex and seemingly better at cheating, it’s ultimately a human interaction. People make a conscious decision to try to cheat and then it’s up to people to catch them. Tech is assistive. Software may alert us to something we didn’t catch at first, but trained test experts know how to interpret what the software is reporting.
We give tests to be confident, to be as certain as possible that the examinee can actually do the tasks they're being asked to do. Companies and testing providers invest a ton in getting that right—not just in the tests themselves but to make sure the outcomes are secured. Every time you fly in an airplane or take medicine, you’re trusting that the pilot and pharmacist took a test that was good and fair and very, very secure. That’s how you’d do it if you were picking a nanny for your youngest child. That’s how it should be, and why so many of us invest so much to be sure it is.
