Credentialing programs and test sponsors face increasingly sophisticated cheating and collusion tactics, compromising test security and program integrity. Compromised and exposed test content, proxy test-takers, and disregard for test security rules all negatively impact program reputation, brand equity, and test-taker futures. Most importantly, these actions erode confidence that qualified, credentialed individuals serve the public sector. Comprehensive test security plans can mitigate these outcomes and provide test sponsors with the right tools to protect their intellectual property (IP) and credentialing programs.
Professional testing organizations can protect their exam content effectively by implementing comprehensive test security measures. Fueled by 360-degree test security plans, advanced technology, and professional expertise, these techniques help mitigate security threats before, during, and after a credentialing assessment has been administered.
In this article, we describe what “comprehensive test security” is and why it’s important for the professional testing industry. Then, we provide 5 best practices for implementing your own comprehensive test security measures.
What Is Comprehensive Test Security?
Comprehensive test security is an approach that considers all the factors that could compromise the security of a test. This holistic approach implements strategies or mechanisms to deter, detect, and prevent cheating tactics. The measures taken help build and fortify security strongholds throughout the entire assessment life cycle—from design to delivery and beyond.
| Comprehensive Test Security Measures | ||
| Test Development | Administration and Delivery | Pre- and Post-Administration |
| Randomized test items and/or options | Technology-assisted identity verification | Continuous online threat monitoring |
| Multiple test versions or forms | Real-time screen and webcam monitoring | Psychometric data forensics |
| Algorithms used to generate unique exams for each test-taker (e.g., linear-on-the-fly testing [LOFT] or computer-adaptive testing [CAT]) | Secure browser technology to protect assessment content during proctored or unproctored examinations | Qualitative data review to identify patterns of anomalous behaviors |
| Innovative item types used to assess higher cognitive functions | Recordings of each test-taker, their screen, and their chat logs during remotely proctored tests | Developing an in-house exam security role |
| Establishing exam misconduct policies & procedures | ||
Why Is Test Security So Important in Professional Testing?
No assessment program can completely shield its exam from security threats. But organizations that deliver medium- and high-stakes exams must remain hypervigilant. The reasons are numerous, including public health and safety concerns and potential loss of credential credibility. Therefore, it’s crucial that these organizations be aware of and prepared to address the following security threats.
Proxy Testing
Someone physically sitting in place of a test-taker or remotely controlling a test-taker’s computer and taking the test on their behalf
Unpermitted Resources
Test-taker using a resource that is not allowed in the exam rules, whether physical or virtual, to gain an advantage
Content Theft or Harvesting
Test-taker stealing test content to share or sell it to others
Collusion
Test-takers working with each other, a proctor, or an administrator to circumvent rules and integrity controls
The Risks of Compromised Test Content
Every assessment program is unique, but credentialing programs must weigh their security risks against the costs of mitigating them. One of the factors to consider is the stakes involved—for test-takers and the organization itself. When the integrity of a medium- or high-stakes exam is undermined, the ramifications can be significant. The effects of cheating on a professional assessment can be financial, legal, and reputational. Here are some examples:
Financial Costs: If a test-taker steals assessment content, the exam owner must replace the stolen IP. That can cost anywhere from $10,000 to $100,000+ per exam. Furthermore, the owner will have to dedicate time and staffing resources to investigating the incident and tracking down the lost content. Those actions will add extra monetary strains.
Legal Repercussions: If you end up certifying someone who is not competent for the job, the certifying body, employers, and test owners could face legal implications, especially if the person deals with the health and safety of society.
Reputational Impacts: If there’s a cheating scandal or your organization faces legal ramifications that end up becoming public knowledge, the press will eventually report on it. This could lead to even more financial (and potentially legal) ramifications in addition to bad word of mouth. If the credibility of your credential comes into question, your program may lose accreditation. That could subsequently put all credential holders at risk and reduce your brand equity.
Best Practices for Establishing Comprehensive Test Security Practices
Comprehensive test security measures are necessary to ensure that all potential security risks are addressed. Credentialing organizations can mitigate vulnerabilities to testing programs when they adopt a holistic approach to test security. It is important to know the threats to test security and have a plan in place that will lead to effective action.
In adherence to the International Test Commission’s guidelines for test security, we’ve outlined 5 best practices to consider in order to maximize test program security.
#1. Create a Test Security Plan
Test sponsors must have clear, concise messaging that test-takers and other stakeholders understand and are held accountable for. There are many factors to consider and steps to take while creating a test security plan, including security best practices for test development and administration processes. Additionally, you should create an incident response plan that dictates what to do when an integrity breach occurs. Publicizing your test security measures and response plans via a communications professional can effectively deter test-taker malfeasance as well.
#2. Build Security Measures into Test Development Processes
Content theft often results in test-takers gaining access to assessment questions and/or answers before administration. Test sponsors can gain greater confidence to protect IP by implementing exam design measures that support content. Examples include randomizing items, using multiple forms, and developing item types that are more difficult to memorize than traditional multiple-choice items.
#3. Evaluate Test Administration and Delivery Processes
If your organization develops medium- or high-stakes exams, weigh the pros and cons of multiple delivery modalities:
- Exams delivered via live remote proctoring
- Exams delivered at a physical test center
- Exams delivered either remotely or in person (often called “hybrid test delivery”)
Each delivery modality has its security benefits, but they’re all highly effective at deterring misconduct simply because they have a human present for the test duration. Proctoring—regardless of modality—can greatly improve test security, especially when it’s coupled with strong misconduct policies and enforcement.
#4. Conduct Data Forensics
Data forensics, sometimes called forensic data analysis, is a growing area in psychometrics that analyzes empirical post-exam data to assist in identifying content that may be compromised and detecting aberrant behavior that could suggest potential misconduct.
“Routine test, item evaluation, and test-taker response monitoring are essential tasks that all testing programs should employ and include as regularly scheduled program maintenance activities. Data forensics empowers test sponsors with both proactive and retroactive approaches to test security.”
Maria Incrocci, Senior Vice President and General Manager of Certification, Assessment Development and Psychometrics, Meazure Learning
According to research by the Institute for Credentialing Excellence, data forensic specialists can identify 4 types of misconduct: unusual score gains among repeat test-takers, response similarities among synchronous test-takers, unusual total test times, and unusual performance among test-taker groups.
#5. Create a Security Tip Line and Encourage Its Use
There are individuals who want to report cheating incidents they’ve witnessed. Oftentimes, they’re test-takers who’ve spent a lot of time preparing for and qualifying to take an assessment. They’re proud of their earned credential and don’t like to see its value degraded. Provide whistle-blowers with an anonymous way to report a suspicious incident, and have a plan in place to act on reports that come in.
Conclusion
It’s important for every assessment program—particularly medium- or high-stakes credentialing exams—to be aware of and prepared to address security threats. By establishing effective measures to protect their exam content throughout the assessment life cycle, credentialing organizations can maintain the value of their credential and overall brand equity.
To learn more about comprehensive test security practices, watch our on-demand webinar “New Advances in Exam Security: How to Safeguard Your Program.”