Quick Overview
Situation
APSCA is an industry association charged with developing a standardized global certification program. A credible, cost-effective solution was needed to reach over 3,200 people in 47 different languages across 93 countries.
Solution
After careful evaluation of other proctoring methods, APSCA chose online proctoring to monitor their three different exam types and selected the ProctorU Proctoring Platform‘s Live+ service line.
Results
Meazure Learning provided a solution that allowed APSCA to quickly scale a new global certification program while also helping them to avoid $4.8 million in costs as well as identify and prevent integrity issues.
Background
The Association of Professional Social Compliance Auditors (APSCA) was established in 2015 to enhance the professionalism, consistency, and credibility of individuals and organizations performing social compliance audits. While APSCA is still a young association, the social compliance industry has been in existence for more than 20 years. APSCA is the sixth attempt to professionalize the industry and the first to find success. Today, APSCA consists of over 3,200 auditors and 40+ member firms across 93 countries, and it is still growing.
A major goal of APSCA was to create a standardized global certification program. This led to the development of the Certified Social Compliance Auditor (CSCA) program. To be designated a CSCA, an individual must pass a three-part examination and meet minimum experience requirements stipulated by APSCA.
“You wouldn’t go hire an accountant that wasn’t a CPA or a lawyer that hadn’t passed the Bar exam. You wouldn’t even think about doing that. We want people to get to that place with our industry as well.”
Rona Starr, President and CEO, APSCA
The purpose of the certification is to establish a level of credibility and validation for APSCA member firms and auditors. The association mirrored its program after legal, accounting, and other similar industry certifications with the intention of establishing a credential that would be sought after and, eventually, required when a company needed social compliance auditing services.
As explained by Rona Starr, President and CEO of APSCA, “You wouldn’t go hire an accountant that wasn’t a CPA or a lawyer that hadn’t passed the Bar. You wouldn’t even think about doing that. We want people to get to that place with our industry as well.”
With five failed attempts to professionalize the industry, it was critical that APSCA succeed and find a credible solution to meet all of their requirements.
- Quick to implement.
- Scalable to reach over 3,200 auditors across 93 countries.
- Cost effective
In-Person Exam Administration Pilots
In the first pilot phase, APSCA chose to administer and proctor their exams in a face-to-face setting. Pilots were administered in Turkey with 40 auditors and in China with 80 auditors. In going through those pilots, they saw that using physical testing centers would not meet their requirements.
“It wasn’t scalable, it wasn’t cost-effective, and we realized that we had a much higher risk of integrity issues than what we originally thought we would have,” said Starr. “We thought we couldn’t beat face-to-face in being able to address integrity issues, and we quickly recognized that wasn’t the case.”
The Search for a Better Solution
“We thought we couldn’t beat face-to-face in being able to address integrity issues. And we quickly recognized that wasn’t the case.”
Starr admits that she was very anti-online proctoring at the beginning of the process. However, as a direct result of what she saw during face-to-face pilots, she began looking for other options, and online proctoring soon rose to the top of her list of alternatives. After investigating the players in online proctoring, APSCA decided to put Meazure Learning (formerly ProctorU) to the test … literally. Starr signed up a few of her team members through our system and instructed them that their sole purpose was to cheat in the most subtle ways possible and see what happened. The outcome was positive.
Exceeding Expectations and Overcoming Barriers
In addition to meeting the initial requirements of quick, scalable, and cost-effective, Starr identified several other areas where the Live+ proctoring service exceeded her expectations.
Identifying and Preventing Integrity Issues
During the face-to-face proctored exam pilots, APSCA recognized how difficult it was to effectively monitor a room full of test-takers all at once.
“It is physically impossible for one or two proctors (also called invigilators) to watch each individual at all times. The proctor is right onto them and saying, ‘What are you looking at? What’s over there?’ We could never pick that up in a live, in-room situation. Some of the other benefits that we’ve seen are that as soon as the proctor says your time is up, the proctor can stop that exam and stop the test-taker from continuing on.”
The ability to identify and proactively eliminate integrity issues translates to value and added credibility for the CSCA credential.
“The more credibility that we can build and demonstrate to the people that are asking for APSCA certified auditors, the better for us as an industry association,” explains Starr.
“I was actually blown away at the process,” said Starr. “I was amazed at the very simple things they caught. Eye movement from the auditor, body language that the proctor was able to identify and then stop the exam to investigate further. It really impressed me and we made an almost overnight change in the organization to move away from face-to-face and only move forward with online proctoring.”
Cost Avoidance
A benefit Starr is most excited about with the transition to online test administration and proctoring is the cost avoidance recognized by APSCA and their member firms. She estimates that the association is saving at least $500 per auditor, per exam. With three exams to administer across 3,200+ auditors, that comes out to a minimum savings of $4.8 million for APSCA. That number doesn’t take into account the money saved by the member firms not having to pay travel expenses (flights, cars, hotels, food, etc.) for each auditor to go to a physical testing center. Furthermore, the estimate doesn’t include the time each auditor would have to spend away from work when traveling to a physical test center. Considering these variables, the quantifiable cost is much higher than APSCA’s estimated savings of $500/auditor/exam.
Validating Integrity Issues
One of the benefits Starr didn’t expect was how easy the ProctorU system makes it to validate and close out an integrity issue.
“Everything is recorded,” she explains. “I can see what the auditor was doing, I can see where they were in the exam, I can see the time, I can see what the chat log is between the individual proctor and the individual auditor. It really makes for a simple closing out of an integrity issue when you’ve got those snapshots and you’ve got that video recording. It’s pretty black-and-white.”
When an integrity issue comes into question at a physical testing facility, it can be difficult to definitively prove the series of events due to hearsay. Starr recalls one integrity issue from the face-to-face pilots that took a very long time to close out because it turned into a he-said, she-said situation.
“The proctor says, ‘This is what I saw.’ The test-taker says, ‘No, this is what actually happened.’ It’s very difficult to close that out.”
Full recordings of a test-taker’s face and screencast plus the chat logs make it very clear and nearly irrefutable when an integrity breach is suspected.
Flexibility Across Different Exam Types
The CSCA examination process is made up of three separate exams that must be passed in sequential order:
- A 100-question multiple choice exam to establish an auditor’s base of knowledge
- A case study followed by 40 multiple choice and short answer questions to simulate an actual social compliance audit
- A one-on-one remote interview, fully recorded
Collaboration in Implementing Solutions
Starr notes her appreciation of working with Meazure Learning as a true partner in defining the exam process and implementing solutions that address their unique situation.
“Some of the most successful parts of the pilot have been the collaboration and teamwork with [Meazure Learning],” she says. “[Meazure Learning has] been really keen to work and learn with us. We’ve made some mistakes along the way, but we’ve also been able to correct along the way.”
Overcoming Language Barriers
A unique challenge faced when creating a global exam process is language. APSCA auditors speak 47 different languages, with 10 of those being spoken by only one person. This is another specific area where Starr values the collaboration with her Meazure Learning team. “[Meazure Learning] has been great in thinking about how we’re going to do that, but I also think it’s going to be one of our biggest issues,” she says.
Conclusion
APSCA’s goal is to have the first round of fully-certified auditors by the end of 2019 and to bring a new level of credibility and value to the social auditing profession. Starr is confident this goal will be achieved.
“We continue to gain momentum with both our stakeholders as well as other interested parties,” she says. “There are a number of brands, retailers, and collaborative initiatives who are already saying only APSCA-enrolled auditors who put their APSCA number on a report are eligible to conduct audits on their behalf.”
With all the challenges faced by previous associations trying to professionalize the social compliance industry, Starr and APSCA felt tremendous pressure to make the global standardized certification a reality quickly, but in the most credible and cost-effective way possible. They’re now certain online proctoring through ProctorU Pro was the only way to meet all of their objectives.