by Matt Jaeh, Chief Technology Officer at Meazure Learning
Welcome to Part 2 of our three-part series on how we navigated and continue to manage the impacts of COVID-19.
In the previous post, we discussed how careful planning and our familiarity with technology helped us quickly adapt to travel and business restrictions while also providing additional support for our employees. Read Part 1 now.
Handling Increased Demand While Maintaining Our High Standard of Quality
Because one of our partners who operated heavily in China was affected by government-mandated shutdowns of testing facilities in early January 2020, we started to see the effects of COVID-19 before it was recognized as a global problem. As other countries enacted lockdowns, new and existing partners started to dramatically increase their use of online proctoring services.
Business expansion is usually recognized as a “good” problem. But because COVID-19 was the cause of this expansion, it also negatively affected us and our ability to operate. As demand for online proctoring services surged, our facilities encountered a number of problems: mandated shutdowns, limits to operational capacity and threats to our employees’ health and safety.
With 8 sites around the world before COVID-19 began to spread, our global presence helped. But more proctors alone were not going to manage the demand spike.
The main challenges we faced included the following:
- Improving platform performance
- Streamlining new client onboarding process
- Enhancing test-taker experience
Platform Performance
The challenge of conducting a greater number of proctoring sessions is more complex than simply hiring more people. Our engineering and platform operations teams saw an increase of 134% in traffic during the first few months of the pandemic. Luckily, we already used cloud services, so our ability to horizontally scale our IT platform was relatively simple.
However, this didn’t solve for code or database query inefficiencies, which could dramatically impact platform performance during peak periods. Thanks to platform optimization procedures and real-time monitoring tools, we were able to pinpoint exactly where our code and query inefficiencies resided, paving the way to solving our problems.
The traffic spike also forced our software quality control team to look at load testing from a different angle, helping us create a more effective scaling strategy and prioritize work.
New Client Onboarding
Before the pandemic, we provided a high-touch, white-glove approach to training new exam administrators. Our specialists worked with administrators in a one-on-one setting or—in some cases with higher ed clients—traveled to their campus to train a large group in person. As demand for online proctoring solutions skyrocketed, we saw the number of new administrator accounts created in our platform surge literally overnight—from an average of 17 per day to 137 per day, which is more than a 292% increase in the month between March 9 and April 9, 2020. At that point, we could no longer train administrators individually. We needed a more efficient method of training without sacrificing quality and customer service. But because our training model didn’t account for this, we had to create multiple resources that could be distributed in bulk. We drafted new support articles, step-by-step tutorials inside the proctoring software, FAQs, how-to documents and more to help administrators train on their own. Additionally, we set up and promoted an optional daily recurring webinar series to train all new admins in bulk. This method gave admins a channel for asking questions and solving unique problems as needed.
Test-Taker Experience
As the number of test providers we partnered with grew, the number of new test-takers—who were unfamiliar with our process, procedures and technology—grew too. Understandably, the new test-takers had a learning curve to overcome when it came to using our system, which resulted in a 300% increase in support interactions by April 2020. Additionally, we saw an increase in concerns about how we proctor and how it could impact test-takers’ privacy, partially because the choice of how and where they could take an exam was removed. Test centers were no longer an option, so those who wouldn’t typically take an exam remotely were forced to do just that.
We wanted test-takers to feel confident that their data and privacy were safe with us. To do so, we needed to make our processes and terms of service more transparent. We started by funding an initiative in partnership with academic leaders, educational institutions, test-takers and industry providers. We wanted to emphasize the essential rights that test-takers should be able to exercise while taking a remote test or participating in online learning. The result was a published document called the “Student Bill of Rights,” which seeks open communication about how we can help test-takers feel safe in their remote and digital work.
We also saw an opportunity to examine our user interfaces and improve the user experience from start to finish. Over the course of a couple months, we reduced our support interaction percentages by 40% and decreased time-to-resolution by 98.8% while keeping our proctor satisfaction metrics at or above 89%.
Lessons Learned
While increased demand was a good problem to have, it still had to be addressed and carefully considered. The increased demand affected all aspects of our business. But through careful consideration, research and execution, we turned our challenges into opportunities to improve.
Use real-time platform monitoring systems and commit to application instrumentation.
Take the time to identify inefficient processes.
