From the desk of Chris Beauchamp, VP Psychometrics, Meazure Learning
As a young boy growing up in a small logging community in Northern Ontario, I always wanted to be a psychometrician. While my friends dreamed of becoming astronauts, police officers, or secret agents, I’d dream of a career in psychometrics, standard-setting, item response theory, job analyses, and complex data arrays.
That’s what dreams are made of.
And now back to reality.
Everything I’ve said so far isn’t true. To be honest, about 2 weeks before I became a psychometrician, I didn’t even know that such a profession existed. To this day, my family, friends and even my wife (although she knows who Angoff is) have no clue what I do for a living. I doubt I’ll ever be invited to speak at my son’s Career Day. Frankly, some of the people I work with every day don’t get it.
First of all, what is psychometrics?
Broken down, the word psychometrics consists of two basic parts. “Psychology” is the study of the human mind, and “metrics” are ways of measuring, tracking, or assessing some type of performance. Ergo, psychometricians measure the human mind.
This sounds a bit like psychic voodoo, with a hint of astrology, so let me explain further.
Unlike some professions like engineering, psychometricians are not able to measure human traits directly. As a result, a person’s standing on a given trait (e.g., intelligence, extraversion, mathematical ability) must be inferred based on how the person performs on a task, like a written examination for example. In a way, psychometricians are experts in indirect measurement. Anyone can use a ruler, a gas chromatograph or a Geiger counter. Try measuring using a test! It’s not easy.
Psychometricians are employed in a number of fields such as human resources, research, K-12 education and licensure/certification. Although there are a number of nuances within each field, there are many similarities. I’ll focus primarily on psychometricians working in the licensure and certification field. Within this field, a psychometrician’s primary duty is to develop and administer valid and reliable assessment tools (e.g., examinations) that are based on generally-accepted testing standards.
Educational background
There is no educational program called “psychometrics”. As a result, psychometricians usually have a PhD or Masters degree from a range of fields including educational measurement, industrial-organizational psychology, or statistics. Almost every psychometrician fortuitously fell into the profession one day, found the work interesting, and built a career out of it.
What skills does a psychometrician need?
The short answer is that a psychometrician needs to be a math whiz with social skills.
Yes, you read that right. Math and people skills. The ungodly Holy Grail-like cocktail. However, that’s only part of the story. Here are a few more details.
Statistics
Contrary to popular belief, most psychometricians are not high-level statisticians or mathematicians (nor do they carry calculators with them at all times!). Many large testing organizations hire associates or analysts with advanced mathematical skills to fill that void. However, to work in psychometrics, one must be comfortable with numbers, understand which statistical tools to use, and be able to interpret the results. If you have a math phobia, don’t worry. You’re still a good person but probably not meant to be a psychometrician.
Facilitation
Psychometricians are not subject matter experts in the technical content of the assessments they produce (e.g., engineering, law, medicine) although, after a bit of time, they tend to pick up the jargon and could play one on TV. As a result, assessment content is generally developed and validated by committees of subject matter experts (usually volunteers) who meet in person or remotely. Some of these meetings can be very contentious due to differences in professional opinion, and a psychometrician must be able to skillfully facilitate these sessions. Some days, we should probably be wearing a referee’s jersey and be armed with a whistle (or maybe a gavel or club).
Project Management
The development and validation of assessment tools (e.g., creation of an exam) is the end product of a complex set of steps. Although these steps typically follow a linear progression, the budget and time constraints dictate that some of these processes must be run concurrently or expedited in order to finish a project on-time. This work also involves a number of other stakeholders such as regulators, subject matter experts and colleagues. In addition, most of these projects operate on a strict budget that must be monitored.
But what does a psychometrician DO?
A psychometrician’s day-to-day work can take on a variety of tasks depending on the type of organization in which they choose to work. At a high level, psychometricians work to ensure exams are valid and reliable. Their responsibilities may include conducting research, collecting and analyzing test data, developing, administering and scoring exams, as well as providing recommendations for how test results should be interpreted. In their spare time, psychometricians enjoy reading, long walks on the beach, and convincing people that they are not, despite popular belief, employed as a psychopath.
Why does this matter?
Public Protection
Every day, we come into contact with people who must meet certain eligibility requirements to perform a job. This includes bus and taxi drivers, heating and ventilation technicians, automobile mechanics, teachers, police officers, accountants, lawyers and physicians. We take it for granted that these people are all competent and qualified. The truth is that in order to gain entry to these professions, these individuals had to go through specialized training and assessment. In cooperation with regulators, psychometricians ensure that these assessments are valid and reliable, and provide protection to the consumer and to the public.
Fairness to Candidates
Every year, millions of people complete assessments to gain access to a desired profession. These people generally complete an approved education program (which can include post-secondary education or a graduate-level degree), and devote time and money in order to prepare to complete the assessment. For many, this includes a number of sacrifices. The end result of this assessment process for the candidate will either be a source of great satisfaction or devastating news. To protect the candidate’s rights, psychometricians must ensure that assessments are developed in a fair and transparent manner, are free of bias, and offer the candidate the best opportunity to demonstrate his or her competence.
Conclusion
I’ve made numerous requests to my Member of Parliament to designate February 16th (Sir Francis Galton’s Birthday) as National Psychometrics Day. On this magical day, children and parents would sit around and discuss nostalgic stories of testing. Municipalities would have events where people would gather to develop and validate assessments, all while fostering a sense of community. Psychometrics would become the glue that brings people together. Sadly, not only have my requests been ignored, I suspect my Member of Parliament has modified his spam filter to reject my emails. Perhaps my quest was a bit too ambitious, so I leave the reader with this:
To paraphrase Irish author Derek Landy, “We don’t do what we do for the glory or the fame or the credit; we do it for the quiet satisfaction of making the world a better place.”