“The study investigates the relation between people’s personality and the content and style of their writing…” - email from Washington University in St. Louis.

We Judge, and Hopefully Well

Eminem wasn’t the first person to shrug his shoulders in exasperation and say “I am whoever you say I am”.  Humans have a long history of expressing frustration with other people’s perceptions of their character and personality. We warn each other not to judge — a book by its cover, lest you be judged, hastily — or even slightly because, after all,  “who are you to judge?”

Yet it’s imperative that people judge each other, and that we do it proficiently and well if we wish to avoid the type of trouble that comes from failing to accurately assess another person’s character or intentions. The question isn’t why we judge, so much as how. What criteria, besides the obvious ones of intuition and appearance, do we use when forming opinions about another person’s personality and character?

In day-to-day life, we have the opportunity to study one another’s general way of existing in the world. We watch actions and reactions, and ask each other questions in a give-and-take sort of way. Our impressions are usually not based on answers alone, but also on tone and expression.

Popular personality tests like the Big Five O.C.E.A.N test (an acronym for openness, conscientiousness, extroversion, agreeableness, and neuroticism), remove the eyes, ears, and experience from human study, and claim to offer an objective and accurate analysis of personality based on the types of scaled questions discussed in part one.

I wanted to test the accuracy of the Big Five test, not just against my own self-assessment, but against the impressions of my real life friends.  I was also curious what judgments total strangers might form about me based on nothing more than my answers to a series of random questions.  I wondered how much variation would exist between real-life impressions, the judgments of strangers, and test results.  What I found was surprising.

But First, Let Me Tell You Why I Have A Problem With This…

Personality tests like the Big Five were originally devised as a therapeutic resource for psychologists. I have no issue with tests like this being used by psychologists and their clients as part of therapy, where there is face-to-face interaction and the give-and-take of discussion. However, personality tests have worked their way into the mainstream, most harmfully in the field of employment, where some companies weed out applicants based on nothing more than a short Q&A test — which can be highly misleading, if not in many cases wildly inaccurate.

Does a person who enjoys her solitude make a lousy customer service representative? Not necessarily. She may simply cherish being alone at night after a long day of work and mothering.  Will a person who loves museums and art be open to a company’s continuous changes? Maybe not. Perhaps their appreciation for art is based on its traditions rather than its fluidity. Will a person who keeps their desk clean be the most conscientious employee? Or simply the office neat freak, who is more interested in color-coordinated paper clips than in the company’s bottom line?

Of course, many people who take these tests in the course of employment are familiar with what answers are expected, and fill in the blanks accordingly. Those who want a job aren’t likely to tell a prospective employer that they’re  messy, uncomfortable around people, and easily stressed out. Yet,  human resources offices around the country continue to rely on personality tests in order to help inform their hiring decisions.

A well-known coffee shop is one such employer. Their online application process includes a personality test. If you fail the test, your application will not be processed and your name won’t make it to the list of potential hires.  I tested their system by filling out two applications with the information of real people. I answered one personality test truthfully, and one as I imagined a “perfect employee” might answer. The fictional “perfect employee” made the list. I did not.

Real Life Judgments vs. Test Scores

I believe that Washington University’s study would be more accurate if they simply asked writers to rate themselves on the Big Five scale of Openness, Conscientiousness, Extroversion, Agreeableness, and Neuroticism. The saying “no one knows you better than you know yourself”  tends to apply when the people in question are generally rational and lucid. I know, for instance, that while I don’t mind talking to a lot of people at a party, I likely wouldn’t accept the invitation in the first place. I also know that while I procrastinate over chores, I’m one of the first people my friends call in an emergency.

The Big Five test I took for the University pegged me as being more neurotic than 63.3% of others, more open to experience than 82.3%, and more extroverted than 63.6% of others. According to the test, 82.7% of other people are more conscientious than I am, and 74.3% are more agreeable.

In other words, according to the Big Five, I’m a highly-strung, open-minded, gregarious person who can’t be counted on. I can’t think of an employer who would want that particular combination in an employee, can you?

Being familiar with my own strong points and shortcomings, I thought it would be interesting to see how others, strangers and friends alike, would rate me on the O.C.E.A.N scale. The friends part was easy — I simply asked three people who know me well to look at the definitions of the Big Five traits and assign me a score from 1-10, with 10 being the highest. Averaged, my friends rated me:

8.0 - for Openness
6.0 - for Conscientiousness
5.3 - for Extroversion
4.7 - for Agreeableness
5.6 - for Neuroticism

My friends tended to agree with the Big Five’s assessment of me as open but not highly agreeable — but they disagreed that I was more extroverted, far more neurotic, or far less conscientious than average.

Having strangers assess my personality was a bit more difficult. I questioned whether I should use the 60 questions from the Big Five test, but then decided no — I wanted to use the type of questions that real people ask in the day-to-day when they’re more interested in getting to know something about another person.

A psychologist would argue that the questions aren’t specific to the Big Five categories, therefore it would be difficult, if not impossible, for lay people to accurately assess O.C.E.A.N. traits from homegrown Q&A’s. My argument is that in real life people form impressions and make judgments not through  pinpointed analysis, but through a much more diffuse and intuitive set of criteria. A question like, “Mayonnaise or Miracle Whip” may not work for the cause of clinical psychology, but in human interactions, answers like “mayonnaise, preferably homemade or organic” tell us something about a person. Some may think the answer indicates a person who is health-conscious; others may think the person is a snob; still others may think this is a person who puts way too much effort into making a sandwich.

It is as often the minutia of another person’s existence that informs real life judgments. The person to whom health is important may assign extra points in conscientiousness to the person who makes their own mayonnaise — while the person who thinks it’s a waste of time to make what can easily be bought may view the mayonnaise maker as more neurotic than most.

The larger question is — on average –  are the resulting judgments made by strangers  based on nothing more than a random series of Q&A’s, markedly different than the results of psychology’s Big Five test? How far off are the results of strangers vs. friends, or a self-assessed score?

Interestingly enough, only one person who was familiar with what I was doing doubted whether they could judge the Big Five traits based on random Q&A’s — and she’s a friend who majored in psychology. None of the five strangers who participated expressed any hesitation or difficulty in assigning O.C.E.A.N. scores based on my answers to 40 Questions Asked by Readers.

Here are the averaged scores assigned to me by strangers, who are not readers of this blog, were not informed that the answers were written by me, and who did not know the reason for this experiment. Their assignment of points is based on their own perceptions of my answers and a provided description of each of the five traits:

7.8 - for Openness
7.4 - for Conscientiousness
6.6 - for Extroversion
5.8 - for Agreeableness
6.0 - for Neuroticism

Looking back on how my friends rated me, I was surprised to find that perfect strangers — based on nothing more than a series of random Q&A’s — rated me similarly in every category. There was a difference of .2 in Openness, 1.4 in Conscientious, 1.3 in Extroversion, 1.1 in Agreeableness, and .4 in Neuroticism.

The University’s Big Five test would seem to agree with friends and strangers alike that I am more Open and Extroverted than average. Friends and strangers, though, both rated me significantly higher than the Big Five would seem to in the categories of Conscientiousness and Agreeableness, and roughly about the same in the category of Neuroticism.

My self-assessed scores (7.5 for Openness, 6.0 for Conscientiousness, 5.5 for Extroversion, 5.0 for Agreeableness and 6.5 for Neuroticism) were highly similar to the scores given to me by friends — so similar that there is not a full one point difference in any category. However, the assessments by strangers also come very close, with the only significant differences being that they gave me 1.4 more points than myself in Conscientiousness and and 1.1 more points in Extroversion.

My Judgment

While personality tests may have their place in psychology, I don’t believe they are accurate enough to use as a tool in studies like the one being conducted by Washington University, nor should they be used in guiding employment decisions.

When a group of perfect strangers can more accurately glean information about another person’s personality through a blind reading of random Q&A’s than a standardized psychological test can, it’s time to reevaluate not just the accuracy of such tests, but how they are being utilized, and to question the conclusions drawn from their use.

While inaccurate theories sprung from the Big Five test on subjects like the personalities of writers may be fairly innocuous, personality testing in other realms, such as employment, are not.

Had I taken Washington University’s test in the course of a job application rather than for a study, I likely would not have been hired by any company looking for conscientious, agreeable, non-moody personalities.  Also, as previously discussed, according to recent theories, (Pt. 1) many geniuses — who are said to be largely introverted and somewhat hostile — would also find themselves unemployed.

While the coffee shop may not want or need a high I.Q. barista, few highly intelligent people start out at the top. Many work dead-end jobs to pay for college or to support themselves while working on other projects. There are also millions of  people who fall in the spectrum between capable and genius. The artist who works in a factory. The banker who’s messy at home but proficient at work. The 21 dealer who spends his weekends meditating.

The ability to perform at a job well often depends much less on a person’s personality than on their basic abilities and desire to earn a living. I can give a pretty good speech, although it’s one of my least favorite things to do. I can — and have — cleaned up after horses, driven a frozen foods truck, soldered diodes to a circuit board, managed an office, bought media, created advertising campaigns, managed million dollar budgets, ghostwritten a book, and delivered mail. I’ve done all of these jobs proficiently, either utilizing parts of my nature and personality, or working around them.

Of course, it doesn’t take a genius to fake a personality test, but the point is that they are misused, likely to be inaccurate, and sorely out of place in human resource offices. They are simply too full of interpretive holes, too black and white, and too narrow in their definitions of what constitutes positive and negative personality traits.

Footnote: I would like to thank Kayce, Catherine, Kimberly, Allison, and Haley for taking me up on my offer to grade a stranger. They did not know who or why, and I appreciate their interest and participation.  I also appreciate the readers who took the time to answer questions. . .coming up next!