Monday, October 10, 2011

The Monday Excerpt: "Why are cats useful animals?"

Every Monday I will post a brief excerpt from my upcoming book, Choose Wisely: the SAT, the ACT, and You. This is good for you because you get to read, for free, what other people will have to pay for later. This is good for me because it means I have to actually write the book if I'm going to have anything to post! So, win-win.


This week's excerpt presents the gripping conclusion to the story of the army Alpha and Beta tests - the direct ancestors of the SAT.  If you missed last week's installment, check it out before you read this.


Alpha, Beta, Crappa

The tests the psychologists administered to over a million US Army recruits were called the “Alpha” and “Beta” mental tests. They were fairly closely modeled on a kind of IQ test – one of the first ever written – known as the “Stanford-Binet”. The main difference was that the Stanford-Binet test was designed to be given to one person at a time, but the Alpha and Beta tests were intended to be given to many people at once. They were the very first group intelligence tests.


The psychologists' idea was that by asking several different types of questions and combining the results into a single number, they would be able to determine the overall intelligence of each recruit who took the test. They could then rank the recruits from dumbest to smartest and thereby determine who should lead men into battle and who should peel potatoes. The whole enterprise hangs on the nature of the questions: they must somehow capture the essence of intelligence without being much affected by what a person knows. Some of the questions were ancestors of ones that you might recognize from modern IQ tests: number puzzles, analogies, and scrambled sentences. Others, however, looked like this:

Crisco is a:
  (A) patent medicine
  (B) disinfectant
  (C) toothpaste
  (D) food product


Why is beef better food than cabbage?
  (A) it tastes better
  (B) it is more nourishing
  (C) it is harder to obtain

Why are cats useful animals?


Wait just a minute. How is a person's acquaintance with Crisco supposed to measure his or her intelligence? That just measures familiarity with American culture (which consists largely of hydrogenated fats). The second question is extremely debatable and biased against vegetarians to boot. And that third one is just... wow. Yet our psychologists claimed that "these group examinations were originally intended, and are now definitely known, to measure native intellectual ability." Yeah, right.

Was the Beta test, which was designed for illiterate soldiers, any better? Here is a section from a sample test. You are supposed to fill in what's missing from each picture (e.g. a mouth in question 1). Give it a try!

The Beta Test: for soldiers who can't read good
It's tough to tell what these questions have to do with “native intellectual ability.” Do you know what number 18 is supposed to represent? It's a phonograph – an old-timey record player – that is missing its speaker horn. Were you “intelligent” enough to get that one correct? Also, test instructors were instructed not to accept “creative” answers. For example, many soldiers drew smoke coming out of the chimney in question 5; others, especially those from devout Catholic families, put a cross on the front of the house. The only answer accepted as correct, however, was a continuation of the chimney above the top of the house. If intelligence ever means “thinking outside the box”, this test sure don't capture it.

Bad Test

There are many kinds of mental ability that might be useful in the army. One recruit might be a mechanical genius, able to fix an engine or build a radio out of spare parts. Another might have a commanding presence and keen emotional insight - he's a natural leader. A third might just be a book-smart fellow who gets nervous on tests. The army Alpha and Beta tests would clearly underrate the capacities of these sorts of people. The psychologists wanted to reward recruits who were good at doing tests with weird questions (i.e. recruits who resembled the psychologists), but it's hard to see why those people would make good lieutenants in the army.

According to one pissed-off officer, the
recruits … who had received low intelligence test scores often turned out to be fine soldiers. One man … was “a model of loyalty, reliability, cheerfulness, and the spirit of serene and general helpfulness. … What do we care about his 'intelligence'?”
So, as it turned out, the army was highly skeptical of its battalion of psychologists. Relatively few of the 1.7 million people who took the Alpha and Beta tests had their fates determined by their results. The psychologists lost that battle; but they won the war. In 1921, they published an 800-page book declaring their tests a huge success. Most Americans, overawed by such a plump tome, believed them. The foundation was laid for standardized testing on a massive scale. One of the psychologists who helped to administer the tests went on to create the SAT less than 10 years later.

Next week: the story of Carl Brigham.  The guy who wrote the SAT.  He was kind of a jerk.

No comments:

Post a Comment