The Monday Excerpt: The Racist Roots of the SAT

Every other Monday, give or take, I will post a brief excerpt from my upcoming book, Choose Wisely: the SAT, the ACT, and You. This is good for you because you get to read, for free, what other people will have to pay for later. This is good for me because it means I have to actually write the book if I'm going to have anything to post! So, win-win.

Today's excerpt continues the story from a few weeks ago. One of the psychologists who worked on the Army Alpha and Beta tests - the first-ever standardized tests administered on a mass scale - has gone on to create the SAT. I'd like you to meet Carl Brigham, the man behind the test.

SAT/ACT Presentation Tomorrow!

This Wednesday, I'm giving a presentation at Washington-Lee High School called - shockingly - SAT or ACT: Choose Wisely! It's going to be fun and informative.  Details below.

NOT the Monday Excerpt

I normally post an excerpt from my book on the SAT and the ACT on Mondays. This post is different for two reasons:

1. It's not Monday
2. It's not an Excerpt

The Clip Show, or You've Already Jumped the Shark

That's right, the Monday excerpt is taking a little break this week! I've been fairly busy with tasks relating to my second career as a photographer, and the book has temporarily been shifted to the... well, I wouldn't quite say the back burner, but perhaps the medium burner. Your stove has one of those, right?


So, in lieu of an excerpt, I will do the blog version of a clip show: the link post!

The Monday Excerpt: "Why are cats useful animals?"

Every Monday I will post a brief excerpt from my upcoming book, Choose Wisely: the SAT, the ACT, and You. This is good for you because you get to read, for free, what other people will have to pay for later. This is good for me because it means I have to actually write the book if I'm going to have anything to post! So, win-win.


This week's excerpt presents the gripping conclusion to the story of the army Alpha and Beta tests - the direct ancestors of the SAT.  If you missed last week's installment, check it out before you read this.

The Monday Excerpt: Uncle Sam and the Psychologists

Every Monday I will post a brief excerpt from my upcoming book, Choose Wisely: the SAT, the ACT, and You. This is good for you because you get to read, for free, what other people will have to pay for later. This is good for me because it means I have to actually write the book if I'm going to have anything to post! So, win-win.

In this week's excerpt, we learn a little bit about the direct ancestors of the SAT: the Army Alpha and Beta Tests.  If there are any psychologists in the audience, please don't be offended by the ire I direct at your profession: it is intended solely for a handful of your less-than-illustrious predecessors. You can read the beginning of this chapter here.

On Tutoring Companies, Part II: Stanley Kaplan's SAT

In this post I will pay my respects to the late Stanley Kaplan of the eponymous Kaplan Test Prep.  He was both the godfather and the midwife of test prep.  All of the countless books and classes out there - including my own - exist only because Kaplan blazed the trail.  He had a very particular view of what the SAT stood for.  Although I don't quite agree with him, I think that his perspective has a lot to teach us about the test.  (If you missed Part I of this series, you can catch it here.)


The SAT as an Engine of Opportunity

Stanley Kaplan, SAT insurgent.

In a fascinating article by Malcolm Gladwell, Kaplan emerges as an earnest, optimistic advocate for the underdog.  His guerrila-style assaults on the SAT's claims of uncoachability, carried out from the basement of his house in Brooklyn, make for a classic tale of American individual enterprise:

Kaplan would have "Thank Goodness It's Over" pizza parties after each S.A.T.  As his students talked about the questions they had faced, he and his staff would listen and take notes, trying to get a sense of how better to structure their coaching.  "Every night I stayed up past midnight writing new questions and study materials," he writes.  "I spent hours trying to understand the design of the test, trying to think like the test makers, anticipating the types of questions my students would face."

The most interesting aspect of Kaplan's crusade, to me, is that it was done not out of cynicism about the test, but out of deep respect.  Although Kaplan knew that the SAT's claim to measure innate ability was false, he also knew that the test provided an opportunity for disadvantaged students to get ahead by the sweat of their brows.  And when young Stanley was getting started, the "disadvantages" a student had to deal with might very well have a racial tinge.  In the 1940s several Ivy League colleges still had de facto Jewish quotas.  If you could do well enough on the new-fangled SAT, though, even those elite schools would have a hard time turning you down.  That is why

Stanley Kaplan was always pained by those who thought that what went on in his basement was somehow subversive. He loved the S.A.T. He thought that the test gave people like him the best chance of overcoming discrimination. As he saw it, he was simply giving the middle-class students of Brooklyn the same shot at a bright future that their counterparts in the private schools of Manhattan had.

Kaplan poopooed the SAT's claim to measure aptitude but idealized the test as an engine for social mobility.  His was a distinctly American vision: if you work hard enough (on the SAT), you'll get ahead.  As I explained in my most recent Monday Excerpt, it doesn't work that way anymore.  Those who do well on the SAT, nowadays, are almost exclusively those who are already pretty well off.


The Moral of the Story


Kaplan was wrong about the SAT's societal function.  His view nevertheless suggests that a national test might yet serve to help hard workers from all backgrounds shine.  Some have suggested using Subject Tests, and I am inclined to agree with them, at least in principle.  That is a matter for another post.  For now, let it suffice to say that Kaplan was decades ahead of his time in publicizing two important truths about the SAT: it doesn't measure innate ability, and it ought to help the little guy (or gal) get ahead.


Coming Up: Herr Professor Johannes Katzmann, Esq.

I had originally planned to talk about The Princeton Review's John Katzman in this same post.  As usual, I had too much to say.  Next time, I will deal with his very different take on the SAT.  Stay tuned!

The Monday Excerpt: Word Games

Every Monday I will post a brief excerpt from my upcoming book, Choose Wisely: the SAT, the ACT, and You.  This is good for you because you get to read, for free, what other people will have to pay for later.  This is good for me because it means I have to actually write the book if I'm going to have anything to post!  So, win-win.


This week's excerpt is the beginning of the chapter in which I discuss the origins of the SAT.  It cuts off right where it starts to get interesting... but over the next few weeks I will post the shocking and exciting continuation!

On Tutoring Companies, Part I: A Conversation with Rebecca Zook

The other day I was having a conversation with my friend and fellow tutor Rebecca Zook when the subject of big tutoring companies came up.  Both of us began our tutoring careers at large companies and left after a few years to form our own independent tutoring businesses.  Figuring I would have an appreciative audience, I started railing away at the evils of Kaplan and The Princeton Review as I often do: with a heaping helping of righteous indignation and a smidgen of self-interest.  I spoke of the fixation of those companies on profit margins, often to the detriment of students; the egregious turnover rates among their tutors; and the huge amount of overhead that is built into their breathtaking hourly rates, only a tiny fraction of which the tutors ever see.  I was just getting warmed up when Rebecca stopped me and with a puckish smile offered to play devil's advocate for the beleaguered mega-corporations.

A company such as Kaplan, she argued, offers several advantages to parents and students over independent tutors:

1. Their tremendous resources allow them to review many tests and do thorough research, the majority of which they make available to students at a reasonable cost through their published materials.
2. They provide a measure of quality control to the tutoring experience: every tutor needs to demonstrate a certain level of competence on the tests and/or in the subjects she hopes to tutor.  An independent tutor might just be some yokel with a craigslist ad.
3. They provide on-the-job training to thousands of fresh-faced college grads every year, many of whom will (like Rebecca and me) use that experience to establish themselves as independent tutors after they leave the company.

I stroked my chin thoughtfully as she made her points; but on the inside I bristled.  As soon as she was done, my populist rant resumed unabated: "Those companies may publish books on the cheap, but they do so mainly to upsell students to their expensive classes and one-on-one tutoring.  That serves to tilt the already imbalanced scales of college admissions even farther in the direction of those who can afford their services.  And they have a disturbing symbiosis with the College Board, which makes more money when people take their tests over and over again!"  Thus I carried on until we at last agreed, for the sake of amity, to talk about only pizza and pawpaws for the remainder of the evening.

Pawpaws: more delicious than tutoring!

Give Credit Where It's Due

Rebecca was right to think that I was unduly harsh on the big companies. After all, I do use some of the Princeton Review's published materials with my own students, since their manuals are usually lucid and light-hearted (and full of typos).  And I certainly would never have been an SAT tutor if Stanley Kaplan hadn't blazed the trail 70 years ago in his Brooklyn basement.  Part II of this mini-series will be an homage to Kaplan and John Katzman, founder of the Princeton Review, each of whom has a lot to teach us about standardized tests.

BUT.  I do believe that parents and students will, in most cases, benefit from choosing small, local companies or individual tutors over the big guys.  In Part III, I will make that case... and I don't plan to pull any punches. Watch out, Stanley Kaplan!  Just because the company you founded makes billions of dollars each year and is basically the only thing keeping the Washington Post profitable doesn't mean I'm scared of you.

The Monday Excerpt: Test Prep and the Stat Game

Every Monday I will post a brief excerpt from my upcoming book, Choose Wisely: the SAT, the ACT, and You.  This is good for you because you get to read, for free, what other people will have to pay for later.  This is good for me because it means I have to actually write the book if I'm going to have anything to post!  So, win-win.

Since I wrote a few posts about the U.S. News rankings last week, I decided to pull today's excerpt from an early chapter in which I write about the onerous effect those rankings have on schools and students.  Enjoy!

Test Prep and the Stat Game

Stroll down the Test Prep aisle at your local bookstore and you'll see shelf after shelf of books about the SAT and ACT. Up your score! Crush the test! Get into your dream school! Each book promises better results than the one next to it; each claims to be your best shot at a more impressive number, a more impressive college, and a more impressive life.

Those books are all part of what I call the Stat Game: the vicious circle of rising test scores and sinking acceptance numbers at America's elite (and, in recent years, semi-elite) colleges. Here's how the Game works, in a nutshell:

1. U.S. News ranks colleges based (in part) on how high the SAT and ACT scores of their incoming freshmen are and how low the percentage of applicants they accept is...
2. Colleges try to get as many students to apply as possible, so that they can accept a handful of high scorers and reject the rest, thereby juking their stats and rising in the rankings.
3. Students freak out en masse, apply to dozens of colleges, spend thousands of dollars on test prep, and buy millions of copies of the U.S. News Best Colleges guide...
4. Repeat steps 1-3.

Once you're hooked on Stats, it's almost impossible to kick the habit. There is tremendous pressure on schools and students to keep up with the pack. The lonely college that tries to buck the trend sees its test scores drop, is punished in the rankings, and as a result gets fewer high-quality applicants the next year. The applicant who dares to submit less-than-stellar SAT or ACT scores is fighting an uphill battle against his or her prepped-up peers.

And that brings us back to those test prep books.

Those books encourage you to take part in the Stat Game unthinkingly. They want you to be as fixated on the SAT and ACT as the college guides are. They want you to feel insecure about your test results, so you'll buy more test prep books and sign up for expensive classes. They want you to play the Game so that they can play you.

Don't do it!  Don't play the game by their rules -- play it by your own rules.

College Rankings (Part III): What Should We Do Instead?

As I explained in my last post, I don't like the effects of the U.S. News rankings on students and colleges, and I would just as soon see those rankings disappear.  But the reams of data about the inner workings of colleges that the rankings have strong-armed into the public eye are certainly a good thing for students who wish to compare schools.  Those 10,000,000 hits that the U.S. News website received when the 2011 rankings came out go to show that there is an insatiable hunger for the information.  Are there alternatives that might help students and parents sift through colleges without the onerous effects of the Best Colleges rankings?  There are three types, two of which exist now and a third that exists in prototype but not in definitive form.

1. Competing aggregate rankings.  U.S. News isn't the only rankings game in town.  Washington Monthly offers its own rankings with a stubbornly alternative vision.  They reward colleges that foster social mobility, public service, and groundbreaking research.  The result is that Berea College (KY) and Morehouse College (GA) beat out big-name schools such as Amherst and Swarthmore.  I wonder, though, whether the social agenda promoted by these rankings really is of interest to students who are struggling to get a sense of what their four years at these various schools will be like.

The website "What Will They Learn?", which a reader of this blog brought to my attention, gives schools a letter grade (A-F) based on the extent to which they require classes in seven core subjects that are central to a liberal arts education.  While I'm not surprised to see the St. John's Colleges at the top of the list, it is certainly eye-opening to see that Baylor, Kennesaw State, and the Coast Guard Academy have more rigorous core curricula than Harvard, Princeton, or Yale.  I looked in vain on this list, though, for any assurance of educational quality beyond the bare requirement of classes.  What good is a core course in math, for instance, taught by a grad student who can barely speak English (an experience I have had more than once)?

I do think that both the Washington Monthly and "What Will They Learn" lists are useful correctives to U.S. News.  My common concern with all three -- and really with any unilateral ranking system -- is that they are chiefly useful insofar as a student shares the viewpoint of the editor of each list.

2. Rankings in individual categories.  College Prowler offers rankings that make it easy for students to compare schools in individual categories, from academics to co-ed attractiveness.  The data come mainly from student surveys, which means that they should be taken with a grain of salt.  For those of an old-school bent, there's also Rugg's Recommendations, which ranks the various undergraduate programs at each college separately.  These methods cede control to the student, who gets to choose which criteria are most important to him or her.  The drawback is that it's difficult to get a sense of how the schools that excel in individual categories stack up overall.

What I would like to see is:

3.  Personalized aggregate ranking.  A ranking system that allows students to choose their own priorities.  I have in mind something akin to those balance-the-budget simulators that have been making the rounds lately.  A student would be able to browse among the various categories in which data are available -- hard data like the student/teacher ratio or graduation rate, but also student survey data -- and assign a level of importance to each category.  Additionally, the student would answer a series of simple questions about their preferences (Rural, urban, or college town?  Deep south or west coast?) that would help narrow the field of choices.  So if a student wants really small classes, doesn't care much about athletics, is looking for a good political science department, and craves gourmet food in the dining halls, she can design her own personalized set of rankings that takes those things into account.  Students unsure about what they want could experiment with the list to see which schools excel in various categories, and then narrow it down once they know more.

Googling turns up a few nascent options along these lines.  The website URanker lets students select the importance of various qualities along a sliding scale.  The options are somewhat limited, and it would be nice if the website displayed the data that go into the rankings more openly, but the clean design makes it extremely easy to use.  StudentsReview offers many more options -- "friendliness" side-by-side with "not being treated as a number" -- but the data seems to be based on a fairly limited number of student surveys, and is thus of questionable reliability.  These websites are excellent starting points, but there is still room for the definitive personalized college ranker.  In fact, given how much colleges like to dump on the Best Colleges list, I'd bet there would be a lot of educators willing to support such a service.  Internet, get on it!

Until this smorgasboard of college data becomes a reality, the closest approximation is to mine the various available ranking systems (this list at College Confidential has most of them) for general ideas, then head to a good college guide or a college counselor you trust.  Also, Washington Post columnist Jay Mathews's book from a few years ago, Harvard Schmarvard, contains a list of schools that are overlooked by the national rankings, but strongly recommended by college counselors.


Please feel free to let me know what you think about the U.S. News rankings (or about this post) in the comments!

(Edit: It would be remiss of me not to link to Malcolm Gladwell's article on the U.S. News rankings from earlier this year.  He takes a closer look at the statistics than I do.  You need a subscription to read the article on the New Yorker website... but you can also (shh!) read it at this link for free.

College Rankings (Part II): Bad (U.S.) News

In my first post on this subject, I wrote a little bit about how influential the U.S. News college rankings (which will be published online tomorrow) are.  I also did my best to come up with some nice things to say about them.  In this post, I take off the kid gloves and lay out a half-dozen reasons why I think the rankings -- at least in their present form -- ought to be canned.  I stop at six only because the post threatened to get out of hand.  If you want to see some of the reasons I left out, check here.  If you want to hear what the other side in this debate has to say, here is U.S. News's defense of their rankings.  And if all this negativity is getting you down, don't worry -- tomorrow I'll offer some armchair generalizing about alternatives (some of which already exist) to the current system!


Six Reasons Why the U.S. News College Rankings Stink


1. Colleges have a perverse incentive to juke the stats.  Even if they're not fabricating data outright, colleges are doing their damnedest to massage the numbers.  As College Confidential reports, "some schools have submitted data that excludes scores from 'special admissions' (e.g., athletes, students identified as learning disabled, etc.).  One school reportedly left out the verbal scores of international students but kept the math scores."  Admissions officers play the numbers game to the hilt because their jobs are on the line.  The Atlantic published an article a few years back that offered a disturbing glimpse of this:

"There's pressure for rankings," says Tom Green, the associate vice-president for enrollment services at Seton Hall. "There's no doubt about it. Presidents get pressure from board members, from alumni. They'll say, You're number eighty-seven. How are you going to get to be number eighty-five?" Eugene Trani, the president of Virginia Commonwealth University, a Tier III school in the U.S. News rankings, carries a laminated card in his pocket to remind him of the school's strategic goal of making it to the next tier. For every year the school stays in the higher tier he will receive a $25,000 bonus—a fact first reported by AGB Priorities, an industry newsletter. A vice-president at Hobart and William Smith was fired when she failed to report current data to U.S. News and the school's ranking dropped from Tier II to Tier III.

Me College Monster!  Me gobble up more students
for better rankings!  OM NOM NOM NOM

2. Students are punished.  Colleges try to boost their average SAT/ACT scores and push down their acceptance rates in order to look good in the rankings.  They encourage students to apply who have almost no chance of getting in, since every mailed rejection letter makes the school look a tiny bit better in the Best Colleges methodology.  As a result, students are getting turned away in droves from schools that, just a decade ago, were safety schools.  The winners are U.S. News (Best Colleges is their cash cow) and the colleges that go up in the rankings; the losers are the students who don't sufficiently boost the stats at their first-choice colleges.

3. Poor students are disproportionately punished.  The Atlantic article from above also reports that "low-income students often suffer in this process. They drag on both revenue and academic profile, and it's hard for outsiders to tell when their numbers are reduced."  As an example of this, observe the importance of SAT and ACT scores in college rankings.  Those tests count for a larger percentage (7.5%) in the Best Colleges metric than class rank (6%), despite their inferiority to high school GPA at predicting academic performance in college.  SAT and ACT scores correlate much, much better with family income than with academic performance for many reasons.  When colleges try to boost their SAT and ACT scores for the sake of a boost in the rankings, poorer students are therefore the first to suffer, and the benefit to the learning environment is questionable at best.


4. The rankings foster cutthroat competition.  The win-at-all costs mentality of Wall Street has spread to high schoolers and their parents.  They spend thousands of dollars on SAT and ACT test prep, some of which goes to yours truly.  A few (I'm sure you've met them) obsess over the rankings and the prestige of colleges, rather than figuring out which schools might actually be a good fit.  This is understandable, given the extreme competitiveness of admissions at any of the top tier schools on the U.S. News list.  I think, however, that we ought to push back against the notion that college admissions are a (deadly serious) game to be won.  For starters, a study tracking the long-term success of students who were accepted at and rejected from "elite" colleges found no difference in their earnings 20 years on.  The only exceptions to this rule were poor and minority students -- another way in which amped-up college competitiveness hurts the disadvantaged disproportionately.  Jay Mathews of the Washington Post wrote a whole book about the silliness of the reverence we have for brand-name schools, arguing that we make an already stressful process even more so.  And Mark Speyer, a college counselor at a prep school in Manhattan, laments the effect of high-stakes admissions on his students' very souls:

I believe that the new obsession with numbers is counter-educational: that it makes for a less educated and less educable kind of student; a less thoughtful, more cynical, more boring, and more exhausted kind of student, not at all the kind of student that colleges on a clear day know they want and need. Pumped-up SATs do not mean better students.

More rejection letters = $$$$$$$

5. The mania for college rankings has spread beyond students and parents.  And I'm not just talking about crotchety alumni.  Bond rating agencies, offered a treasure trove of data about college selectivity, have incorporated it into their risk analysis for institutional borrowing rates.  In other words, colleges that reject a higher percentage of their applicants get to borrow money more cheaply because they are considered a safer investment.  That means colleges would just love for you and all your friends to apply... so they can reject you and then dive, a la Scrooge McDuck, into a vault full of borrowed doubloons.  Do we really want the same guys who helped to cause the recession by green-lighting bad mortgage loan packages influencing how colleges choose whom to admit?  According to William Shain, formerly the Dean of Undergraduate Admissions at Vanderbilt University, "institutions fail or default on financial obligations very infrequently, and I find the reliance on admissions data for these purposes to be hard to justify beyond extreme situations."

6. The rankings are not actually objective.  Those who call the the U.S. News rankings "objective" -- including Mr. Morse -- are suckered by what I call the "Objective Yardstick Fallacy".  A single measure used to assess a complex situation only merits the term "objective" if people can generally agree on the value of the results.  For instance, a yardstick is good at measuring length because we agree on the definition of length and on the units we use to measure it.  When we're measuring weight or electrical current, we reach for a scale or an ammeter and set the yardstick aside without qualms.  If we need to measure a length, a weight, and a current all at the same time, it makes as little sense to combine those results into a single number as it does to indicate by a single number whether one college has more "academic quality" than another.  Not only do we not all agree on what constitutes "academic quality", but the aggregate ranking necessarily embodies the prejudices of the aggregator: it is the polar opposite of an "objective yardstick".  Most students probably don't agree with Best Colleges that alumni giving rate (5% of the total) is more important than student/teacher ratio (1%), for example.  (I plan to write a post at some point about how the Objective Yardstick Fallacy applies to the use of SAT scores in particular.)

What Do You Think?

That's what the U.S. News rankings look like from my perspective (which, in brief, is that of an SAT tutor who loathes the SAT).  I'd love to know what your experience of Best Colleges as a parent, a student, or a teacher has been.  Please let me know in the comments!

College Rankings (Part I): Rank, Indeed

10 million hits in one day.  No, I'm not talking about my blog (yet).  That's how many hits the U.S. News and World Report website received on the day their annual Best Colleges guide dropped last year.  The number is likely to be matched, if not roundly thrashed, next Tuesday, when the 2012 edition makes its debut.  Over the weekend the Washington Post ran an enlightening article on the mild-mannered man who determines the algorithm for ranking colleges.  The author aptly concluded that Bob Morse "is to colleges what Robert Parker is to wine": a man who compelled an entire industry to bow down before the altar of statistics.

In some ways that's a good thing.  Before rankings came along, colleges could handle their business just as they pleased, and no one was the wiser.  They were under no obligation to let prospective students know about student/faculty ratios, graduation rates, or average SAT scores.  The information that did trickle out was fitfully reported and not standardized, so that it was almost impossible to compare across institutions.  Then the rankings came along.  They were a hit, and they put the fear of God (or rather, Bob) into deans everywhere.  A college that failed to supply the requested information could see its ranking drop or get left out of the rankings altogether.  As a result, there is now more information about the internal workings of colleges available to students and parents than ever before.

And that's where my praise for the Best Colleges guide stops.  The rankings, despite the good intentions of Mr. Morse, have contributed substantially to the present overheated climate in college admissions.  Not only do colleges waste time and money each year in jockeying for position -- why wouldn't they, if a modest bump in the rankings can mean a substantial pay raise for the Dean? -- but students suffer as well.  In my next post, I'll outline my principal objections to the rankings.  In a third post, I will then offer my own suggestions for alternatives.

P.S.  Why am I writing about the U.S. News college rankings in a blog that's ostensibly about the SAT and the ACT?  Well, SAT/ACT scores count for 7.5% of a college's rank in the U.S. News methodology.  That's about 7.5% too much, in my opinion.  The use of the SAT and the ACT to rank colleges is just one of a panoply of misuses of those tests that I aim to examine in the coming months.

Firsties!

Welcome to my new blog!  I plan to post my ruminations on tutoring, tests, and my upcoming book on the SAT and the ACT, which is also tentatively titled "Choose Wisely".  Expect excerpts!  Heck, you might even get to read the whole thing for free, if you don't mind small doses.