18 October 2008


Filed under: Psychology, Reflections — ktismatics @ 8:32 am

It’s eight o’clock Saturday morning and our daughter Kenzie is back at the high school taking the PSAT. The SAT, or Scholastic Aptitude Test, is the standardized entrance exam used by most U.S. universities as part of their admissions criteria. This is the pre-SAT: a practice version of the real thing taken by many if not most college-bound high school juniors. The fact that Kenzie and most of her pals, only sophomores, are sitting over there taking the test has to say something. I don’t believe that Anne and I drove her to it, but we didn’t dissuade her either. Hey, it’ll be fun, I told her the other day as she was deciding at the last minute whether or not to sign up. I happen to find these tests amusing and I suspect she does too, since in her free time she often tracks down mini-IQ tests on the internet and reports her results to us.

Should you guess if you don’t know the answer to a question? This was the focus of the family breakfast discussion this morning. I said yes, but then we wondered whether the test scorers simply count up the right answers or if they impose a penalty for wrong answers. Anne did a quick internet search, and it turns out there is a penalty for mistakes. But how big a penalty? The testers’ recommendation: if you can eliminate one or more options, guess; if you can’t, then leave the question blank. So let’s say each multiple-choice question presents 4 options from which to choose: if you guess randomly you have a 25% chance of getting it right. If you can eliminate one clearly wrong answer then you choose from the remaining 3 options, leaving you with a 33% chance of guessing correctly. That means the penalty must be somewhere between 0.25 and 0.33 per wrong answer. Problem solved.

But this formulation of the problem seems awfully logical, awfully Bayesian in its solution space. What if you’ve just got a feeling that one of the answers might be right? Should you go with your instinct, your gut, the vibe coming at you from the test booklet?

(a) yes
(b) no
(c) only after you’ve eliminated any obviously wrong responses
(d) it depends

For extra credit please explain your answer.


  1. My experience with taking these kinds of tests, and I test very well in them it seems, is that the approach cannot simply be, “which answer is right” but also, “which answer do the test makers want to mislead you into thinking might be right”. This can often be seen by looking at the method of answer framing for questions whose answers you do know. Often there are two kinds of misleading answers. One is wrong because you have made a conceptual mistake that simply sends you off in the wrong direction. The other is kind of misleading answer close to the right answer, but just a bit off. So really the elimination process comes from building a profile across answers, once you get a sense of the test makers strategy. When you have sense of this, guess away. Often you can identify the pair of nearly right answers, where a guess gives you a 50% chance. Of course if you have no bloody idea, then that question should be left blank, to be returned to at the end of the test.

    An important sidenote in test taking like this, which has bearing on guesses. People sometimes guess because they feel that they have to answer each question before preceding to the next, and time pressures build up. Instead, a best approach can be to run through the whole test, answering those questions for which you have a pretty good sense of the two best answers, leaving all others blank. That way you have a strong sense just how much of the test there is (its not endless). Go back then and use a finer tooth comb to address the more confusing questions, after you have a picture of the test in your mind.

    This is probably a much longer answer than you expected, but I find these multiple choice tests fascinating.


    Comment by kvond — 18 October 2008 @ 3:59 pm

  2. Kenzie reported that the proctor told the assembled test-takers in her classroom that you lose points for wrong answers so they shouldn’t guess. I thought that was a shitty instruction based on what the PSAT’s own website says. Different instructions about guessing introduces an uncontrolled source of variance in the results. Maybe I’ll write to the home office and complain.


    Comment by ktismatics — 19 October 2008 @ 4:25 pm

  3. Kvond is right. It especially helps when you recognize where the hard questions are (at the end still, I presume – but they have altered the test in recent years) so you won’t be misled, particularly with the math questions.

    Usually gut answers worked best for me on analogies and sentence correction, while I tried to quickly work backwards on the latter (harder) sections of the math ones.


    Comment by Seyfried — 21 October 2008 @ 8:04 am

  4. Fascinating.

    My first response was to say that one should go with the gut, statistics be damned.

    But Kvond’s comment is insightful and makes sense. It resonates with the tests I’ve taken in past (including the CPA exam): best to go through and answer only those I am confident in and then go back once I have a “feel” and “sense” of the exam.

    Such an approach would still leave some answers that one is uncertain of. I suppose at that point I would proly go with the statistics….unless, of course, the spirit led otherwise!


    Comment by Erdman — 21 October 2008 @ 9:42 am

  5. This uncertain feeling that one of the answers might be right: I suspect it can result from the accumulation of neural firings going down a particular trajectory of thought and memory that hasn’t quite reached the “tipping point” between uncertainty and what suffices for certainty. But the question-writers are tricksters who know the garden paths and dead ends respondents are attracted to, purposely setting up the misleading quiver of “maybe.” E.g., for the word “enervate” they’re liable to include a definition something like “to make anxious,” thereby exploiting the proximity of enervate to nervous in most people’s neural networks.


    Comment by ktismatics — 22 October 2008 @ 10:19 am

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: