::  Posts  ::  RSS  ::  ◂◂RSS  ::  Contact

SAT Coaching: What Effect Size?

September 10th, 2014
sat, statcheck

In a discussion on Steven Pinker's piece advocating more reliance on academic qualifications for college admission and specifically test scores, two former SAT tutors objected to his claim that test prep courses "increase scores by a trifling seventh of a standard deviation (with most of the gains in the math component)":

[Pinker's] assertions about SAT score improvements are incorrect. I tutored SAT for about 5 years, at $80-$100/hr. With 12 1.5 hour weekly sessions, a student gets scores up by 60-100 points for math and 50-70 points for verbal. Either I am a genius and incredible tutor or this article is not well researched. — Harriet

Hundreds of points better? A seventh of a standard deviation? How is there such wide disagreement? First, how much disagreement is that? The SAT is scored out of 2400 these days and the standard deviation was 347 in 2013 (pdf), so Pinker is claiming coaching gives people a boost of 50 points on average. This is much less than my friends' claims of 1/2 SD (hundreds of points) being typical with their students.

There has been some research on the efficacy of SAT coaching, and that's where Pinkers's claim is coming from. Unfortunately it's all correlational:

The research supports Pinker's claim of modest effects, with reported gains between 12% and 19% of a standard deviation (42 to 62 points on the 2013 SAT). So why is it so different from my friends' experience as tutors? Some possibilities, in descending order of how likely I think they are to be the issue. Overall, it looks to me like Pinker is right, SAT prep probably doesn't have a large effect, and the apparent effects are just from not having a control group.

(Separately, the next SAT sitting is 10/11. I'm kind of tempted to sign up to take it to see if I can get the minimum score. Is that worth $52.50 and a Saturday morning?)

[1] This and later numbers are based on a total score of 1600 and a standard deviation of 200, which was the target during this period.

[2] This is from 1995-6 data, after the target standard deviation had been raised to 210.

Comment via: google plus, facebook

More Posts:

Older Post:

Newer Post:

  ::  Posts  ::  RSS  ::  ◂◂RSS  ::  Contact