I just checked my bank balance, and saw that ETS had deducted its pound of flesh: thirty dollars to have my recent GRE test scores reinstated. Having verified that transaction, I went over to the My GRE login and checked my online account. There, I saw that my score status has been changed from "canceled" to-- frustratingly-- "absent or unavailable." In other words, I don't have my unofficial score range (based on the previous 200-800 scale). It looks as if I'll have to wait until November for the actual scores (on the new 130-170 scale).
Question: why the new scale? It's apparently divided into 1-point increments, which means there are 41 "shades" of GRE achievement. The old system went from 200 to 800 in 10-point increments, which allowed for 61 "shades" of achievement. It seems to me that the new scale makes for a blockier-looking bell curve. Although I was never sure why the old scale was so weirdly numbered (why not a straightforward 0-100 spread?), I think it was a better scale than the new one is going to be. I can imagine ETS eventually revising the new scale to include half-points in order to increase the fineness or granularity of its test result data.
Or does the new scale somehow work in the test-taker's favor, thereby obviating the need to complain? I can see how that might be the case: with the new, blockier scale, there could theoretically be more people included in the topmost tier-- i.e., those who score a 170-- as opposed to the raw number of people who would have scored an 800. Does this mean it's now easier to get into the 99th percentile?
It's a complicated question. ETS test designers "weight" their problems based on performance statistics garnered through the testing of "guinea pig" test-takers. The net effect of this weighting is that every single GRE test is nuanced in terms of how scaled scores are calculated from raw scores. The only sure notion is that, if you get a perfect score on a Verbal or Quantitative section, you'll get the top scaled score (i.e., an 800 or a 170); anyone falling short of perfect is on his own, and can only guess at what his scores will be. It's possible, then, that the new scoring scale employs a mathematical hocus-pocus resulting in a top-tier slice that's just as thin as it had been when the previous scoring scale was in place. In other words, when it comes to understanding revised GRE score distributions, fucked if I know how it all works.
_
Saturday, September 03, 2011
reinstated
6 comments:
READ THIS BEFORE COMMENTING!
All comments are subject to approval before they are published, so they will not appear immediately. Comments should be civil, relevant, and substantive. Anonymous comments are not allowed and will be unceremoniously deleted. For more on my comments policy, please see this entry on my other blog.
AND A NEW RULE (per this post): comments critical of Trump's lying must include criticism of Biden's or Kamala's or some prominent leftie's lying on a one-for-one basis! Failure to be balanced means your comment will not be published.
Subscribe to:
Post Comments (Atom)
I just can't understand why these "holier that thou" testing companies can't get with the program and realize that 0-100 is just common sense to us common folk.
ReplyDeleteHere's how the old scale originated:
ReplyDeleteThe testmakers wanted scores to fall on a bell curve, with 100 points for each standard deviation. That gave a 600 point spread, but the median (50th percentile) score would have been 300 (on a 0-600 point scale), and "300" just doesn't sound like a median score. So, the median was set at 500, make the lowest score a 200 and the highest score a 800.
Draw a bell curve, with the three standard deviations each way, and you'll see how the scoring originally worked.
Remember a few years ago when the SAT got renormed? That was because, over the years, the median scores were drifting down into the 400's.
With the GRE's, the percentile drifting got terrible. The population of people taking the test has changed over the years, and a huge portion of people taking the test are non-English speakers who are trying to come to the US to study in technical fields. As a result, 8% of test takers were scoring 800 on the math, and the median score had drifted into the mid-400's. If scores were adjusted for the math, many people scoring 800 should really be scoring in the mid 600's, because that's where the bell curve would place them.
Conversely, many of the same people scoring at the top of the GRE math scale were doing miserably on the verbal section, a 680 on verbal was in the 90-something percentile (which is to say, a higher percentile than an 800 on the math.
ETS could have just renormed the scores, the way they did with the SAT, but then grad schools would have a hard time comparing old scores and new scores. Plus, scores are good for 5 years, so there would be a lot of confusion during those years.
So, instead, there are completely new scores. If they'd kept 10 or 100 points for each standard deviation, people would have made incorrect assumptions about converting from old to new scores. The scale of 130-170 will have 150 as the 50th percentile for each subscore, and three standard deviations will fit into the 151-170 and 120-149.
It won't be easier or harder to get in any of the percentiles, because the popultion taking the test is the same. However, it will once again be possible to distinguish among the people scoring in the 92nd to 99th percentile on the Math, which has been impossible for years with so many people hitting the highest score.
ETS isn't going to release the "concordance"(translation from old to new scores) scales until November, when enough people have taken the new test to norm it properly. That's partly why they're offering a discount last month and this one, to entice lots of people to take the new test. They also normed it over the last year by giving new test items to old GRE test takers.
Hope this helps you understand the new scoring, to the extent that anyone can understand it yet.
Elisa,
ReplyDeleteI don't know whether you're an ETS insider who's taken pity on this poor blighted soul, or an ETS veteran who's willing to divulge company secrets, or just a math-savvy citizen with a deep intuitive understanding of how the ETS bureaucracy works, but thank you. Your explanation is better than what I deserve after all my moaning and groaning.
(I still need to understand what a standard deviation is, though.)
Kevin,
ReplyDeletePhew, I'm really glad you didn't mind my screed!
I teach for Kaplan (GRE and GMAT mostly, but I used to teach SAT also), which is why I understand the current scoring issues. But I understand the original scoring constructs because my dad worked at ETS when I was growing up, and I used to work there writing test questions for a couple summers while in college.
I wish I could make a drawing for you of a bell curve, because it's easier to explain standard deviations with a picture. So, draw one. Doesn't have to be perfect, but what you want is a tall wide section in the middle third, with a sharp drop easing to tails on the left third and the right third.
Now, draw a vertical line through the middle of the bell curve. That's the 50th percentile of any set of data. Draw six more lines, three equally spaced to the right (the last line will be at the end of the right tail) and three to the left. 99.9% (or something like that) of all observed data, for any sort of experiment, should fit into a bell curve.
The middle third, which is to say one standard deviation to the left and one to the right of the median, contains over 60% of the whole population. I don't remember the exact number, but it's close to 65%. That means that most people who take, say, the SAT will score in the 400's or 500's.
The next standard deviation has another 30% of the population. In SAT scoring terms, that means that 15% of the test takers get scores in the 600's and 15% get scores in the 300's.
That leaves only about 5% of the population getting the very highest scores (2.5% get scores in the 700's) and the very lowest (2.5% score in the 200's).
Now on the GRE there be about 7 scores per standard deviation. My guess (based on no inside knowledge, just by looking at the bell curve) is that the entire range of scores from 160-170 on the math will be taken up by people who previously would have scored an 800. Even MIT and Cal Tech will be taking people with lower than the highest score.
BTW, I liked your square inside a circle inside a square problem.
Elisa,
ReplyDeleteThanks again. As for that math problem-- it's a collaborative effort between me and a Chinese student of mine, so I can't take all the credit (as much as I'd like to).
You can see who is getting all those 800's on the math section these days. :-)
ReplyDeleteI wrote a blog post on a similar (though simpler) problem, in order to point out the concept of "solving for the negative" on the GRE. It's a useful technique for all sorts of problems, like: "what is the probability in five flips of a fair coin that there will be at least 2 heads?" (Easier to solve for the probability that there will be not be at least 2 heads, then subtract it from 1.)
http://blog.kaplangradprep.com/2010/11/03/geometry-solving-for-the-negative/