In my previous animadversions concerning grades, I complained about how positive correlation between classroom scores and future achievement is waning, and I opined that standardized scores would benefit interviewers and mitigate nonsensical school grading. Only fittingly, the efficacy of contemporary standardized exams needs to be critiqued.
Standardized exams truly and dutifully bear the responsibility to provide a metric on which students can be compared, thereby giving institutions insight on applicants' relative success in the future. In theory, the purpose of any exam is to emulate "the ideal exam", a formally defined concept. In reality, however, avarice adulterates the specification. To understand how existing exams are flawed, let us first understand the ideal.
Mathematically, the ideal exam is represented as an exam E for some success predicate S where, for all pairings (A,B) such that A scores higher on E than does B, then A does better at S than does B. Obviously, when E=S, we have a trivial solution of prediction. The commercial value of E exists only when the results for E can be attained long before the results for S are realized. Such augury comes at the cost of accuracy. An imperfect E can only try to maximize the percentage of its better-worse pairings (A,B) which satisfy the condition that A does better at S than B. When the percentage is over 50%, then we say E positively correlates with S. Since a random set would have 50% of the pairings satisfying the relative-success criteria, E must have a positive correlation significantly above 50% to avoid being dismissed as a joke.
Generally, any university aims to admit students who will excel at the curriculum provided by the university. To aid this endeavor, standardized tests like the SAT try to provide positive correlation with doing well in an undergraduate curriculum. Yet, certain universities, feeling behooved to reanalyze their past students in an attempt to improve the selection process, quantitavely discovered that the SAT correlates more poorly than ACT, AP exams, and other standardized scoring means in predicting undergraduate performance. For quite some time it had been popular for schools to simultaneously accept ACT or SAT, but the growing futility of the SAT goaded the powerful University of California system to almost abandon the SAT altogether.[1] At the risk of losing the patronage of Berkeley, UCLA, and others in the UC system, and for fear such a UC decision would cascade to Texas and New England, ETS has dangerously pulled a prototype exam away from scrutiny and is today recasting it as the new SAT.
Having taken the SAT II subject tests MathIIC, Writing, and Chemistry, I do feel that ETS is correct in principle to strengthen the SAT with the essay portion of the Writing exam and the slightly more difficult math questions from the MathIIC exam. However, the triviality of each question and the pressures of time make the premise of the SAT as irremediably meaningless as a thousand-question examination of single-digit addition problems. Speed at so automatable a task can hardly correlate with any professional success.[2] The AP exams - also overseen by ETS - ask fewer, more arduous questions and are akin to undergraduate finals. Why will ETS not follow their own standards for crafting AP exams when they ponder on the SAT standard? Profits, perhaps. Despite ETS being a nonprofit organization, they have US$153mil in their coffers,[3] mostly from the inexpensive to manufacture SAT exams with comparatively high test-taking fees. Moreover, modeling the SAT after the costlier AP exams would necessitate fees many would consider exorbitant,[4] and would therefore shoo ETS customers to rival test makers. While the problem can be remedied by pledging to operate on a wafer-thin profit margin, ETS is unlikely to be as charitable as its tax-exempt status stipulates it be. It is a pity such pecuniary aspects plague academia so.
Special Mention: The inspiration for this blog, and most of the facts and figures, come from an NPR article by Bob Schaeffer, SAT: A Cynical Marketing Ploy. In NPR's defense, despite Schaeffer's harrowing tale of the deleterious effects of the SAT, there are numerous NPR stories giddily espousing the exam and its new format.[5]
No comments:
Post a Comment