Does the Answer Order Matter on Multiple-Choice Exams?

Surprising version-dependent differences are noted in student performance on certain questions in a standardized general chemistry exam. The exam in question has two versions, on which both questions and answers are ordered differently. For the questions suspected of answer-order bias, the performance is better in ten of twelve cases when students see the correct answer earlier in the list of four choices. Expansion of the study to more abundant data for the same exam from the ACS DivCHED Examinations Institute supports the interpretation of these observations as a primacy effect, characteristic of guessing, but with an aversion for the first choice. However, the single problem that shows the greatest disparity between versions in both data sets is one most chemists would view as so simple as to make guessing unnecessary, hence unlikely as an explanation.