Monday, August 5, 2013

Breaking Down the MBE Curve

A number of readers have inquired about the MBE curve. While I posted on this issue in the past, it's still worth revisiting. 

First, let's squash the rumors. A tougher MBE does not mean you will receive a better score. Nor does taking the bar in July instead of February guarantee you a better score. Nor are you better off taking the bar in July because July scores are higher than February's. It's best to think about the MBE curve from two angles.

First, Mean Scaled Scores

In the last ten years, the February mean scaled score varied by fewer than two points above or below 138.6; in the same ten years, the July scaled score varied by fewer than two points above or below 143.8. What does that tell us? Only two things. First, that the levels of performance of applicant pools over the past decade have stayed fairly constant. Second, that a collective group of first-time takers generally score higher than a collective group of repeat takers (i.e., the July mean scaled score is five points higher because the applicant pool is comprised of more first-time takers.). 

The mean scaled scores do tell us anything else. They become misleading when examinees assume that, by sitting for the July MBE, they will automatically achieve a higher score. Continue reading.

Second, Raw Score v. Scaled Scores

The MBE is a standardized exam. Each examinee receives two scores: a raw score and a scaled score. Raw scores from different MBE administrations (e.g., February v. July) are not comparable because each administration carries varying degrees of difficulty (i.e., questions from the February administration might be more difficult than those appearing on the July administration, or vice-versa.) The NCBE then converts your raw score into a scaled score. The conversion is a complicated statistical process that is not worth understanding. All you need to know is that scaled scores are comparable from different administrations. Your scaled score represents the same level of performance across all MBE administrations. The scaled score is the only score you will see, and the only one you care about. 

For example, a particularly difficult MBE administration means that scaled scores will be adjusted upward to account for the difficulty, more so than the scale adjustment for a particularly easy administration. But the upward adjustment neither gives examinees an advantage nor a disadvantage. A 130 raw score on an easy administration might equate to a 140 scaled score, whereas a 130 raw score on a difficult administration might equate to a 144 scaled score. That makes sense, too, because the levels of performance varied (i.e., a 130 raw on a difficult exam is more impressive than a 130 raw on an easy exam). Again, the scaling of raw scores simply accounts for those variations so that your scaled score represents the same level of performance across all MBE administrations. 

Similarly, if you received a 144 scaled score on the February administration, your level of performance would roughly equal a 144 scaled score on the July administration. Your 144 score in February might place you in the 65th percentile of that administration (more repeat takers = lower mean), whereas your 144 score in July 2012 might only place you in the 50th percentile (more first-time takers = higher mean). But either way, you still have a 144.

So whether an MBE administration is easy or difficult, in February or July, in 2008 or 2013, these variables won't affect your scaled score. The variables may affect your raw score, but they have no bearing on your actual level of performance (i.e., your understanding of the law). The scaled score will adjust for these variables because a scaled score only measures your level of performance.

As an Aside. . . . How the VBBE uses Scaled Scores

The VBBE creates a raw score based on your essay and short answer point totals. It then converts the raw score into a scaled scale, which is equivalent to the same scale of the concurrent MBE administration. If the mean scaled MBE score is 143.8, the mean scaled written score will also be 143.8. (Again, the July exam tends to have a higher passage rate only because first-time takers generally score higher than repeat takers.) The two scaled scores are then averaged, 60% for the written scaled score and 40% for the MBE scaled score. If your combined scaled score is 140 or higher, you pass.

Let's say, for example, you received a scaled MBE score of 135 and a written scaled score of 142, both low-ish scores but helpful for our purposes here. (135 x .40) + (142 x .60) = 139. FAIL. Now let's say you got just three more MBE questions correct and received a scaled MBE score of 137. (137 x .40) + (142 x .60) = 140. PASS. Alternatively, let's say you sailed through the MBE but bombed the written portion, with 155 and 131 scaled scores, respectively. (155 x .40) + (135 x .60) = 141. PASS. Representing 60% of the Virginia Bar Exam, the written portion is the backbone. But the key to passing -- the heart of it all -- lies with the MBE. The better you do on the MBE, the better your chances are for passing the bar.

1 comment:

  1. I understand the scaling of the MBE generally but just cannot figure out the essay scoring to this day. You know how they grade your essays on a 1-10 scale? How does that end up equaling the 150ish range that you should shoot for? Also, how low would you have to score on average on your essays to bomb (i.e. get the 130 score you mentioned?)