Tuesday, July 31, 2012

Understanding the MBE Curve

I trust that all of you are warming up to your post-bar exam hangover. Remember having free time and fun? It's nice, right? At least the weather has cooled down, and the Olympics are off and running.
A few readers have inquired about the MBE curve. How, exactly, does it work? Does a tougher MBE mean a better score? July scores are higher than February's, so am I better taking the exam in July? . . . .

No.

Mean Scaled Scores

In 2011, the February mean scaled score was 138.6, whereas in July the mean scaled score was 143.8. Those means scaled scores varied by fewer than two points when compared to February and July mean scaled scores from the preceding ten years. What does that tell us? First, that the levels of performance of applicant pools over the past decade have stayed fairly constant. Second, that a collective group of first-time takers generally score higher than a collective group of repeat takers (i.e., the July mean scaled score is five points higher because the applicant pool is comprised of more first-time takers.). The mean scaled scores tell us nothing more. They become misleading, however, when examinees assume that, by sitting for the July MBE, they will automatically achieve a higher score.Why not? See below.

Raw Score v. Scaled Scores

The MBE is a standardized exam. Each examinee receives a raw score and a scaled score. Raw scores from different MBE administrations (e.g., February 2012 v. July 2012) are not comparable because each administration carries varying degrees of difficulty (i.e., questions from the February administration might be more difficult than those appearing on the July administration.) The NCBE then converts your raw score into a scaled score. The statistical process is complicated and not worth understanding. All you need to know is that scaled scores are comparable from different administrations. Your scaled score represents the same level of performance across all MBE administrations.

For example, a particularly difficult MBE administration means that scaled scores will be adjusted upward to account for the difficulty, more so than the scale adjustment for a particularly easy administration. But the upward adjustment neither gives examinees an advantage nor a disadvantage. A 130 raw score on an easy administration might equate to a 140 scaled score, whereas a 130 raw score on a difficult administration might equate to a 144 scaled score. That makes sense, too, because the levels of performance varied (i.e., a 130 raw on a difficult exam is more impressive than a 130 raw on an easy exam). Again, the scaling of raw scores simply accounts for those variations so that your scaled score represents the same level of performance across all MBE administrations. Your level of performance, which got you a 144 scaled score on the difficult administration, would have gotten you a 144 scaled score on the easy administration, despite that your raw scores would have varied.

Similarly, if you received a 144 scaled score on the February 2009 administration, your level of performance would roughly equal a 144 scaled score on the July 2012 administration. Your 144 score in February 2009 might place you in the 65th percentile of that administration (more repeat takers = lower mean), whereas your 144 score in July 2012 might only place you in the 50th percentile (more first-time takers = higher mean). But either way, you still have a 144.

So whether an MBE administration is easy or difficult, in February or July, in 2009 or 2012, these variables won't affect your scaled score. A scaled score measures your level of performance only, and those variables have no bearing on your level of performance.

As an Aside. . . . How the VBBE uses Scaled Scores

The VBBE creates a raw score based on your essay and short answer point totals. It then converts the raw score into a scaled scale, which is equivalent to the same scale of the concurrent MBE administration. If the mean scaled MBE score is 143.8, the mean scaled written score will also be 143.8. (Again, the July exam tends to have a higher passage rate only because first-time takers generally score higher than repeat takers.) The two scaled scores are then averaged, 60% for the written scaled score and 40% for the MBE scaled score. If your combined scaled score is 140 or higher, you pass.

Let's say, for example, you received a scaled MBE score of 135 and a written scaled score of 142, both low-ish scores but helpful for our purposes here. (135 x .40) + (142 x .60) = 139. FAIL. Now let's say you got just three more MBE questions correct and received a scaled MBE score of 137. (137 x .40) + (142 x .60) = 140. PASS. Alternatively, let's say you sailed through the MBE but bombed the written portion, with 155 and 131 scaled scores, respectively. (155 x .40) + (135 x .60) = 141. PASS. Representing 60% of the Virginia Bar Exam, the written portion is the backbone. But the key to passing -- the heart of it all -- lies with the MBE. The better you do on the MBE, the better your chances are for passing the bar.

2 comments:

  1. I believe my jurisdiction is similar , except we only test the MBE, MPT, and MEE.

    If you don't mind, would you mind taking a look at AR's particular grading scheme. My brain is literally fried and couldn't even put together the calculations and to figure it out exactly.

    Also, would that curve work the same way as discussed above?

    Pass/Fail Determination


    The answers to each essay question and each MPT question will be graded on a scale ranging from 65 to 85. This score shall be designated as the applicant's "raw" score on a question. The raw score on each MPT question will be multiplied by 1.5. The resulting products from the MPT questions will be added to the sum of the raw scores from the essay questions to yield a "total written raw" score.

    The distribution of the total written raw scores acquired by applicants on a given examination will be converted to a score distribution that has the same mean and standard deviation as those same applicants' Multistate Bar Examination scale scores on that examination . The score on this converted scale that corresponds to the applicant's total written raw score shall be designated as the applicant's "written scale" score. An applicant's total examination score shall be determined by the following formula: total score = written scale score + MBE scale score. An applicant shall pass the examination if he or she earns a total score of 270 points or higher

    ReplyDelete
  2. I think your post is very interesting and mostly, I keep looking for like this type of websites where I learn or get new concept. I am happy to visit on your site. Thanks to share it. I was more than happy to find this web site. I need to thank you for your moment due to this unbelievable read!! I definitely savored every bit of it and I have you book-marked to
    see new things in your web site. custom essay writing help

    ReplyDelete