Jefferson Review

"Your Liberty is Our Interest"

Home /  Archives / Links / Quotes / Book Reviews / Advertise /Contact us / Subscribe / Calendar

Kentucky finally gets trustworthy test scores

By Richard G. Innes

Introduction

The most frustrating aspect of the 1990 Kentucky Education Reform Act is its failure to provide credible school measurements.

First, the state experimented with the ill-fated Kentucky Instructional Results Information System (KIRIS) assessments. No one believed the scores, and the Legislature got rid of this assessment in 1998. Then the Commonwealth Accountability Testing System (CATS) was implemented, accompanied by more doubts. Student proficiency skyrocketed under the CATS – in some cases soaring to more than double the proficiency rates reported by federal testing programs.

Finally, in 2006, the Legislature – not educators – decided to get a better handle on our students’ actual performance. This led to the requirement that all eighth-, 10th- and 11th-graders take the ACT Inc.’s EXPLORE, PLAN and ACT assessments, respectively.

This test system – known as EPAS – evaluates a student’s readiness for college and/or for the vast majority of those better-paying jobs that do not require a college degree. ACT’s extensive experience in creating assessments both for college and for the workforce is a testament to the value of EPAS.

EPAS isn’t like CATS

The same educators held accountable for CATS results also control the CATS tests. So educators have every reason to want those results to look as good as possible. This “fox guarding the hen house” situation benefits educators and the bureaucracy, not students.

In contrast, the nonprofit ACT designs the EPAS. No one gets richer if more students take the ACT tests. ACT’s motivation is to meet the need for accurate information about student abilities for its college clients. So, educator self-interest doesn’t play the same role with the ACT as it does with CATS.

Kentucky EPAS results

Results for the 2006-07 academic year for EXPLORE and PLAN are already available, and Figure 1 tells part of the sad – but not unexpected – story. The ACT results for this year’s high-school graduates won’t be available until late summer.

In mathematics, reading and science, only a handful of Kentucky middle or high schools produce an average equal to the EPAS “Benchmark” scores that indicate reasonable preparation for college and/or better-paying jobs. In almost every Kentucky school, the majority of students lack preparation for college and/or the workforce because these three subjects are highly important in today’s marketplace.

The low performance in reading might represent the most serious concern. If students cannot handle the reading load present in almost all college courses, succeeding in any major becomes difficult.

Kentucky schools seem to do better in English, but that may occur because college English courses may be less rigorous than other subjects. As a consequence, the EPAS “Benchmark” score for English is considerably lower than that of other subjects. That means students can more easily achieve passing grades in college English than in other subjects.

More compelling evidence that even college English standards may be too low comes from employers who express disappointment with grammar, spelling, punctuation and the overall writing skills of recent college graduates.

For example, a recent report from the National Commission on Writing says that more than 40 percent of firms responding to its recent survey either offer or require training for salaried employees with writing problems. One firm reports, “We’re likely to send out 200–300 people annually for skills upgrade courses like ‘business writing’ or ‘technical writing.’” Not long ago, a Wall Street Journal publication quoted a Whirlpool Corporation executive who says that too often, MBA candidates the company considers write with “rambling thoughts littered with misspellings and grammatical gaffes.”

It is also notable that students in other states perform much better on EPAS English than Kentucky students. A Bluegrass Institute analysis of the Kentucky Department of Education’s 2006-07 EPAS Excel data files shows that 72.2 percent of Kentucky’s middle schools and 87.4 percent of Kentucky’s high schools get lower EPAS English scores than the national average.

CATS proficiency versus EPAS performance

The new EPAS data also allows another analysis of the scoring difficulty on the CATS assessments. Figure 2 compares the percentage of students scoring “Proficient” or better on the Kentucky Core Content Test (KCCT) parts of the CATS with the percentage of students who score at or above the “Benchmark” scores for each subject on the EPAS tests.

It is clear in Figure 2 that in science, the CATS grading in both middle and high schools looks far too easy.

Inflation also appears in CATS math scores at both school levels, but the inflation rate seems somewhat greater in high schools. Part of this discrepancy might occur because more students drop out by the time schools administer the KCCT in math to 11th-graders versus the number of students remaining in school when PLAN tests 10th-graders. If KCCT math is corrected for this effect, the percentage proficiency drops slightly.

In reading, strong inflation appears in middle school CATS results. But the high school reading results — which show little more than one of three 10th- graders on track for college and/or better jobs — seem accurate.

Thus, in five of the six examples covered by Figure 2, the CATS tests show clear inflation. Moreover, after 17 years of very costly school “reform” efforts, comparison of Kentucky’s performance to the EPAS “Benchmark” scores shows less than one of four students perform adequately in math and science, and far less than half have acceptable reading skills.

Figure 2 also shows the difference in EPAS performance between middle and high schools is rather small. This raises a serious challenge to the often-heard statement that Kentucky high schools notably under perform compared with lower-level schools. EPAS data shows it isn’t true.

Summary

EPAS confirms that Kentucky’s performance in mathematics, reading and science remains unacceptable by providing some of the strongest evidence yet of the serious inflation of CATS scores.

If Kentucky schools respond properly to these EPAS results, their students should start to see real improvement in learning. However, if that does not happen soon, many Kentucky students will face graduation from high school lacking skills required to join – and succeed in – the 21st-century workforce.

— Richard G. Innes is an education analyst for the Bluegrass Institute, Kentucky’s free-market think tank.

Endnotes

¹ The National Commission on Writing, “Writing Skills Necessary for Employment, Says Big Business Writing can be a ticket to professional jobs, says blue-ribbon group,” The College Board, at http://www.writingcommission.org/pr/writing_for_employ.html, accessed June 8, 2007.

² “Poor Writing Skills Top M.B.A. Recruiter Gripes” by Ronald Alsop, College Journal from the Wall Street Journal, at http://www.collegejournal.com/mbacenter/newstrends/20060117-alsop.html, accessed June 8, 2007.

The Bluegrass Institute is an independent research and educational institution offering free-market solutions to Kentucky's most pressing problems.

Weather (Louisville) / Mapquest /  White Pages / Business Search / CNN / Dictionary / E-card / MSN