Jefferson Review

"Your Liberty is Our Interest"

Home /  Archives / Links / Quotes / Book Reviews / Advertise /Contact us / Subscribe / Calendar

Make the ACT count

Success of new testing policy hinges on the response of the Kentucky Department of Education

By Steve Newman

Executive Summary

The new law created by Senate Bill 130 (SB 130) requiring all Kentucky public-school students to take the Educational Planning and Assessment System (EPAS) tests from Act, Inc. is the most important education development in our state since passage of the Kentucky Education Reform Act (KERA) in 1990.

The overwhelming bipartisan support that SB 130 received during the 2006 Kentucky General Assembly demonstrates the high regard that state lawmakers and the many other supporters of this new policy have for the EPAS, which includes EXPLORE for eighth-graders, PLAN for 10th-graders, the ACT for 11th-graders and the WorkKeys jobs-skills assessment tests for students entering the workforce.

There are good reasons for this esteem, including the following:

• This policy has a good track record in other states. Colorado and Illinois, which use the ACT as a prominent factor in evaluating their students and schools, have experienced extraordinary improvements in college enrollment and preparation – especially among minorities. These improvements are summarized in this report.

• Kentucky colleges and universities have a high regard for the ACT test and use it for admissions and placement. The ACT is required for all students who wish to enter a public college or university in the Bluegrass State and is used as a factor in granting Kentucky Educational Excellence Scholarships (KEES).

• Many Kentucky school districts have effectively used EPAS tests in the past to provide excellent educational planning and diagnostic feedback for their students.

However, there are concerns among the many supporters of SB 130 about how the Kentucky Board of Education (KBE) and Kentucky Department of Education (KDE) will implement the new law. The easiest way to incorporate EPAS tests into the existing CATS structure is to weight them as only 5 percent of the accountability index used to evaluate schools. Such an approach would clearly be incompatible with both the spirit and letter of the new law.

Instead, this report offers some options for effectively implementing SB 130 in a way that puts prominent emphasis on the ACT, is compatible with the new law and yet keeps Kentucky’s accountability system intact.

Introduction

Legislators in Frankfort and education leaders nationwide hail the passage of Senate Bill 130 (SB 130) as Kentucky’s most important piece of legislation since enactment of the Kentucky Education Reform Act (KERA) in 1990.

SB 130 requires all Kentucky public-school students to take the Educational Planning and Assessment System (EPAS) tests from ACT, Inc., including EXPLORE for eighth-graders, PLAN for 10th-graders and the ACT for 11th-graders, beginning in the 2007-08 school year. The law also calls for ACT scores to be recorded on high-school transcripts and for ACT, Inc. to supply an individual report for each student based on the results of these three tests.

These reports will provide teachers and parents with excellent – and accurate – feedback about the academic progress being made by students. They will also help ensure that students with academic deficiencies are provided with an individual, accelerated learning plan designed to correct these deficiencies. The feedback will also help challenge excellent students by urging them to take accelerated courses, with an emphasis on Advanced Placement (AP) courses.

The new law includes the WorkKeys job-skill assessment tests, which will be made available on an optional basis to students planning to enter the workforce directly, and mandates that WorkKeys scores also be recorded on transcripts.

Strong support provided by the Kentucky Chamber of Commerce, the Kentucky Council on Postsecondary Education and the Kentucky School Boards Association helped ensure passage of SB 130 with overwhelming bipartisan support during the 2006 General Assembly. As a result, Kentucky is the first state in the nation to fully implement EPAS by utilizing the EXPLORE, PLAN and ACT as part of its assessment system.

However, there are concerns about how the Kentucky Board of Education (KBE) and the Kentucky Department of Education (KDE) will implement these new requirements, including how much weight EPAS tests will carry in determining schools’ accountability indexes as part of the state’s Commonwealth Accountability Testing System (CATS).

This report explores some options for incorporating EPAS into CATS. The plan that involves the fewest possible changes in the current system would count the ACT as 5 percent in the accountability index. The flaws in this plan are discussed in detail. An alternative plan is presented. This plan puts prominent emphasis on the ACT, is compatible with the new law, and keeps Kentucky’s accountability system intact.

The easy way out?

The easiest way to incorporate the EPAS tests into the existing CATS structure is to weight these tests at a given level – including the 11th-grade ACT – as only 5 percent of the accountability index used to evaluate schools. This is the same weight currently given to a nationally norm-referenced test at the elementary, middle and high school levels. The remaining 95 percent of the index would remain unchanged, with 90 percent based on the academic factors in the Kentucky Core Content Test (KCCT) and the writing portfolio, and 10 percent based on nonacademic factors.

This plan would make the fewest possible changes in the current system, but it would be incompatible with both the spirit and letter of the new law. State legislators intended – and the many other supporters of SB 130 clearly want – to give much more weight in the accountability formula to the EPAS tests, especially the ACT.

There is good reason to do so. If the EPAS tests were weighted at only 5 percent, then there naturally would not be much incentive for high-school teachers and administrators to focus on these tests. Instead, they would continue to focus on the KCCT.

The EPAS assessments would be viewed as add-on tests in an already over-tested system. And the intervention strategies based on the EPAS tests would be less effective if these tests are weighted at 5 percent in the accountability index. The primary focus would always be on the KCCT.

Also, an unintended, but very clear, message would be sent to Kentucky citizens that our state has two independent educational systems – the K-12 system that values the KCCT and the postsecondary system that values the ACT. The goal of creating a seamless educational system that provides a smooth transition from secondary to postsecondary education would be put out of reach. The remarkable improvements in college enrollment and preparation experienced in Colorado and Illinois, where prominent emphasis is placed on the ACT, would almost certainly not be realized in Kentucky.

Five-percent weighting: Incompatible with the law

Subsection 6 of the new law (KRS 158.6453) created by SB 130 presents a serious dilemma for proponents of weighting the EPAS tests at 5 percent of the accountability index.¹ This portion of the law states that the KDE should only use the KCCT reading, mathematics and science tests to cover topics in the core content not covered by EPAS tests.

On the one hand, proponents could ignore subsection 6 altogether and keep the current policy of using the KCCT to cover the entire core content in reading, mathematics and science. This approach would not only be incompatible with the law, but would result in Kentucky assessing the same content with different tests. It would lead to the type of over-testing that the General Assembly was trying to avoid by including subsection 6 in the law.

On the other hand, proponents could implement subsection 6 and construct KCCT tests in reading, mathematics and science that cover topics not covered by the EPAS tests. Combined with a 5 percent weighting of the EPAS test in 2003, there was a 42-percent increase in the number of Colorado high-school graduates earning an ACT Composite score of 18 (the low end of the range for admission to colleges with liberal policies) or higher.

• Based on ACT scores, there was a 32-percent increase in the number of Colorado graduates prepared for college-level algebra and a 42 percent increase in those who were ready to tackle college-level English composition during the same period.

Illinois

Illinois administers the ACT on the first day of a two-day testing period. The second day is devoted to a state-developed writing, science and social studies test, and the WorkKeys applied mathematics and reading tests.

The state, which implemented the new assessment policy in 2002, uses the ACT and WorkKeys tests to determine accountability for its schools’ mathematics achievement. Two-thirds, or 60 questions, of the assessment is comprised of ACT queries, while the remaining third, or 33 questions, is made up of WorkKeys material. Accountability in other subject areas is determined in a similar manner.

Like Colorado, Illinois has experienced substantial and demonstrable gains in college enrollment and readiness, including:

• Between 2001 and 2003, the number of in-state, ACT-tested fall freshmen enrolled in Illinois colleges increased by 23 percent.

• The number of Illinois high-school graduates earning an ACT Composite score of at least 18 was 27 percent higher in 2003 compared to 2001.

• Twenty-one percent more Illinois high-school graduates were ready for college algebra and 29 percent more were ready to write college-level English compositions in 2003 than had been in 2001, according to ACT scores.

• The number of in-state, ACT-tested minorities enrolled in Illinois colleges rose by 17 percent between 2001 and 2002.

Michigan

Michigan will require all high-school juniors to take the ACT beginning in the 2006-2007 school year.

Michigan’s three-day testing process will use a mix of the ACT, WorkKeys and state-developed tests to determine accountability in each subject area. The ACT and ACT writing tests will be given the first day; the second day will consist of WorkKeys reading and applied mathematics tests plus a state-developed mathematics test; science and social studies tests designed by the state will be given on the third day.

To determine mathematics accountability, Michigan will combine the ACT mathematics test (60 questions), the WorkKeys applied mathematics test (33 questions) and the state-developed mathematics test (14 questions). The majority of the weight will be given to the ACT. Accountability in other subject areas will be determined in a similar manner.

An alternative implementation plan for Kentucky

The alternative implementation plan recommended in this report would be similar to the testing regiments used by Illinois and Michigan, but would be modified to keep Kentucky’s accountability system intact.

Such a plan would use the ACT reading, mathematics and science test as the principal component of the accountability score in each of these academic areas at the high-school level.

KCCT tests would be constructed in each of these academic areas to cover portions of the Kentucky Core Content for Assessment not covered by the ACT. The ACT and KCCT components would be combined to determine the accountability score in reading, mathematics and science. Since other Kentucky accountability subjects are not covered on the ACT, the KCCT would be used exclusively to determine the accountability score in social studies, arts and humanities, practical living and vocational studies, and on-demand writing.

The EXPLORE test would be used to assess eighth-graders the same way the ACT is used at the high-school level. Scores on the PLAN test, which is closely aligned with the ACT, would serve as fairly accurate predictors of students’ future ACT scores. It would be preferable to use the ACT, rather than the PLAN, in determining the accountability index at the high-school level. The PLAN, however, would serve as excellent preparation for the ACT and would undoubtedly enhance ACT scores.

The relative weighting of the ACT and KCCT in reading, mathematics and science would be determined as described in subsection 6 of the law. Suppose, for example, that the KDE compares the ACT content standards with Kentucky’s core content standards in mathematics and determines that the ACT covers 75 percent of the topics in the Kentucky Core Content for Assessment in high-school mathematics. The high-school mathematics version of the KCCT would then be written to assess the remaining 25 percent of the core content. The relative weighting would not be changed until either the ACT or core content standards were changed.

In this case, 75 percent of the formula used to determine schools’ accountability score in mathematics would be based on the ACT with the remaining 25 percent determined by the KCCT. The same approach would be used in reading and science.

A new accountability formula

In the alternative implementation plan, accountability scores for each academic and nonacademic area at Kentucky high schools would be based on the current CATS 0-140 scale, with 100 representing proficiency.

While the accountability index would still be calculated using essentially the existing weights for the different areas, the process could be simplified because a separate score based on the nationally norm-referenced test, currently part of Kentucky’s assessment package, would not be included.

The new formula for determining the state’s high-schools’ accountability index would be as follows:

I = RDx.15 + MAx.15 + SCx.15 + SSx.15 + WRx.15 + AHx.075 + PLx.075 + NAx.10

For each high school’s accountability index (I), reading (RD), mathematics (MA), science (SC), social studies (SS) and writing (WR) scores would each count as 15 percent; arts and humanities (AH) and practical living/vocational studies (PL) scores would each count as 7.5 percent; and nonacademic factors (NA) would count as 10 percent.

Each of the social studies (SS), writing (WR), arts and humanities (AH) and practical living/vocational studies (PL) scores would range between 0 and 140, with 100 representing proficiency, and would be based exclusively on the KCCT and the writing portfolio. The reading (RD), mathematics (MA) and science (SC) scores would also range between 0 and 140, with 100 representing proficiency, and would be based on a relative weighting of the ACT and KCCT in these academic areas.

In the mathematics example discussed above, for instance, the ACT math test would count for 75 percent of the mathematics score (MA) and the KCCT in mathematics would count for 25 percent.

Conclusion

The alternative implementation plan recommended in this report will place prominent emphasis on the ACT at the high-school level, is compatible with changes in the law created by SB 130 and will keep the state’s accountability system intact.

SB 130 is landmark legislation designed to spur dramatic improvements in Kentucky’s K-12 education system. But such reform will occur only with proper implementation of the law that makes ACT scores count for more – much more – than just 5 percent of the accountability index formula.

If the policy is effectively implemented, Kentucky could reap impressive gains in educational achievement comparable to those in Colorado and Illinois. In fact, our state’s improvements could exceed and be more sustainable in the long run because of the inclusion of – and intervention strategies built around – the EXPLORE and PLAN tests for grades 8 and 10.

Such possibilities should motivate advocates for better Kentucky schools to insist that the new law be implemented in a way that helps give our students the best chance to succeed as they enter postsecondary education or the workforce.

– Steve Newman is professor of mathematics at Northern Kentucky University and director of the Kentucky Early Mathematics Testing Program. He has served as a member of the national Content Expert/Employer Panel in mathematics for the American Diploma Project and a consultant for the Achieve, Inc., study of high-school exit exams. He was a member of the Mathematics Postsecondary Placement Policy Group convened by the Council on Postsecondary Education and a member of the Commissioner's Mathematics Focus Group chosen by the Kentucky Department of Education. Endnotes

¹ (6) (a) The Kentucky Department of Education shall conduct periodic studies comparing the standards in reading, mathematics, and science for middle and high school within the Kentucky core content for assessment and the concepts and content measured by the ACT and the high school and college readiness examinations under subsection (4)(a) of this section. (b) If the department determines that reading, mathematics, and science assessments required under subsection (4)(a) of this section are shown to provide direct measures of content standards and concepts identified in the Kentucky core content for assessment, the Kentucky Board of Education shall seek the advice of the Office of Education Accountability, the School Curriculum, Assessment, and Accountability Council, and the National Technical Advisory Panel on Assessment and Accountability regarding reducing the number of questions on the Commonwealth Accountability Testing System. (c) The Kentucky Department of Education shall continue to include open-response or multiple-choice items, or both, that assess student knowledge and skills in reading, mathematics, and science to the degree necessary for adequate coverage of the elements of the Kentucky core content for assessment not covered by the examinations.

The Bluegrass Institute is an independent research and educational institution offering free-market solutions to Kentucky's most pressing problems.

Weather (Louisville) / Mapquest /  White Pages / Business Search / CNN / Dictionary / E-card / MSN