Office of Planning and Budgeting

The recently announced National Governor’s Association initiative ‘Complete to Compete’ outlines a promising plan to create a national set of performance metrics to enhance accountability and shape funding strategies. The NGA, under the leadership of incoming Chair West Virginia Gov. Joe Manchin III, convened a Work Group on Common College Completion Metrics to make recommendations on the common higher education measures that states should collect and report publicly. The goal is to improve college completion rates and overall productivity in a new era of fiscal constraints coupled with unprecedented demand for higher education. Reliable, comparable data within the sector will be key to achieving these goals as NGA and others attempt to identify which policies and practices are tied to successful outcomes.

The initiative has gained supporters across the country, including among the Higher Education Funding Task Force created by Governor Gregoire in Washington this past summer. Below is a summary of the proposed Complete to Compete metrics.

They use the following definitions:

Completion rate: The percentage of individuals who complete a certificate or degree (e.g., associate and bachelor’s).
Attainment rate:
The percentage of a population that has obtained a certificate or degree.
Productivity:
Awarding more higher education certificates and degrees within the same resources, while maintaining quality.

They recommend the following metrics:

OUTCOME METRICS:

  • Degrees awarded: annual number and percentage of certificates, associate degrees, and bachelor’s degrees awarded;
  • Graduation rates: number and percentage of certificate- or degree-seeking students who graduate within normal program time (two years for associate’s degrees; four years for bachelor’s degrees) or extended time (three years for associate’s degrees; six years for bachelor’s degrees);
  • Transfer rates: annual number and percentage of students who transfer from a two-year to four-year institution; and
  • Time and credits to degree: average length of time in years and average number of credits that graduating students took to earn a certificate, an associate degree, or a bachelor’s degree.

PROGRESS METRICS:

  • Enrollment in remedial education: number and percentage of entering first-time undergraduate students who place into and enroll in remedial math, English, or both;
  • Success beyond remedial education: number and percentage of first-time undergraduate students who complete a remedial education course in math, English or both and complete a college-level course in the same subject;
  • Success in first-year college courses: annual number and percentage of entering first-time undergraduate students who complete entry college-level math and English courses within the first two consecutive academic years; and
  • Credit accumulation: number and percentage of first-time undergraduate students completing 24 credit hours (for full-time students) or 12 credit hours (for part-time students) within their first academic year;
  • Retention rates: number and percentage of entering undergraduate students who enroll consecutively from fall-to-spring and fall-to-fall at an institution of higher education;
  • Course completion: percentage of credit hours completed out of those attempted during an academic year.

In order to track whether access to higher education is sacrificed in the name of completion, NGA also recommends the following ‘context’ metrics:

CONTEXT METRICS:

  • Enrollment: total first-time undergraduate students enrolled in an institution of higher education;
  • Completion ratio: annual ratio of certificates and degrees awarded per 100 full-time equivalent (FTE) undergraduate students; and
  • Market penetration: annual ratio of certificates and degrees awarded relative to the state’s population with a high school diploma.

The UW has worked with the State for years in efforts to create a robust performance agreement. As those efforts continue, the influence of a national initiative such as Complete to Compete will be interesting to note.

After five years, $4 million and a lot of effort across many institutions, the National Research Council has released an update to their 1995 assessment of doctoral programs. A Data-Based Assessment of Research-Doctorate Program in the United States analyzes 2005-06 academic year data collected from over 5,000 doctoral programs at over 200 universities. The NRC collected data directly from faculty, students, graduate programs, and institutions. The Graduate School coordinated UW participation in the assessment, which you can learn more about on their website.

Programs are ranked on the same 21 key variables by two different methodologies, the results of which are reported separately. These methodologies were very complex, but, essentially, the “S” (Survey-Based) rankings weight the relative value of the 21 key variables by program, based on faculty ratings of the relative value of each variables in a given discipline. For example, in the physical sciences, the number of external grants won is weighted more heavily than it would be for an English program. Alternatively, the “R” (Regression-Based) ratings are more similar to the traditional ‘reputation ranking’ where faculty were asked to rank a set of random programs, and then the key variables most associated with the highest ranked programs were assigned the most weight in the overall analysis of programs. Both sets of rankings are reported as ranges (e.g. a program might be ranked as somewhere between 3rd and 11th, at a 90% level of confidence).

While many UW programs do well in these rankings, criticisms of both the data and methodology are important to consider. Inside Higher Ed weighs in with an assessment of the ambivalence surrounding the veracity of the rankings, and the UW’s own Dean of Engineering, Matt O’Donnell, released a statement about possible shortcomings. UW Computer Science & Engineering also issued a strong critique, on which the Chronicle of Higher Education reported. The Chronicle also compiled these data in an easy to use format and offered its own analysis of the report’s delay and overall worth.

How meaningful these rankings are will be debated in the days ahead, but there is at least one important and indisputable conclusion included in the report, which is that public universities play an outsized role in educating our nation’s graduate students:

“Seventy-one percent of the programs ranked in the NRC study are in public universities. The proportion of programs in the universities with the largest programs is similar (70 percent). Among the 37 universities that produced 50 percent of Ph.D.’s from 2002 to 2006, 70 percent were public. Although public universities rely increasingly on nonpublic sources of funding, cutbacks in public funding for universities has a powerful effect on doctoral education simply because of how many large Ph.D. programs exist in public universities.”

← Previous Page