TIMELINE SPECIFIED BY LEGISLATURE

	1999-2000 trial 1	2000-01 trial 2	2001-02 trial 3	2002-03 pilot	2003-04 final
Define info/tech literacy
Develop measurement strategy
Determine strategy feasibility
Implement assessment program

DEFINITION OF INFORMATION AND TECHNOLOGY LITERACY

During the 1999-2000 academic year, the Information Literacy Competency Standards for Higher Education published by the Association of College and Research Libraries (ACRL) was adopted by the inter-institutional working group as the basic definition of information and technology literacy. Subsequently, this definition was "operationalized" in the form of a scoring rubric to be applied to samples of student work. As initially created, the rubric corresponded very closely to the Standards but its structure has been simplified (and made more generally applicable) for use in the 2001-2002 trial.

DEVELOPMENT OF MEASUREMENT STRATEGY

2001-2002 academic year.

Assessment Colloquy. The autumn 2001 Assessment Colloquy will be devoted to the discussion of the three areas of mandated assessment (writing, quantitative and symbolic reasoning and information and technology literacy) in the context of a probable grant-funded state-wide assessment of critical thinking. It is possible that modifications may be made in the mandated assessment programs to capitalize on the work being done within critical thinking.

Third trial. Samples of student work within Humanities will be collected at each institution and evaluated at the third scoring trial at the Summer Workshop 2002.

2000-2001 academic year.

Working sessions. Three working groups met separately during winter 2001 to specify the type of student work most appropriate for assessment of information and technology literacy, revise the way in which students would be asked to reflect on the process by which they created their work, and continue development of the scoring rubric begun in summer 2000.

Second trial. Student work and reflections were collected at each institution within the same five disciplines as those chosen previously, and subjected to a second scoring trial at the Summer Workshop 2001. It was concluded, first, that the scoring rubric should be simplified both in terms of categories rated and the rating scale used, and second, that technology use at the upper-division level is discipline-specific and it might be desirable to implement a supplementary, cross-discipline assessment of more commonly used technologies. Additionally, discussions were carried out relative to the feasibility of the proposed approach (see below).

1999-2000 academic year.

Assessment Colloquy. The autumn 1999 Assessment Colloquy provided the first opportunity for inter-institutional discussion of the assessment of information and technology literacy. Participants identified three basic assumptions to guide development of an assessment program: 1) Student ability to access and use information is a complex task best observed in samples of student work; 2) Work products should be supplemented by some type of student reflection to provide a more complete picture of the process used; and 3) Because information is stored and used somewhat differently within various subject areas, assessment of information and technology literacy should take place within the academic discipline.

First trial. Samples of student work were collected at each institution within five disciplines (Business, Education, Humanities, Sciences and Social Sciences), along with reflective checklists and/or essays. A preliminary scoring rubric was developed during the first scoring trial at the Summer Workshop 2000. It was concluded, first, that some assignments are better suited than others to demonstrate student use of information resources and technology, and second, that the original specification of the student reflection needed to be redefined.

FEASIBILITY OF MEASUREMENT STRATEGY

Several considerations bearing on the feasibility of the proposed measurement strategy were identified in discussions during the Summer Workshop 2001:

The measurement strategy itself. Because the assessment program is still under development uncertainties remain: 1) The scoring rubric and set of specifications of student work are not yet finalized; 2) Methods for training and norming raters are not yet developed; and 3) It may be necessary to implement a separate assessment of technology use common to all disciplines. A third scoring trial is scheduled for the Summer Workshop 2002.

Institutional commitment. A primary difficulty in conducting scoring trials has been obtaining appropriate student work and the corresponding reflective statements. The assessment requires considerable effort of both faculty and students and to be successful must be integrated into ongoing educational processes.

Validity of the findings. The proposed assessment stragegy calls for evaluation of actual student work in order to ensure the validity of the results relative to the students' educational programs. Because the method is time-intensive, a relatively small number of students will be included each year and generalizations will not be made to disciplines or institutions. It should also be noted that the results of the proposed study will not be useful for inter-institutional comparisons given the large number of factors that contribute to student learning both before and after students enter a given baccalaureate institution.