Outcomes Assessment

UW-Madison > Provost > Assessment

Home

Provost units

Diversity and Climate

Enrollment management

Office for Equity and Diversity

Extended Programs

Faculty and Staff

Teaching and Learning

Special initiatives

Strategic planning

Department Chair's Toolkit

Policies, reports, memos

Academic Planning and Analysis

Assessment of Academic Programs and Student Outcomes

VI. ASSESSMENT INSTRUMENTS AND METHODS AVAILABLE TO ASSESS STUDENT LEARNING IN THE MAJOR

Assessment of student learning can be conducted using a variety of available instruments and methods. Many experts believe that a combination of assessment approaches can be the most effective way to measure student learning. Fortunately for assessment planners, many departments on campus and at other institutions have acquired some experience with many of the more commonly used instruments. Faculty in a variety of academic programs at large and small research universities have tested and used a wide range of assessment methods to determine whether students were attaining prescribed educational goals. In this section, many of these assessment approaches will be presented providing handbook users with information that can simplify the development of assessment strategies.

Direct Indicators of Learning
Indirect Indicators of Learning

A. Direct Indicators of Learning

1. Capstone Course Evaluation

Capstone courses integrate knowledge, concepts, and skills associated with an entire sequence of study in a program. This method of assessment is unique because the courses themselves become the instruments for assessing student teaching and learning. Evaluation of students' work in these courses is used as a means of assessing student outcomes. For academic units where a single capstone course is not feasible or desirable, a department may designate a small group of courses where competencies of completing majors will be measured.

Capstone courses provide students with a forum to combine various aspects of their programmatic experiences. For departments and faculty, the courses provide a forum to assess student achievement in a variety of knowledge and skills-based areas by integrating their educational experiences. Also, these courses can provide a final common experience for student in the discipline.

Many research universities are currently using capstone courses in a variety of academic disciplines including general education programs and other academic units in the Arts and Sciences. Departments at other research institutions using this instrument to gather information about student learning in the major include many general education programs, chemistry, political science, physics, music, religious studies, theatre, history, and foreign languages.

Relevant Publications

Upcraft, M. L. Gardner, J. N. & Associates. The freshman year experience: Helping students survive and succeed in college. San Francisco: Jossey-Bass Publishers, 1989.

Julian, Faye D. "The Capstone Course as an Outcomes Tests for Majors." Assessment in Practice. Banta, Trudy W., Lund, Jon P., Black, Karen E., & Oblander, Frances W., (Eds). San Francisco: Jossey-Bass Publishers, 1996. pp. 79-81.

2. Course-Embedded Assessment

Assessment practices embedded in academic courses generate information about what and how students are learning within the program and classroom environment. Course-embedded assessment takes advantage of already existing curricular offerings by using standardized data instructors already collect or by introducing new assessment measures into courses. The embedded methods most commonly used involve the development and gathering of student data based on questions placed in course assignments. These questions, intended to assess student outcomes, are incorporated or embedded into final exams, research reports, and term papers in senior-level courses. The student responses are then evaluated by two or more faculty to determine whether or not the students are achieving the prescribed educational goals and objectives of the department. This assessment is a separate process from that used by the course instructor to grade the exam, report, or term paper.

There are a number of advantages to using course-embedded assessment. First, student information gathered from embedded assessment draw on accumulated educational experiences and familiarity with specific areas or disciplines. Second, embedded assessment often does not require additional time for data collection, since instruments used to produce student learning information can be derived from course assignments already planned as part of the requirements. Third, the presentation of feedback to faculty and students can occur very quickly creating a conducive environment for ongoing programmatic improvement. Finally, course-embedded assessment is part of the curricular structure and students have a tendency to respond seriously to this method. Departments at other research institutions using embedded assessment include general education programs, classics, economics, English, film studies, geography, fine arts, history, kinesiology, philosophy, political science, physics, and religious studies.

3. Tests and Examinations

In most cases, a test will be one part of a fully developed assessment plan. Tests are commonly used in association with cognitive goals in order to review student achievement with respect to a common body of knowledge associated with a discipline or program. Departments have traditionally used tests in assessment programming to measure whether students have acquired a certain process- and content-related knowledge.

Using this approach, there are two primary testing alternatives; first, locally developed/ faculty generated tests and examinations, and (2) commercially produced standardized tests and examinations. Locally developed testing and examinations are probably the most widely used method for evaluating student progress. For assessing the validity of an academic program, examinations designed by the instructors who set the educational goals and teach the courses is often the best approach. Cost benefits, interpretation advantages, and quick turnaround time all make using locally designed tests an attractive method for assessing student learning.

Tests designed for a specific curriculum can often prove more valuable when assessing student achievement than commercial instruments. These tests focus on the missions, goals, and objectives of the departments and permit useful projections of student behavior and learning. A well-constructed and carefully administered test that is graded by two or more judges for the specific purpose of determining program strengths and weaknesses remains one of the most popular instruments for assessing most majors. Departments at other research institutions using locally designed tests and examinations include mathematics, physical education, psychology, and English.

Commercially generated tests and examinations are used to measure student competencies under controlled conditions. Tests are developed and measured nationally to determine the level of learning that students have acquired in specific fields of study. For example, nationally standardized multiple-choice tests are widely used and assist departments in determining programmatic strengths and weaknesses when compared to other programs and national data. Compilations of data on the performance of students who voluntarily take national examinations such as GRE and MCAT enable faculty to discover useful data that often leads to programmatic improvements.

When using commercially generated tests, national standards are used as comparative tools in areas such as rates of acceptance into graduate or professional school, rates of job placement, and overall achievement of students when compared to other institutions. In most cases, standardized testing is useful in demonstrating external validity.

There are a number of advantages for using commercial/standardized tests and examinations to measure student achievement; first, institutional comparisons of student learning are possible. Second, very little professional time is needed beyond faculty efforts to analyze examinations results and develop appropriate curricular changes that address the findings. Third, in most cases, nationally developed tests are devised by experts in the discipline. Fourth, tests are traditionally given to students in large numbers and do not require faculty involvement when exams are taken by students.

As part of their assessment efforts, many institutions and programs already use a multitude of commercially generated examination and tests. Some of the more commonly used national tests include:

ACT - COMP (College Outcome Measures Program): This is an assessment instrument that measures knowledge and skills acquired by students in general education courses. Administered by ACT, Iowa City, IA.

GRE (Graduate Record Examinations): The GRE is widely used by colleges, universities, departments, and graduate schools to assess verbal and quantitative student achievement. Also, many discipline-specific examinations are offered to undergraduate students in areas such as Biology, Chemistry, Education, Geology, History, Literature, Political Science, Psychology, and Sociology. The GRE is published and administered by Educational Testing Services, Princeton, New Jersey.

Major Field Achievements Tests: Major field examinations are administered in a variety of disciplines. They often are given to student upon or near completion of their major field of study. These tests assess the ability of students to analyze and solve problems, understand relationships, and interpret material. Major field exams are published by Educational Testing Services, Princeton, New Jersey.

Departments with a successful history in using commercial tests and examinations include many general education programs, mathematics, chemistry, biology, computer science, geology, physics, psychology, sociology, education, engineering, foreign languages, music, exercise science, and literature.

Relevant Publications

Anthony, Booker T. "Assessing Writing through Common Examinations and Student Portfolios." Assessment in Practice. In Banta, Trudy W., Lund, Jon P., Black, Karen E., & Oblander, Frances W. (Eds.) San Francisco: Jossey-Bass Publishers, 1996. pp. 213-215.

Kubiszyn, Tom and Borich, G. Educational Testing and Measurement: A Guide for Writing and Evaluating Test Items. Minneapolis, MN. Burgess Publishing Co., 1984.

Popham, W. J. "Selecting Objectives and Generating Test Items for Objectives-based Tests." In Harris, C., Alkins, M., & Popham, W. J. (Eds.) Problems in Criterion-Referenced Measurement. University of California, Los Angeles: Center for the Study of Evaluation, 1974.

Priestley, Michael. Performance Assessment in Education and Training: Alternative Techniques. Englewood Cliffs, NJ: Educational Technology Publishers, 1992.

Osterlind, Steven. Constructing Test Items. Boston: Kluwer Academic Press, 1989.

4. Portfolio Evaluation

Portfolios used for assessment purposes are most commonly characterized by collections of student work that exhibit to the faculty and the student the student's progress and achievement in given areas. Included in the portfolio may be research papers and other process reports, multiple choice or essay examinations, self-evaluations, personal essays, journals, computational exercises and problems, case studies, audiotapes, videotapes, and short-answer quizzes. This information may be gathered from in-class or as out-of-class assignments.

Information about the students' skills, knowledge, development, quality of writing, and critical thinking can be acquired through a comprehensive collection of work samples. A student portfolio can be assembled within a course or in a sequence of courses in the major. The faculty determine what information or students' products should be collected and how these products will be used to evaluate or assess student learning. These decisions are based on the academic unit's educational goals and objectives.

Portfolio evaluation is a useful assessment tool because it allows faculty to analyze an entire scope of student work in a timely fashion. Collecting student work over time gives departments a unique opportunity to assess a students' progression in acquiring a variety of learning objectives. Using student portfolios also gives faculty the ability to determine the content and control the quality of the assessed materials.

Portfolios at other research institutions are widely used and have been a part of student outcomes assessment for a long time. Departments using portfolio evaluations include English, history, foreign languages, fine arts, theatre, dance, chemistry, communications, music, and general education programs.

Relevant Publications

Aubrey Forrest. Time Will Tell: Portfolio-Assisted Assessment of General Education. Washington, DC: AAHE Assessment Forum, 1990.

Belanoff, Pat & Dickson, Marcia. Portfolios: Process and Product. Portsmouth, NH: Boynton/Cook Publishers, 1991.

Black, Lendley C. "Portfolio Assessment." In Banta, Trudy & Associates (Eds.) Making a Difference: Outcomes of a Decade of Assessment in Higher Education. San Francisco: Jossey-Bass Publishers, 1993. pp. 139-150.

Jones, Carolee G. "The Portfolio as a Course Assessment Tool." Assessment in Practice. Banta, Trudy W., Lund, Jon P., Black, Karen E., & Oblander, Frances W. San Francisco: Jossey-Bass Publishers, 1996. pp. 285-287.

Portfolio News. Portfolio Assessment Clearing House, Encinitas, CA.

5. Pre-test/Post-test Evaluation

Pre-test/post test assessment is a method used by academic units where locally developed tests and examinations are administered at the beginning and at the end of courses or academic programs. These test results enable faculty to monitor student progression and learning throughout prescribed periods of time. The results are often useful for determining where skills and knowledge deficiencies exist and most frequently develop. Academic departments at other research institutions currently using this form of assessment to measure student learning include communications, economics, geography, linguistics, theatre, and dance.

6. Thesis Evaluation

A senior or graduate student thesis, research project, or performance paper that is structured by the department to give students an opportunity to demonstrate a mastery of an array of skills and knowledge appropriate to the major can be a useful assessment instrument. Thesis evaluation has been used effectively for program improvement in such disciplines as foreign languages, literature, and the sciences.

7. Videotape and Audiotape Evaluation

Videotapes and audiotapes have been used by faculty as a kind of pre-test/post-test assessment of student skills and knowledge. Disciplines, such as theatre, music, art, communication, and student teaching, that have experienced difficulty in using some of the other assessment methods have had significant success in utilizing videotapes and audiotapes as assessment tools.

B. Indirect Indicators of Learning

1. External Reviewers

Peer review of academic programs is a widely accepted method for assessing curricular sequences, course development and delivery, and the effectiveness of faculty. Using external reviewers is a useful way of analyzing whether student achievement correlates appropriately with departmental goals and objectives. In numerous instances, recommendations initiated by skilled external reviewers have been instrumental in identifying program strengths and weaknesses leading to substantial curricular and structural changes and improvements.

Relevant Publications

Fong, B. The External Examiners Approach to Assessment. Washington, DC: Association of American Colleges. 1987.

2. Student Surveying and Exit Interviewing

Student surveying and exit interviews have become increasingly important tools for understanding the educational needs of students. When combined with other assessment instruments, many departments have successfully used surveys to produce important curricular and co-curricular information about student learning and educational experiences. During this process, students are asked to reflect on what they have learned as majors in order to generate information for program improvement. Through using this method, universities have reported gaining insight into how students experience courses, what they like and do not like about various instructional approaches, what is important about the classroom environment that facilitates or hinders learning, and the nature of assignments that foster student learning.

In most cases, student surveys and exit interviews are conducted in tandem with a number of other assessment tools. In many universities where surveys have been adopted as a method of program assessment, findings have results in academic and service program enhancement throughout campus. Among the departments currently using these methods are general education programs, mathematics, philosophy, social work, speech and hearing science, chemistry, biology, fine arts, geology, kinesiology, and engineering.

Relevant Publications

Lenning, O. Use of Cognitive Measures in Assessment. In Banta, T. W. (Ed.) Implementing Outcomes Assessment: Promise and Perils. New Directions for Institutional Research, no. 59. San Francisco: Jossey-Bass, p. 41-52.

Muffo, John A., & Bunda, Mary Anne. "Attitude and Opinion Data." In Banta, Trudy & Associates (Eds.) Making a Difference: Outcomes of a Decade of Assessment in Higher Education. San Francisco: Jossey-Bass Publishers, 1993. pp. 139-150.

Riess, R. Dean, & Muffo, John A. "Exit Interviews in Mathematics." Assessment in Practice. Banta, Trudy W., Lund, Jon P., Black, Karen E., & Oblander, Frances W. San Francisco: Jossey-Bass Publishers, 1996. pp. 129-131.

Staik, Irene M., & Rogers, Julia S. "Listening to Your Students." Assessment in Practice. Banta, Trudy W., Lund, Jon P., Black, Karen E., & Oblander, Frances W. San Francisco: Jossey-Bass Publishers, 1996. pp. 132-134.

3. Alumni Surveying

Surveying of alumni is a useful assessment tool for generating data about student preparation for professional work , program satisfaction, and curriculum relevancy. As an assessment supplement, alumni surveying provides departments with a variety of information that can highlight program areas that need to be expanded or enhanced. In most cases, alumni surveying is an inexpensive way to gather data and for reestablishing relationships with individuals that want to help the program continually improve.