Introduction ASSESSMENT STRATEGIES & INSTRUMENTATIONUniversities, schools, colleges, and departments at UW-Madison have traditionally engaged in a wide variety of evaluation processes for their major programs of study such as qualifying and course examinations, teaching evaluations, course evaluations, and program reviews. Recently, the use of additional evaluative methods and instruments for measuring student outcomes are being added as effective means of assisting faculty and departments in determining whether students are achieving the educational goals and objectives. All of these forms of evaluation produce a variety of results attempting to measure student learning and progression in the academic major.During the 1996-97 academic year, departments incorporated a wide variety of evaluative assessment strategies and techniques. For some departments, developing assessment plans provided faculty with an opportunity to formalize existing assessment practices while adding new assessment initiatives and adjustments to strengthen academic programs. For other departments, this opportunity created a forum for reviewing curricular and programmatic needs and for determining essential educational goals and objectives that their students are expected to achieve. In developing assessment strategies, departments, schools, and colleges have identified an assortment of useful instruments and methods. The most commonly adopted assessment methods include:
Many of the most useful departmental assessment strategies have incorporated a combination
of several of these methods into their plan to assess student learning over a period of years. By
using an assortment of assessment techniques, departments are able to conduct a more
comprehensive analysis of student achievement, satisfaction, and curricular effectiveness. Student surveys and exit interviews have been identified by various schools, colleges, and departments as useful instruments at the outset of assessment programming for assessing student achievement and satisfaction. Questioning students during and near completion of academic programs about the strengths and weaknesses of the program can provide essential feedback for making curricular enhancements and recognizing programmatic problems. For many departments, surveys are seen as a good way to gauge student perceptions of the major. UW-Madison departments, schools, and colleges using student surveying and exit interviews include: Schools & Colleges Business (Senior Assessment Survey, Graduating Master's survey) Journalism and Mass Communications (Undergraduate students survey) Medicine (End-of-year student questionnaire) Nursing (Graduating seniors and ETS program assessment survey, graduate students) Pharmacy (Exit interviews) Veterinary Medicine (Student questionnaire)
Departments
Alumni and employer surveys are an effective way to measure programmatic qualities from former students and many external constituencies. They are also a useful way to generate data about student preparation for professional work and curriculum relevancy. A number of schools and colleges are using or anticipate using alumni and employer surveys as a way of determining a variety of educational and programmatic objectives. UW-Madison departments, schools, and colleges using or student surveying and exit interviews include:
Schools & Colleges
Departments
Nationally standardized multiple-choice tests have been identified as a good means of evaluating student outcomes in many programs and disciplines. National examinations and standards enable faculty to use assessment results as comparative tools to monitor the progress of their students in many academic areas. UW-Madison departments, schools, and colleges using national standardized tests include: Schools & Colleges Law (Student success on the Bar Examination) Medicine (Clinical examinations) Nursing Pharmacy (Student performance on national Pharmacy and state licensure examinations are analyzed annually) Departments Biochemistry Bio. Syst. Engineering French and Italian (Undergraduate student proficiency test) Kinesiology Nutritional Sciences Spanish and Portuguese (National standards) Locally Devised Examinations Locally developed examinations are a good means of evaluating student outcomes in many areas. A well-constructed and carefully administered test that is graded by two or more judges for the specific purpose of determining student achievement is a widely accepted and popular instrument for assessing most majors. UW-Madison departments, schools, and colleges using locally developed examinations include: Schools & Colleges Medicine (Clinical examinations)
Departments Capstone Courses Capstone courses are potentially significant assessment tools because they integrate knowledge, concepts, and skills associated with an entire sequence of study in a program. While some departments already have capstone courses in place making it easier to incorporate assessment practices, others are introducing these courses for the first time. It is anticipated by the department implementing capstone courses that assessment information obtained from students in the course will provide valuable data for addressing program improvements. UW Madison departments using capstone courses include:
Departments
Embedded testing is a method of assessment that has proven to be effective and efficient because of the limited amount of time required to implement the process. In this method, questions intended to assess student outcomes are incorporated or embedded into final exams, research reports, and term papers in higher-level courses. The questions are then evaluated by two or more faculty to determine whether or not the students are achieving the prescribed educational objectives. UW-Madison departments using capstone courses include: Departments Biology Core (Final examination for undergraduate students) Bio. Syst. Engineering East Asian Languages and Literature (Undergraduate students) Mathematics (Undergraduate students) Philosophy (Undergraduate students) Scandinavian Studies (Undergraduate students) Soil Science
Using portfolios as an assessment method takes advantage of work students already are scheduled to do in class. In most cases, portfolios are characterized by collections of student work that exhibit to the faculty and the student the student's progress and achievement in given areas. Included in portfolio assessments may be research papers, reports, essay and multiplechoice examinations, self-evaluations, personal essays, journals, etc. The information is most often gathered in class but also can be collected as out-of-class assignments. A number of departments and schools and colleges are either using or planning on using portfolios to assess student learning in the major. Among the departments, schools, and colleges that are collecting student portfolios include:
Schools & Colleges
Departments
A graduate student or senior thesis, research project, or performance paper that is structured by the department to give students an opportunity to demonstrate a mastery of an array of skills and knowledge appropriate to the major or have proven to be useful assessment instruments. By evaluating a number of such projects, departments are able to discern general patterns of student performance that might lend insight into the strengths and weaknesses of the program as a whole. UW-Madison departments using student essays, projects, or thesis include: Departments Atmospheric and Oceanic Sciences Classics (Master's papers periodically reviewed) Comparative Literature (Honors thesis for undergraduates) French and Italian (Undergraduate student paper evaluations, randomly selected) German (Periodic review of graduate student doctoral dissertations) History (Undergraduate students, Honors Thesis) History of Science (since 1996-97: Scholarly paper comparisons for graduate students)
Peer review of academic programs is a widely accepted method of assessing curricular sequences, course development and delivery, and the effectiveness of faculty. Using external reviewers is a useful way of analyzing whether student achievement correlates appropriately with departmental objectives and goals. In numerous instances, recommendations by skilled external reviewers have been instrumental in identifying program strengths and weaknesses. Departments, schools, and colleges that have successfully used external reviewers or are planning of using them for assessment purposes include:
Schools & Colleges
Departments
Many professional accreditation associations require that their academic members use a variety of assessment instruments to measure student and programmatic achievement. Professional schools and colleges face increasing pressures from accreditation agencies and other peer agencies to change academic programs to better coordinate national and regional curricular requirements. Information derived from these assessments allows these associations, departments, and faculty to compare and contrast academic strengths and weaknesses of various programs, schools, and colleges. Even though much of the assessment data is gathered to meet externally prescribed specifications, it is possible for academic units facing these externally driven pressures to use existing assessment techniques and results for internal programmatic improvements.
At UW-Madison, a number of departments, schools, and colleges undergo periodic
accreditation reviews and use these as an ongoing source of assessment data. The Schools of
are required to use national outcomes assessment methods to assess student learning and
program success. In these instances, Nursing, Pharmacy, and Engineering have many externally
defined educational goals and objectives that need to be regularly addressed in addition to the
objectives established by their own departments and faculty. UW-Madison schools and
colleges using performance evaluations include:
Schools & Colleges
Performance evaluations have been used by faculty as a kind of pre-test/post-test assessment of student skills and knowledge. Disciplines that have traditionally demonstrated difficulty in developing adequate information about student learning such Theatre, music, and art have had significant success in using performance evaluations as assessment tools. UW-Madison departments, schools, and colleges using performance evaluations include: Schools & Colleges Medicine (Performance evaluations for graduates in residency) Departments Theatre and Drama (Undergraduate students)
Assessment in General Education - Communication and Quantitative Reasoning With the fall semester of the 1996-97 academic year, these new university-wide general education requirements, which include more specific outcomes expectations in communications and quantitative reasoning, became effective. The general education assessment program is now in the process of shifting its focus to analysis of the work being done in the relevant freshman and sophomore courses. We are seeking to develop suitable measurements to help us evaluate both the success of the new curricula and the student achievements in the revised courses. Special emphasis will be placed on assessment involving the A and B level requirements in these two areas. The Quantitative Reasoning Assessment Project uses a pre-test/post-test methodology as the principal means of obtaining feedback. Students are assessed to determine competencies in (a) correctly reading and interpreting problems, (b) establishing appropriate mathematical models, (c) solving for the unknowns in mathematical models, and (d) interpreting the solution with the original context.
The Communications Assessment Project has relied to this point on a portfolio
methodology developed in the years since 1990. That portfolio technique was used to assess
outcomes in four of the courses approved recently as meeting the new B level requirements in
communications; Business 530, Psychology 225, Philosophy 253, and English 215. (Assessment
results were not available at the time of compiling this report.) It is expected that additional
assessment methods will also be used in the next several years.
|