Formative And Summative Outcomes Assessment:
What Do We Mean By Doing It With Meaning?

Brooke Hallowell, Ph.D.
Ohio University

With the recent enhanced focus on educational outcomes in accreditation standards of ASHA's Council on Academic Accreditation (CAA) and with all regional accrediting agencies in the Unites States now requiring extensive outcomes assessment plans for all academic units, it is increasingly important that we share assessment ideas and methods among academic programs.  It is also important that we ensure that our assessment efforts are truly meaningful, relevant, and useful to our students and faculty.  I will provide a brief discussion of "meaningful" assessment practices to serve as an introduction to the next two presentations, which will focus on some hands-on means of outcomes assessment in audiology and in speech-language pathology.  I will also review and clarify a few of the key terms we use to discuss and analyze outcomes measures, and then provide some materials to help member programs evaluate their own assessment programs (Appendix A) and strategize for improved assessment practices (Appendix B).

Members of the CAPCSD's Working Group on Educational Outcomes have spent a great deal of time over the past two years discussing and clarifying what it means to engage in meaningful outcomes assessment.  In our 1998 address to the Council (Hallowell & Lund, 1998),  Nan Lund and I summarized that meaningful assessment practices:

Meaningful outcomes assessment programs are advantageous in that they highlight our accountability for student learning, exploit the phenomenon that students learn according to how they are assessed, allow for strategic program planning, lead to more bottom-up than top-down emphases in handling bureaucratic assessment demands, and encourage academic faculty and clinical staff members to communicate and collaborate with one another.

Agreeing on Terminology

Reference to previous presentations to the CAPCSD (Hallowell, 1996; Hallowell & Lund, 1998) may help to clarify details pertaining to the constructs that are most helpful educational outcomes practices. The specific terms used to discuss educational outcomes are variable across contexts.  The way we develop and use assessments methods matters much more than our agreement on the definitions of each of the terms we might use to discuss assessment.  Still, a few key terms are presented here just for the sake of establishing common ground before our further discussion.

Outcomes

We use the term "outcome" to refer to the status of the output of our programs.  We might assess the outcome of student's experiences in a component of a course, in a whole course, clinical practicum or research experience, or in a degree program, as well as in a department, college, institution, or region. With outcomes measures, we address the question of how students are changed by their experience in one of our training programs.  It is important to differentiate between (1) outcomes, which involve the assessment of groups of students to make statements about the effectiveness of one or a set of teaching/learning experiences, and (2) student evaluation, which involves grading or estimation of individual students' accomplishments.  The distinction between outcomes assessment and student evaluation is especially critical when we discuss the use of student evaluations grouped together across classes of students (e.g., graduates from a particular program for a specific year) as indicators of educational outcome.  Some of the specific methods of evaluating individual students to be presented by Karen Richardson and Cynthia Bartlett are mentioned in today's discussion of educational outcomes because, when used across groups of students, they may yield important information about aspects of program effectiveness.  Let us clarify, however, that indices of individual student gains do not constitute outcome measures per se.

There is often some confusion of the term "outcome" in the discussion of accreditation standards as compared to the discussion of certification standards in speech-language pathology and audiology.  Accreditation standards, overseen by ASHA's CAA, involve a focus on outcomes assessment.  Candidates for accreditation are expected to demonstrate that they effectively assess programmatic learning outcomes of students and graduates and that they use their assessments to shape program modifications.  By contrast, certification standards, overseen by ASHA's Council on Professional Standards in Speech-Language Pathology and Audiology, involve a focus on individual accomplishment.  Individuals who wish to be certified are expected to demonstrate specific knowledge and abilities.
 
Formative and Summative Outcomes

Summative outcome assessments are indices of the final product or the end result of an educational program.  They are used to characterize programs, college divisions, or even whole institutions.  By contrast, formative outcome assessments are those that may be used to shape the experiences and learning opportunities of the very students who are being assessed.  A balance between formative and summative assessments is essential to strategic, effective outcomes assessment practice.

Cognitive/Affective/Performative Outcomes

A basic taxonomy of different types of learning outcomes helps us to verify that we are addressing an appropriate blend of different type and areas of learning in our assessments.  Cognitive outcomes are those that relate to intellectual mastery, or mastery of knowledge in specific topic areas.  Performative outcomes relate to a students' accomplishment of behavioral tasks.  Affective outcomes relate to personal qualities and values that students ideally gain.

Assessing Your Own Assessment Practices

Appendix A is presented as a tool for directors and faculty members to evaluate their own assessment practices.  Nan Lund and I originally developed it for the ASHA CAA to help plan for site visitor training in light of new outcomes assessment foci in the accreditation standards that became effective in January of 1999.

Action Planning for Improved Outcomes Assessment

Appendix B is offered in an effort to stimulate specific actions for improved assessment practices among member programs.  Nan Lund and I have used and modified the content of both appendices over the past three years in our consultations with representatives from numerous programs in communication sciences and disorders, as well as with CAA board members and site visitors.

References

Hallowell, B. (1996). Innovative models of curriculum/instruction: Measuring educational outcomes.  In Council of Graduate Programs  in Communication Sciences and Disorders, Proceedings of the seventeenth annual conference on graduate education, 37-44.

Hallowell, B. & Lund, N. (1998).  Fostering program improvements through a focus on educational outcomes.  In Council of Graduate Programs in Communication Sciences and Disorders, Proceedings of the nineteenth annual conference on graduate education, 32-56.

APPENDIX A
EVALUATING OUTCOMES ASSESSMENT PRACTICES WITHIN
AN EDUCATIONAL/TRAINING PROGRAM

Brooke Hallowell
Ohio University

Nan Lund
Buffalo State College

STRATEGIC PLANNING

FACULTY INVOLVEMENT RESOURCES USE OF ASSESSMENT DATA


APPENDIX B
 RECOMMENDED STEPS IN MEANINGFUL OUTCOMES ASSESSMENT

Brooke Hallowell
Ohio University

Nan Lund
Buffalo State College

Cognitive outcomes were defined as those relating to intellectual mastery, or mastery of knowledge in specific topic areas (e.g., ability to describe the basic anatomy of the human hearing mechanism).

Performance outcomes are defined as those relating to the functioning of a student, or of a program graduate, in a professional setting (e.g., ability to select appropriate formal assessment instruments for effective diagnostic problem solving for a variety of
 patient/client types.

Affective outcomes are defined as personal qualities and values that students ideally gain from their experiences during a particular educational and training program (e.g., demonstration of sensitivity to and appreciation of multicultural issues in a variety of contexts).

Continuously: