Issue II: Summary, Small Group Discussion

Jack Roush, Ph.D.

University of North Carolina-Chapel Hill

1. General Issues

Why do outcomes assessment?

Outcome studies should be formative" rather than "summative."

Not all assessments need to be done using "formal" instruments. A less structured process may be more valuable.

Perhaps outcomes assessments should include faculty as well as students

We already are providing a enormous amount of data for ASHA, Deans, annual reports, etc.

These data should help us with our outcome studies - they should not be just for someone elseís use.

Assessment must be program-specific. Faculty must be involved so there is a sense of ownership.

Exit interviews (graduate and undergraduate students) can be a valuable part of the self-study process.

The University of Colorado-Boulder reportedly has useful instruments for outcomes measurement.

Should not overlook outcomes data already being collected in the University, outside CSD.

Clarification is needed re: CAA expectations for outcomes assessment.

Outcome measures need to be broad enough to reflect a variety of outcomes. It is important that outcomes not be limited to the course level, but they should also look at outcomes "across the curriculum."

It is difficult to predict what future graduates will need in our rapidly expanding information age; thus, timely outcome measures are difficult to design.

Evaluation of outcomes will be significantly different in medical vs. educational settings.

Those with positive and successful experiences with outcomes assessment should be encouraged to share their experiences.

CSD programs are often well ahead of other departments because of the requirements imposed by CAA. Some feel their programs are doing a good job with outcomes assessment already, through a combination of exit interviews and surveys of graduates and their employers.

Surveys of graduates are considered useful but what is the optimal time to send out questionnaires?

The longer you wait, the lower the return rate. Also, some graduates may be difficult to locate. Most agree that employers should also be surveyed.

Many programs using the survey approach fail to use the data to make curricular adjustments.

Commitment to outcomes assessment is often difficult to achieve because of faculty inertia. Administrative encouragement for this activity is also lacking at some universities. Others put substantial weight on this activity.

More outcome data are needed at both undergraduate and graduate levels, but undergraduate needs are the most acute.

2. Undergraduate Issues

Admission to graduate school can be an indication of "outcomes" for undergraduate programs. This metric is often overlooked (purposely?) as part of the self-study process.

Some universities are attempting to set minimal levels of proficiency for their undergraduate CSD majors; however, come are prevented from doing this by University policies.

There is a great deal of diversity among undergraduate curricula. Outcomes must become part of what we do, i.e., standard procedure.

Perhaps a "portfolio assessment" could be applied as a means of evaluating undergraduate students.

Some donít believe undergraduates should be permitted to take professional courses. What value is a bachelorís degree in communication disorders? Our students question this because we do. Psychology and political science colleagues donít seem to have a problem with their undergraduates. We need to emphasize that the undergraduate degree has value - should not be called "pre-professional."

In contrast, some believe that we should not offer an undergraduate degree in CSD, for the same reason that you donít earn a bachelorís degree in law or medicine: it is not an entry level degree.

Some programs are willing to discourage students who are unlikely to be successful at the graduate level, but this means loss of majors. Students should be assisted in identifying other viable career paths, e.g., early childhood special education. Instead we often create "boat people" with nowhere to go professionally.

It is difficult to consider outcomes assessment as part of self-study, when we generally lack any data on baseline skills when students enter our programs. Often poor performance in our undergraduate programs is due to insufficient preparation earlier in their academic careers (e.g., at the high school level).

3. Doctoral Issues

Outcome measures seem to be especially lacking at the doctoral level. This is due in part to the wide variation in doctoral experiences and career paths of Ph.D. students.

It is imperative that we determine why so few are pursuing the Ph.D., why relatively few who do go on for a Ph.D. ever complete it, and among those who do, why relatively few choose a career in academic teaching and research.

4. ETS Issues

Many universities are looking for an exit examination to evaluate competency of students at the end of their graduate programs. ETS cautions against using their examination as an exit evaluation because you cannot measure program performance by using it.

Standards Council discourages use of the ETS examination as a formative examination but considers it okay for use as a summative instrument. It should not be used as a comprehensive examination.

It is important to remember that students can choose not to report their scores to the University so ETS data are not complete.

Other disciplines get more information on their examinations than we get from ETS, e.g., percentiles, areas of strength, etc.

ETS is currently under revision.

The bar is reportedly lower for SLP than it is for Audiology.

The ETS examination is used by some universities as a substitute for a comprehensive examination, but this is not allowed by others.

4. Role of the Council

The Council can help by developing a general framework for outcomes assessment (specific details can be left to individual programs); however, the diversity of programs (e.g., professional vs. liberal arts) may pose a challenge in developing a common framework.

The Council should develop a closed ListServ to facilitate communication, dissemination, and cooperation among universities in the area of outcomes assessment.