Defining and Teaching Learning Outcomes

Sunday, June 24, 2001

 

Facilitator: Cynthia Wilson

Recorder: Bill Castellano

Resources: Robert Barr, Robert McCabe, and Ned Sifferlen

 

Learning Outcomes are statements of the knowledge, skills, and abilities the individual student possesses and can demonstrate upon completion of a learning experience or sequence of learning experiences (e.g., course, program, degree).

 

Processes for Defining Learning Outcomes

 

Collegewide Learning Outcomes

 

1. Core Competencies 

2. Research 

3. Structure/Design
4. Process 

§         industry

§         legislative

§         external accrediting bodies

§         national standards

§         K-12 and four-year institutions 

§         board

§         mission/values

§         K-12 and four-year institutions

§         faculty teams (x discipline) 

§         standardized syllabi

§         course for students to teach better consumerism  

5. Resources 

§         money; release time

§         clearinghouse/bank of ideas

§         institutional research for processing, cataloging, and dissemination              

§         consultants with expertise to help others do this

                                               

Program Level Learning Outcomes

 

1. Definition of a program (i.e., something with specific outcomes) 

2. Process: external definitions 
3. Process: internally defined 
4. Guidelines
5. Outcome

 

Course Level Learning Outcomes

 

There are a number of processes across the institution that contribute to the identification and defining of course-level learning outcomes; for example: 

 

 

Roles of Learning Facilitators in Student Achievement of Learning Outcomes

 

Observations

Needs

§         training in design and implementation

§         design resources (design specialists, time, incentives)

§         implementation resources (implementation specialists, time, incentives)  

§         of faculty

§         of others

 

Assessing and Documenting Learning Outcomes

Monday, June 25, 2001

 

Facilitator: Cynthia Wilson

Recorder: Dianne Cyr

Resource: Ned Sifferlen

 

Agenda

·         Revisiting Yesterday

·         Assessment

·         Documentation

 

Revisiting Yesterday

Outline of yesterday’s discussion for review by participants for additions/amendments—stand as written for now; invitation to make changes throughout Session III prior to posting on website.

 

Assessment

 

·         Why do we want or need evidence of student achievement of learning outcomes?

 

·         How are we going to get this evidence?

 

·         What are we going to do with it?

 

·         What do we fear will be done with it, and why does that scare us?

 

·         What can we do to overcome these fears?

 

Assessment Challenges

 

·         Maintaining partnerships

·         Lack of knowledge of methodologies and tools

·         Limited resources (financial, time, personnel)

·         Difficulty of getting buy-in

·         Effective communication at all levels

·         Assessing the assessment

·         How does assessment compete with other institutional and workload priorities?

·         Viewing assessment as integral to our mission as educators at every level and not separate

·         Overcoming our fears

·         Creating incentives and recognition for assessment efforts

 

Kay McClenney’s observations about why the work of learning outcomes is so difficult:*

 

·         lack of collaboration among disciplines and other groups within the institution

·         lack of knowledge about assessment processes and tools

·         lack of awareness of the need for outcomes-based education

·         lack of appropriate, effective assessment tools and models

·         a perception that some important learning outcomes are not measurable

·         traditional insulation from accountability for individual student learning at the classroom level

·         traditional resistance to self-assessment in higher education

·         traditional external requirements for accountability, funding, and policy that are rarely tied to individual student learning, leading to a lack of incentive for outcomes-based efforts

·         increasing demands and constricting resources, which leave little time or incentive for educational reform efforts of this magnitude

 

*from an address to participants in the League’s 21st Century Learning Outcomes Project, March 4, 2001, Atlanta, GA

 

Group Process: Identify strategies you know will work and strategies you think or believe will work.

 

Partnerships

 

·         Interdisciplinary Outcomes Teams

·         Connect and communicate with the movers and shakers and keep them informed of progress and results

·         Communicate well—get all stakeholders involved (faculty, board members, students, media).  Involve as many people as possible on campus and in feeder schools

·         Emphasis on communication, implementation and shared ownership

 

Resources

 

·         The League serves as a clearinghouse for information, creating a resource database

·         The League provides a Database/Evaluation Consultant to work with each of the Vanguard Learning Project colleges to identify common themes and data collection and to work with a college point person for customized data to create a hybrid instrument that can serve multiple purposes

·         League colleges make a commitment to share information on assessment projects

 

Needs

 

·         support from across institution

·         Palomar three-year project stealth approach not as successful. Needs to be a celebration of risking assessment.

·         People need to be able to vocalize fears as part of moving through; goes along with fear of change, going out of comfort zone.

·         Facilitates departmental discussions (Moraine) Needs to be across functionality.

·         Lane has contract that calls out the parameters for undertaking assessment.

·         Madison is trying to design email to college community celebrating success.

·         Kirkwood has efforts to communicate best practices.

·         Departmental review (academic and non-academic) at Moraine Valley

·         Using student/faculty satisfaction measures to improve—Kirkwood multilevel assessment of current practice (levels and stages).  Can target areas that need assistance and provide needed resources.

·         Need to be able to find $ resources and identify untapped internal resources to assist in this effort.

·         Kirkwood—train-the-trainer; brought in national experts to work on this effort (Barb Wolvaard, Notre Dame).

·         Valencia—funds Title III and Title V; about 75 involved.  Has gone through models of assessment; now working from outcomes backward.  A resource library on assessment has been developed, 7 assessment tools have been created to address competencies.  Destination 2001 is the name of the project.

·         Hiring practices need to reflect the learning college culture.  Valencia is moving forward on this—faculty serve on every board.

·         Learning by Design is a two-semester program that measures inclusive learning and critical thinking.  Hypotheses are developed about what students will need to do to demonstrate higher learning outcomes, strategies are applied and outcomes are measured again resulting in higher grades and retention rates.  Used internal faculty to train others.

·         Buddy system is used to call students who are not attending; also work as study groups.

·         Sinclair—NSF architecture of courses requires teamwork, demonstrated transference of information, assessment and external evaluation.

·         Involve adjunct faculty in assessment projects; it is like a grow-your-own for developing new full-time faculty.

·         Use  Classroom Assessment Techniques, Cross & Angelo, for potential assessment tools.

·         Kirkwood—Work with Wolvaard allowed for common rubrics among disciplines and allowed for broader/fuller collection of data.

·         Category of Awards and Recognition—Award for most improved programs (most improved in terms of learning outcomes; not measuring learning outcomes). 

·         Kings College model (Don Farmer-author); course-embedded assessments for core competencies—not overburdensome to faculty at discipline level.  Faculty read other students’ exams (even if multiple choice final) and note items that demonstrate a core competency.

·         Moraine Valley—knows good tool for measuring critical thinking—developing rubrics for beginning, developing and accomplished.  Teach students how to self assess/validate their lifelong learning (how do students know they know rather than thinking they know?).

·         Have students put examples of their work in portfolios—will work but need to tighten up benchmarks (instructor assessment, self-assessment and demonstrating improvement over time).

·         Johnson County CC—interdisciplinary work to come up with rubrics for core competencies, collect artifacts from randomly selected classes—Institutional Research collects assignments and organizes them for reading and evaluation in cross-sectional groups working together.

·         Budding partnerships—Interdisciplinary Learning Outcomes Team—Faculty/Student Services/etc.  All full-time employees are expected to be part of one team (Learning Outcomes, Critical Thinking, Communicating, Active Learning, Interacting in Diverse Environments).

·         Competencies/Values—Core Values need to be the same to develop viable strategies.

 

Documentation

 

In documenting individual student learning, what do users need and want to know?

 

·         Depends upon the audience

·         Digital/electronic portfolio

·         Industry certifications that show competencies

·         Log of experiences in internship or clinical experiences

·         Checklist of competencies within a discipline to accompany the transcript and provide a record of demonstration of individual competencies with level of accomplishment

·         Partnership opportunities with colleges and universities

·         Cocurricular transcripts

·         Documentation of service learning experiences and essential skills as measured by Work Keys

·         Alternative assessments such as portfolio, special projects, student self-assessment, education plan, and faculty critiques