Program Review: Developing a Faculty
Driven Process
Spring 1996 The Educational Policies
Committee - 1995-1996 Regina Stanback-Stroud, Chair - Rancho
Santiago College
Linda Collins, Los Medanos College
Jill Harmon, Fresno City College
Jim Higgs, Modesto Junior College
Paul Setziol, DeAnza College
Jean Smith, San Diego Community College District
- ECC
Robert Smith, College of San Mateo
Robert Rockwell, CIO Representative, Mt. San Jacinto
College
PROGRAM
REVIEWIntroduction
As a result of its desire to see the quality and
educational effectiveness of community colleges
maintained, the Academic Senate for California
Community Colleges (Academic Senate) has had a
long-standing commitment to faculty-based program
review mechanisms. As public support for funding
colleges and universities diminishes and fiscal
resources become increasing constrained, planning
and effective use of the sparse educational dollars
is paramount. Additionally, standards of accreditation,
general and categorical accountability, and community
educational needs further motivate colleges to
strive for sound educational practices that support
the integrity of the college and its programs
and services.
Over
the past six years, The Academic Senate, through
resolutions, has called for increased attention
to program review. The 1988 Research Committee
of the Academic Senate developed a preliminary
paper outlining the various models of program
review and some of the related issues and concerns.
The paper was forwarded as a resource to the Educational
Policies Committee that has attempted to respond
to annual resolutions calling for the development
of a model program review process.
A
few years and several drafts later this paper
seeks to respond to the resolutions. The attempt
to develop a "model" was abandoned as
consideration was given to the varying characteristics
of 106 local community colleges and 71 local community
college districts. To that end, this paper focuses
on various issues regarding program review. The
method emphasized is the self-study validation
method that is similar to the method used in the
accrediting process. This paper, complete with
recommendations throughout the body, is intended
to serve as a reference for community college
faculty and staff as they develop or revise their
program review process.
Ideas
and recommendations contained in this document
represent a culmination of efforts to address
program review. These efforts include, but are
not limited to:
A.
Seven breakouts at the Academic Senate Fall and
Spring Plenary Sessions that included the participation
of over 200 local academic senate presidents.
B.
Three different draft documents, initiated and
reviewed by three separate Educational Policies
Committees.
C.
Six presentations by Academic Senate representatives
both separately and in conjunction with the Researchers
professional association (RP Group).
D.
Fourteen sample program review processes from
large and small, urban and rural, diverse and
homogeneous colleges.
E.
Discussions with our CSU and UC colleagues on
program review and implications for faculty.
F.
A review of the limited amount of literature available
on program review specific to community colleges.
G.
A discussion with the Accrediting Commission for
Community and Junior Colleges regarding the impact
of external accountability processes on accrediting
processes and related legislation.
Background
Community
college faculty and administrative staff have
evidenced varying degrees of interest in program
review. As fiscal constraints placed upon colleges
increased and as program review, or some similar
process became a primary mechanism to determine
program expansion, continuation, or termination,
the schism of perspectives regarding the purpose
and application of program review developed. Many
departments across the state facing the prospect
of program review manifested common features of
"self-preservation" and "self-interest.'
The perception that the program review process
was a survival of the fittest contest, both divisive
and punitive in nature, eclipsed some of the more
noble but less prevailing perspectives that regarded
program review as an opportunity to promote educational
excellence and improve instruction and services
to students.
Previous
Resolutions
A
glance at the adopted resolutions reveals that
as early as 1987 and as recently as 1995, faculty
have increasingly expressed an interest in program
review. Concerns included the linkage of program
elimination to program review, the relative collective
bargaining implications in program review and
elimination, the necessary criteria for program
review, the need for information/workshops on
program review, and the extent to which program
review should include student, administrative,
and ancillary services, as well as instructional
programs, and the role of local academic senates
in program review. (See Appendix 1)
Legislation
Related to Program Review
Work
Force Preparation
As
the workforce preparation initiatives develop
on the state and national level, several processes,
models, and pieces of legislation addressing accountability
have emerged. Senate Bill 645 (Johnston), the
Report Card Bill, was recently signed by the Governor.
This bill calls for the State Job Training Coordinating
Council (SJTCC) , formerly the advisory body to
the federal Job Training Partnership Act (JTPA)
program, to develop performance-based indicators
upon which funding decisions can be made. Of particular
interest to California Community Colleges are
the parts of the bill that indicate that certificated
programs may be included.
While the inclusion of certificated programs raises
some very fundamental questions about the role
of SJTCC in making educational policy and evaluating
the community colleges, of greater significance
is the fact that this legislation seeks to supersede
previously legislated and regulated accountability
processes, as well as traditionally respected
educational processes such as accreditation. This
legislation minimally should put faculty and college
administrators on notice of the Legislature's
and the public's interest in accountability.
Regulations
Related to Program Review
In responding to
the legislative mandate to develop processes to
strengthen academic senates, the Board of Governors
for California Community Colleges adopted Title
5 Article 2. Academic Senates. These regulations
identify the definitions, formations, responsibilities
and powers of the academic senates. These regulations
require governing boards to "consult collegially"
with academic senates on "academic and professional
matters."(1)
Processes for program review are situated among
the eleven items listed in "academic and
professional matters."(2)
According
to the regulations, "Consult collegially"means
that the district governing board shall develop
policies on academic and professional matters
through either or both of the following methods,
according to its own discretion:
(1)
Relying primarily upon the advice and judgement
of the academic senate; or
(2) That the district
governing board, or such representatives as it
may designate, and the representatives of the
academic senate shall have the obligation to reach
mutual agreement by written resolution, regulation,
or policy of the governing board effectuating
such recommendations.(3)
This
regulation, which has the effect of law, means
that academic senates must play a central role
in the development of the program review processes.
It is the professional responsibility of the local
academic senate to either provide the primary
advice or develop a senate position in order to
have a basis upon which representatives can seek
to come to mutual agreement with the board.
If the college
does not have a process adopted by the board that
was developed with the collegial consultation
of the local academic senate, the local senate
should immediately seek to initiate such a process.
The regulations required the local boards to adopt
policies for the appropriate delegation of authority
and responsibility to its college and/or district
academic senate.(4)
To that end, local senates should identify
which process of collegial consultation the board
has chosen for program review and begin the endeavor.
If the local board does not have a delineation
of authority policy, local senates should (1)
bring the issue to their attention and seek to
support the board in the development of the delineation
of authority; (2) begin working on gathering information,
seeking input from affected constituencies, and
develop a senate position through senate processes
on the issue of program review. The discussion
contained in this section is not intended to suggest
that faculty should wait for the boards to develop
the delineation of authority policy before addressing
program review.
Role
of Local Academic Senates
The
involvement of the local academic senate, as the
representative of the faculty, is critical to
the successful development and implementation
of a program review process. While the role of
the local academic senate is delineated in regulation,
academic senates have a greater opportunity to
create a collegial, supportive, student-centered,
faculty-driven, academically relevant process
for the evaluation of the college's programs and
services. Local academic senates should see their
role as an opportunity to redefine program review
in order to eliminate unwanted or ineffective
characteristics historically associated with the
process. Administrations and boards need the professional
expertise and judgment inherent in the collective
wisdom of the faculty represented by the local
academic senate. Through an organized resolution
process or the development of a position paper,
the local academic senate can affect one of the
most important processes for determining how well
the students' educational and support needs are
being met on a college campus.
A
Q&A document developed by Past President Jim
Locke, and Vice President Bill Scroggins suggests
standard language. With some modification, this
language appears below. Based on local issues
and discussions, revisions can be made. This resolution
was developed with the intention of providing
local academic senates a basis upon which they
can build their program review discussions into
a process.
"Whereas,
faculty members derive their authority from their
expertise as teachers and subject matter specialist
and from their status as professionals and as
a result, the faculty has an inherent professional
responsibility in the development and implementation
of policies and procedures governing the <insert
college name> program review processes; therefore
Be
it resolved that the <insert college name>
program review process shall promote professionalism,
enhance performance, and be effective in yielding
a genuinely useful and substantive process for
determining program effectiveness, and
Be
it further resolved that the program review processes
at the <insert college name> will
require
1) an articulation of clear, relevant criteria
upon which reviews will be based,- 2) the establishment
of reasonable and timely intervals; 3) the establishment
of the specific purposes for which program reviews
are conducted and articulation of those purposes
to everyone involved, and
Be
it further resolved that the principal purposes
of the review process are to recognize and acknowledge
good performance, to enhance satisfactory performance
and help programs which are performing satisfactorily
further their own growth, and to identify weak
performance and assist programs in achieving needed
improvement, and
Be
it further resolved that one of the purposes of
the program review process at <insert college
name> is not that of providing a mechanism
or justification for program elimination; and
Be
it further resolved that a program 's students,
administrators, faculty and their colleagues shall
all contribute to the program review, but the
program's faculty shall play a central role in
the program review process and, together with
appropriate administrators, assume principal responsibility
for the effectiveness of the process; and Be it
further resolved that procedures of the program
review process shall foster a joint and cooperative
exercise of responsibility by the faculty, administration,
and governing board of <insert college name>
and shall reflect faculty and administrator expertise
and authority in evaluating professional work
as well as the governing board's legal and public
responsibility for the process, and
Be
it finally resolved that the < insert college
name > program review process shall provide
an ongoing and thorough review of the relevance
and responsiveness of vocational education programs,
consideration of the relationship between other
similar programs throughout the state, and the
appropriate balance between vocational and general
educational programs.(5)
Local
academic senates will need to identify the process
they will use, the participants time lines, and
the needed resources/cost for the development
of a program review process. It is recommended
that the senate form a subcommittee that is responsible
for the development of the program review process.
Resource people, including students and staff,
are valuable components of the committee. The
senate should consider that either primary advice
or mutual agreement does not exempt the senate
from seeking the input from others affected by
the process. Because a successful program review
will depend upon an institutional effort by faculty,
staff, and students, and because staff and students
have a regulatory right to participate in the
discussions of district policies that have a significant
effect on them, senates that fail to see the value
in the participation of other groups may face
extreme difficulty in either coming to agreement
or giving primary advice on an unjustifiable and
illegal position that excludes the participation
of all affected groups in the development of the
processes. Sound judgement by the senate would
exercise a process that is open and collegial
in the development of program review processes.
To that end, as the model is being developed,
staff and students should be asked to contribute
to the discussion in order that the senate may
take the ideas presented under consideration as
the process is developed.
The
subcommittee should prepare for the local senate
a recommended model (in the case of primary advice)
or position (in the case of mutual-agreement)
on program review. In order to ensure all interested
parties an opportunity to comment or contribute
to the discussion, the local senate should consider
holding a hearing on the proposed process that
will be considered for reading and action. Subsequently,
the senate should subject the recommended process
as appropriately modified by input from the hearings,
and resource people, to the scrutiny of the senate
resolution processes. The item should be handled
under the strict scrutiny of the Brown Act with
the proper notification, first reading, and second
reading/action agenda items. The local senate
should operate a fully public meeting, open to
the public with provisions for public comment
as designated by the local senate. If heavy attendance
and participation is anticipated, local senates
may want to adopt procedures for public comment
similar to those used by local governing boards,
thereby ensuring the opportunity for all to speak
with a time limit. The adoption of such a procedure
provides a record for the local senate that serves
to indicate the opportunity of the campus community
and the public to comment. Ultimately, the local
academic senate should recognize its responsibility
and right to recommend a process that represents
the informed and collective wisdom of the faculty.
Purposes
of Program Review
The
development of the local program review process
will be influenced by the purposes, assumptions,
and philosophy under which the program review
process is being developed. A review of the literature
reveals a common assertion that in order for the
program review process to be successful it should
serve as a mechanism for the assessment of performance
that recognizes and acknowledges good performance
and academic excellence, improves the quality
of instruction and services, updates programs
and services, and fosters self-renewal and self-study.
Further, it should provide for the identification
of weak performance and assist programs in achieving
needed improvement. Finally, program review should
be seen as a component of campus planning that
will not only lead to better utilization of existing
resources, but also lead to increased quality
of instruction and service. A major function of
program review should be to monitor and pursue
the congruence between the goals and priorities
of the college and the actual practices in the
program or service.
Other
purposes include meeting accountability mandates
and demonstrating a systematic planning process.
While these functions may occur as a result of
program review, they should not serve as a primary
function of the review. Otherwise, program review
could be perceived as a time-consuming process
for a product with no effect or bearing on the
program's activities or any relevance to serving
students or maintaining academic excellence.
Finally,
some program review processes have also served
the purpose of justifying program and/or personnel
elimination. The process by which programs and
services are reduced or eliminated should be clearly
defined, as well as be separate and distinct from
program review. Additionally, individual faculty
and staff evaluations should remain a separate
and distinct entity from the program review process,
following peer review procedures developed by
the collective bargaining agents, where they exist,
in consultation with the academic senate.
Linkages
Many
processes and activities of a college could and
should be correlated or coordinated with the program
review process. The probability of decreasing
the duplication of effort and effectively managing
time, human, research, and fiscal resources is
maximized with such coordination.
Educational
Planning
Community
college educational planning requires that the
college examine what it does and how what it does
compares to a forecast of what is needed in the
future. Program review is thus one of the two
essential components of planning. Short-term adjustments
in resource allocations frequently are made using
immediate enrollment data; we see enrollments
decreasing in course A and wait lists growing
in course B so we offer fewer of A and more of
B. But planning asks why these changes are occurring,
whether the condition can be changed by the college
and the faculty, and what is to be done.
Program
review contributes to the search for the answers
for these questions. It does so by evaluating
program quality and by weighing enrollment and
outcome information against the context in which
the college exists (the role of the program in
transfer patterns, or in preparation for the job
market.) In the context of educational planning,
scarce resources and growing student populations
compel us to focus on difficult questions; which
program or service deserves additional staff,
equipment, or supply budgets? Which programs or
services are organized in such a way that they
must be reconstructed to more effectively meet
student needs? Which programs and services, as
determined through an appropriate process, are
not needed and by their presence, are preventing
the development of needed programs and services?
While
there is a tendency to be reticent to pose some
of these questions, faculty should and must become
involved in the posing and asking such questions.
The colleges can HI afford to be lacking in the
knowledge and expertise of the faculty on these
important planning issues.
Accreditation
Process
"Accreditation
provides assurance of the institutional integrity,
quality, and effectiveness .... More importantly,
accreditation is the system by which the internal
community of an institution evaluates itself and
plans for improvement in quality and effectiveness."(6)
The purposes of program review and accreditation
may be similar, notwithstanding the general nature
of accreditation and the specific nature of program
review. However, a constitutional difference between
accreditation and program review is the locus
of control over the standards and the validation
process. In the accrediting processes, the Commission
defines the standards; in the colleges program
review process, the college defines the standards.
The faculty role
and involvement in accreditation processes, including
self-study and annual reports, is one of the areas
on which local academic senates must be collegially
consulted.(7)
As local academic senates plan program review
processes, many of the general principles and
processes can be transferred from the accreditation
model and modified to more specifically address
the program under review. If time lines are coordinated
with institutional accrediting processes, programs
can take advantage of the overall climate in the
institution reflectively focused on self-study
and-improvement. This positive climate alone could
and would be a sharp contrast with the negative
climate many faculty have historically associated
with program review.
Finally,
as the institution develops goals in response
to the accreditation process, program improvement
or support could be incorporated in or coordinated
with the developed goals. The institutional goals
will be placed in an overall and legitimate context
that faculty are likely to support. This type
of comprehensive acknowledgment of the goals lays
a strong foundation for the second cycle of program
review whereby the goals are seen as legitimate
and concerns regarding the legitimacy of the planning
process become irrelevant.
Budgetary
Processes
A
multitude of budget and governance discussions
have yielded a common axiom, "Planning should
drive the budget; the budget shouldn't drive the
planning." This statement in the context
of many California Community Colleges is usually
made as college leaders announce the fiscal constraints
and the need to eliminate courses, programs, and/or
services. Despite the mission of the college,
the master planning (usually done at five-year
intervals), the goals of the college, and the
objectives of the departments, the decrease in
funds, by default, call for reprioritizing and
modification of the products of any systematic
planning.
The
"planning-by-budget" phenomenon may
be exacerbated by fiscal constraints imposed by
the state, combined with the reality of college
political processes in which contracts are negotiated,
projects are pursued, and efforts to maintain
a positive image in the community are made. At
times, a relatively sophisticated institution,
with complex planning processes, stable funding,
and the best of intentions, may be faced with
having to make cuts due to factors beyond the
control of the college. In this context, the establishment
of legitimate goals and the coordination of program
review with the budgetary process is essential.
As
with program review, the local academic senate
has responsibility for collegial consultation
on the budget process. In fight of the similar
roles of the local academic senate on program
review and the development of the budget process,
a great opportunity exists to coordinate the two
processes; recommendations developed through program
review can be considered when resources are prioritized.
This coordination alone stands to ensure daily
activities actually are affected by the recommendations
developed in the program review process.
Curriculum
Process
The curriculum
process is yet another area of primary responsibility
for the academic senate.(8)Curriculum
processes are known to be time consuming. A central
argument in support of such a commitment is that
it allows for the deliberation and scrutiny necessary
to ensure the academic integrity of the curriculum.
This integrity is respected within and outside
of the college. Business and industry, transferring
institutions, community members, students, and
the college itself, all benefit from the academic
integrity that results from educationally sound,
deliberate, curriculum processes. Whether the
issue is the determination of an overall curriculum
model or the approval of a particular course,
the local academic senate plays a primary role.
If
a product of the program review process I is
a need to enhance, modify, or radically
change the curriculum, the coordination of the
program review process with the curriculum process
facilitates a more successful approach to meeting
student and program needs. Careful attention should
be given to programs that may have to respond
to outside agency standards, curriculum frameworks,
or legislative acts. Such programs may include
some of the certificate-granting and/or licensing
vocational education programs.
Student
Equity
The commonly used
quantitative program review measures are consistent
with the indicators recommended in the student
equity regulations.(9)
Program review provides the opportunity for faculty
to examine whether all students and the entire
community are being successfully served; the goal
of more equitably serving students can thus be
furthered. For example, a math department may
identify that it has a 75% student success rate.
If further analysis reveals that of the 25% who
were unsuccessful, a disproportionate number were
women, perhaps the math faculty would want to
examine whether their pedagogy and curriculum
are linked to these inequitable outcomes.
Local academic
senates are also to be collegially consulted on
standards and policies regarding student preparation
and success.(10)
Because student equity is an issue of student
success, program review processes should also
be coordinated with student equity processes.
Regional
Academic Planning
As colleges and
universities experienced the elimination of courses
and programs, the Intersegmental Committee of
the Academic Senates (ICAS), that five representatives,
often the ranking officers of the state senates
of the University of California, the California
State University, and the California Community
Colleges, began discussions on regional academic
planning.(11)
The concern centered around the elimination of
programs without the coordination necessary to
ascertain the impact on the geographical regions
of the state. Consequently, the probability of
eliminating advanced foreign language courses
or baccalaureate level nursing programs in the
entire southern or northern region of the state
could be minimized. Instead, course and program
elimination could be made in an informed manner
that would include the regional impact.
The
1993 Budget Act states in part:
"Long-term
Planning for Program Course Offerings in Higher
Education. It is the intent of the Legislature
that the UC, California State University (CSU),
and, to the extent possible, the California Community
Colleges (CCC) consult with each other on a regional
basis as plans for campus budget reductions are
developed, in order to ensure that particular
geographic regions continue to offer an adequate
balance of academic program offerings and courses...."(12)
While
this concern is primarily aimed at academic program
offerings and courses, this language should also
provoke community colleges to identify the inherent
imperative of the language as it relates to the
mission for California Community Colleges. The
Master Plan's Mission includes the offering of
general education/transfer, vocational education,
and basic skills. To that end, program/course
elimination, irrespective of regional considerations,
and out of the context of UC and CSU regional
planning, could have educationally and economically
devastating results for the individual seeking
an education for transfer or employment, for the
business and industry in the community, and for
the community itself.
Although
the recommendation herein assumes separation of
course elimination processes from program review,
the impact of program goals developed without
the regional planning should be considered. The
perceived standards of success could be strengthened,
legitimate goals of the college could be better
pursued, and college and program relationships
with the community could be supported by the coordination
of regional academic planning and the program
review process.
Considerations
When Developing A Local Model
Essential
to the development and implementation of a program
review process are the underlying assumptions
upon which it is based. These assumptions should
be identified and delineated in the program review
process. Some of the more common issues and assumptions
are discussed in this section in an attempt to
provoke greater deliberation at the local
level.
Goals
Essentially every
piece of literature and program review process
referenced identified the need to link program
review with the district mission, the college
master plan, and the department goals and objectives.
Inherent in this assumption is that the mission,
master plan, goals and objectives are legitimate
to the faculty. This legitimacy would depend upon
the process by which the district, college, and
departments arrived at such missions, plans, goals
and objectives. As with program review and other
academic and professional matters, institutional
planning is one of the eleven items on which local
academic senates must be collegially consulted.(13)
If the program review process was one in which
faculty were consulted collegially, thereby ensuring
a respect for and ownership of the goals by the
faculty, the assumption could serve the process
well. If, however, the faculty were not consulted
collegially, this linkage to program review should
not be used until legitimate missions, master
plans, goals and objectives are developed.
Application
of Program Review Process
The
question of which programs and services should
be reviewed can be addressed in the college's
program review assumptions. If the purpose of
the process is to promote educational excellence
and better serve the students, no function of
the college should be exempt. The effectiveness
of the ancillary units and administration of the
academic divisions, instruction, student services,
the executive offices, and the governing board
all have an impact on the academic integrity of
the programs and on the college's ability to serve
the students. As local academic senates address
this issue, it may be necessary to define the
term "program" or answer the question
of whether the process is an instructional/services
program review process or an institutional program
review process. An institutional review process
would include the review of all programs and services,
including units not commonly perceived as programs.
This perspective is also consistent with defining
programs as they are defined in the college budget
processes (cost centers) and in the planning and
goal setting process (planning units).
Institutional
Support
A
fundamental assumption necessary for the successful
development and implementation of a program review
process is the commitment of the institution to
provide institutional resources. Irrespective
of the model used, faculty leadership in self-study
and review will require institutional support
for time, materials, and staff. Historical models
that designated administrative deans to assume
a leadership role may not have had to address
the issue raised in this assumption because most
administrative deans have budgets, materials,
and clerical staff support. If the institution
is willing to support program review while it
is under the leadership of an administrative dean,
its inability to do so for processes under faculty
leadership is unjustifiable. A delineated budget
should be available for program review. Faculty
should resist any attempt to sacrifice the integrity
of the program review process in order to economize.
Another type of
institutional support necessary for the successful
implementation of a program review process is
that of research support. Depending upon the program
review process developed, both qualitative and
quantitative research may be necessary. Faculty
scheduled for program review should influence
the research agenda. For small colleges without
research facilities, faculty should have access
to whatever staff person(s) serve the research
function of the college. In commenting on program
review, George Boggs, President of Palomar Community
College and then Chair of the Commission on Research
for the California Association of Community Colleges
stated, "One of the
most basic and important types of institutional
research for a community college to do is program
evaluation. In its most basic form, an evaluation
is a study, based upon objectives of the activity
or program to be assessed, that provides useful
information about the degree to which those objectives
have been met. Program evaluation is simply a
matter of asking useful questions, and then making
valid conclusions based upon the data. 7he current
emphasis on documenting student outcomes is, intact,
a call to conduct more and better program evaluation."(14)
Fairness
and Credibility
In
developing local program review processes, local
academic senates will have the responsibility
to develop a process that is perceived to be fair
and credible. The credibility of the evaluators
and the review should be established. Failure
to do so could decrease the propensity for faculty
to buy into the process, invest their time, utilize
it to its fullest potential, or pursue endeavors
arising from the product of the process.
There
must be trust that the purposes and goals of the
process are truly dedicated to supporting and
nurturing departments instead of destroying them,
reducing the faculty, or pursuing a cheaper approach
to delivering instruction or services. Indications
of this need might be manifested in such questions
as:
1.
Is confidentiality an issue?
2.
Who will review the data and reports? For what
purpose will they be used?
3.
Are there any provisions for rebuttal or minority
reports?
The
local academic senates will need to ensure that
these types of concerns are adequately addressed
in the development and the implementation of the
program review process.
Finally,
the administration of the college must believe
in and respect the process of program review.
Administration must be willing to help enact the
changes that are recommended as a result of the
process. If fiscal or other restraints preclude
some changes, they should be presented at the
onset and perceived to be true by the faculty,
thereby further defining for future reference
the range of options available to the faculty.
Nonetheless, in performing a program review, faculty
should not hesitate to identify the ideals that
would improve or support the programs or services;
however, the identification of constraints can
assist faculty in preparing realistic recommendations
that should affect their day to day operations.
Identify
Criteria and Determine Research Needs
A
universal set of criteria for program review is
not practical given the diverse characteristics
of our local colleges. Our colleges are "community"
colleges, specifically designed to meet the educational
needs of the community. As communities drastically
differ across the state, so do their community
colleges. As local academic senates develop the
program review processes, the research needs and
methodologies must be considered. Because most
program review process have both qualitative and
quantitative components, these methodologies should
not be considered mutually exclusive of each other.
Qualitative
Program
review presents an opportunity for discussion
of educational philosophy and the contributions
of each program to the development of the whole
student. General education is usually dedicated
to some notion that in addition to technical skills
and competencies, education should be concerned
with contributing to the development of generally
educated persons. Difficult to measure abilities
such as the capacity for ethical reasoning and
critical thinking skills are central to preparing
students for life-long learning and effective
citizenship. Such qualitative outcomes are not
easy to quantify but are reflected in the spirit
that animates classroom discussion, the values
that are modeled in pedagogy, and the habits of
mind of graduates. It is not uncommon for program
review processes to leave out the examination
of such qualitative dimensions.
Explicit
attention should be given to qualitative aspects
of programs. If the focus of the process is qualitative,
less statistical and more value-laden assessments
will be made. Faculty will assess areas that do
not lend themselves to objective evaluation, but
nonetheless, are legitimate and important areas
for consideration. Examples include assessments
of the learning environment, including facilities
available, financial resources, instructional
equipment, and the impact of class size on student
success and the ability of the program to meet
the students' needs. The quality of the breadth
of the curriculum and the teaching and learning
process should be taken into consideration. The
student satisfaction and the quality of the program
as perceived by them and the quality of the program
as perceived by the articulating universities,
or employing businesses and industry. is a crucial
factor in evaluating the program for effectiveness
in serving the students and community. Local academic
senates should further develop the possible qualitative
factors to be assessed, as this area represents
a small portion of the possibilities.
Quantitative
Recommendations
of explicit attention to qualitative factors is
not meant to imply that faculty should reject
the utility of quantitative data in the program
review process. The quantitative factors, too,
have value in the context of a comprehensive program
review process. Items such as access, enrollments,
persistence, transfers, graduation, successful
course completion, grades, and other student outcome
factors draw a numerical picture for further analysis.
However, preoccupation with quantitative measures,
particularly productivity, will have the effect
of directing the program review processes rather
than concerns over educational soundness or student
success. Such preoccupation could lead to the
failure to consider important factors such as
(1) the comprehensiveness of the college model
and how the curriculum model supports it, (2)the
educational philosophy or commitment and how the
various program contributions translate into that
philosophy and commitment, (3) the educational
needs of the community and how the programs support
those needs.
It
is important to emphasize that quantitative data
should not be presented without a narrative explanation
nor should it be used for comparison among a college's
programs. Such use would tend to promote the use
of program review for program reduction or program
elimination and would completely disregard the
qualitative value of a program. Rather, trends
in data over a period of time within the program
itself may be most useful for the program faculty
to identify their needs and design the necessary
intervention or support.
Self-Study
Validation
Consistent
with the assumptions and/or purposes, self-study
is an opportunity for self-renewal. The security
of understanding the process will result in support
for the program whether there are strengths or
weaknesses. Self-study allows for the people with
the greatest level of expertise in a particular
program, the faculty, to examine and scrutinize
the program for effectiveness in serving students
and educational excellence.
The
program review process should include the specific
constitution and function of the self-study process,
including but not limited to:
A.
Will it be a committee? If so, what if the program
only has one or two faculty? What will be the
duties of the team members?
B.
Will it involve full-time faculty, part-time faculty?
What if a program has no full-time faculty?
C.
The senate should identify its process for naming
the faculty members. Will it be by appointment,
or ratification of the departments decision?
D.
How will other units of the college be included?
How is the decision to be made?
E.
What is the local administrator's role? Is it
as part of the study team or the validation team?
F.
Who is responsible for writing the report? What
should be included?
G.
Once the data is interpreted and the report is
drafted, how will the strategies for specific
actions be developed and incorporated in the report?
What should be the implementation time lines?
H.
What is the time for commencement and completion
of the process?
I.
How will the data be gathered? What will be the
methodology?
J.
What are the components of the validation process,
review of documents, program visits, review of
data/survey results?
K.
What happens after the validation stage? What
is the flow of the paper? What happens with the
results?
L.
Are there any further meetings, discussions, conferences,
before the final report is issued? Who will present
it to the governing board and the academic senate?
How will the report be used?
M.
How is the necessary support given to the program,
once the review is complete, the results are finalized,
and the recommendations are made?
N.
What is the comprehensive time line for each program,
service, or unit to be reviewed?
While
this is not an exhaustive list, local academic
senates can see the attention to detail called
for in developing the process.
Models
In 1988, the Ad-Hoc
Research Subcommittee of the Educational Policies
Committee of the Academic Senate developed a paper
identifying the various models available in the
literature concerning data gathering, program
review models, and accountability in education.(15)
The models included:
The Research-Numerical
Methods Model - This model has a quantitative
orientation that concentrates on the collection
of hard data.(16)
This method or aspects of this method may provide
practitioners with information regarding why a
specific trend has appeared or whether an effect
has a cause.(17)
The Peer Evaluation-Professional
Judgement Model - This model is similar to a "mini-accreditation"
process and involves personal observations, interviews,
and the review of documentation.(18)The
delineation of a program's or service's relative
worth is derived from the informed opinion of
the visiting expert.(19)
The Improvement
- Progress Toward Goals Models- Citing Don Gardner's
September 1977 article in the Journal
of higher Education, "Evaluation models
based on this definition assume that the most
important decisions regarding the thing to be
evaluated are contingent on its objectives and
the criteria established for judging ... [and]
the relative attainment of those goals..."(20)
The Issue Oriented
Model - This approach is a positive and humanistic
qualitatively centered process that treats problems,
solutions, successes, and consequences as having
the same status and importance.(21)
The
issues emphasized in this document could easily
be applied to a program review process that includes
some aspects of each of the four models.
Conclusion
Program
review is an essential component of serving the
community's educational, training, and basic skills
needs. There are many models, as well as factors
to be considered in the development and implementation
of program review. A deliberative well planned
process that is faculty driven and respected throughout
the college can and will result in meaningful
evaluation from which vital information can be
derived for the maintenance of the integrity of
the college community and its educational programs.
Appendix
- Related Resolutions
S94
8.5 Be it resolved that the Academic Senate for
California Community Colleges direct the Executive
Committee to collect data identifying programs
eliminated by individual colleges and the reasons
for elimination, and
Be
it further resolved that the Academic Senate for
California Community Colleges direct the Educational
Policies Committee to research program discontinuance
policies and procedures, and
Be
it finally resolved that the Academic Senate for
California Community Colleges direct the Educational
Policies Committee to develop a position paper
with recommendations concerning implementing local
program discontinuance policies.
S93
5.1 7 Be it resolved that the Academic Senate
for California Community Colleges recommend to
local senates that the collective bargaining units
be consulted in program evaluation procedures
and in program reduction decisions
S93
6.2 Be it resolved that the Academic Senate for
California Community Colleges develop a model
for program review including criteria which balance
quality and economic feasibility.
S92
61 Be it resolved that the Academic Senate for
California Community Colleges direct the Educational
Policies Committee to develop a model program
review policy for consideration at a future session,
and
Be
it further resolved that the following issues,
among others, be considered for inclusion within
that model:
Class
size and its effect on instructional quality;
b.
The appropriate mix among transfer, vocational,
basic skills, and noncredit courses;
c.
Considerations of the quality of instruction as
well as issues of productivity;
d.
Facilities considerations;
e.
Connecting the outcomes of program review with
the college and district budgets; and
f.
Contribution to the community based on student
success in job placement.
F88
5.3 Be it resolved that the Academic Senate for
California Community Colleges recommend to local
senates that libraries be included in all college
wide program reviews.
S88
4.5 Be it resolved that the Academic Senate for
California Community Colleges requests that the
Chancellor's Office organize workshops on the
program review process.
F8714.1
Be it resolved that the Academic Senate for California
Community Colleges requests that the Chancellor's
Office notify the LOCAL academic senate in a timely
fashion regarding the impending visitation of
m@tedprogrwn review teams for programs such as
Equal Opportunity Programs and Services and Disabled
Students' Programs and Services, and be it further
resolved that the Academic Senate for CCC recommend
that during these visitations the local academic
senate be actively involved in the interview process
for the purpose of evaluation of these programs.
1.
Barclay California code of Regulations, Article
2 Sections 53203(a). Register 93. No. 6;2-5-93.
Page 325
2.
Barclay California code of Regulations, Article
2 Sections 53200(c). Register 93. No. 6;2-5-93.
Page 325
3.
Barclay California code of Regulations, Article
2 Sections 53200(d). Register 93. No. 6;2-5-93.
Page 325
4.
Barclay California code of Regulations, Article
2 Sections 53203(a). Register 93. No. 6;2-5-93.
Page 325
5.
Scroggins, Bill and Locke, Jim "Questions
Faculty Ask, Answers and Suggestions on the Faculty
Role in College Program Review Policy and Procedures."
1994 page 1
6.
Accrediting Commission for Community and Junior
Colleges, Guide to Institutional Self Study and
Reports to the Commission. 1990, page 1
7.
Barclay California code of Regulations, Article
2 Sections 53200(c)(7). Register 93. No. 6;2-5-93.
Page 325
8.
Barclay California code of Regulations, Article
2 Sections 53200(c)(1). Register 93. No. 6;2-5-93.
Page 325
9.
California Code of Regulation, Subchapter 4, Section
54200, Student Equity Plans.
10.
Barclay California code of Regulations, Article
2 Sections 53200(c)(5). Register 93. No. 6;2-5-93.
Page 325
11.
ICAS committee minutes, December 13, 1993
12.
Budget Act, Item 6420-001-001, "Supplemental
Report to the Committee on Conference on the 1993-1994
Budget Act
13.
Barclay California code of Regulations, Article
2 Sections 53200(c)(10). Register 93. No. 6;2-5-93.
Page 325
14.
Boggs, George, Leadership Abstracts, "The
Research Function of Community Colleges,"
League for Innovation in the Community College,
Laguna Hills, CA Vol. 1 No. 13 August 1988, page
1
15.
Ad Hoc Research Subcommittee of the Educational
Policies Committee of The Academic Senate for
California Community Colleges, "Toward
an Acceptable Program and Services Review,"
1988, page 2
16.
"Toward an Acceptable
Program and Services Review," 1988,
page 2
17.
"Toward an Acceptable
Program and Services Review," 1988,
page 2
18.
"Toward an Acceptable
Program and Services Review," 1988,
page 3
19.
"Toward an Acceptable
Program and Services Review," 1988,
page 2
20.
"Toward an Acceptable
Program and Services Review," 1988,
page 4
21.
"Toward an Acceptable
Program and Services Review," 1988,
page 5
