Academic Senate Ad Hoc Committee on Student Perceptions of Teaching Effectiveness Questionnaires and Student Opinion Surveys

 

Background:

In 2013, the Academic Senate passed AS-759-13, Resolution on Student Evaluations following the 2012-14 revision to the Collective Bargaining Agreement that stated “written or electronic student questionnaire evaluations shall be required for all faculty unit employees who teach” (CBA, 15.15¹-15.18) and the distribution of Administrative Memo 21030222, New Student Evaluation Requirements Elective Winter Quarter 2013. The University Faculty Personnel Policies document chapter 8.4 was thus updated and established instrument requirements (8.4.2), Criteria for Conducting Student Evaluation (8.4.3), Procedure for Conducting Student Evaluation (8.4.4), and Student Evaluation Results (8.4.5).

 

Use of online student evaluations commenced in Spring 2015 and has continued since. Student response rates have ranged from a high of 66% in Fall 2016 to a low of 35% in Spring 2020 and have remained in the 40% range since Fall 2021 across the University. Such low response rates prevent faculty from drawing conclusions about their students’ perceptions of teaching and course effectiveness due to non-response bias. The value of responses to the two required questions on the student evaluation instruments has come into question due to their ambiguity (“overall, this instructor was educationally effective”; “overall, this course was educationally effective). This ambiguity calls into question the reliability and validity of the responses.

Further, research on student evaluations of teaching has found evidence of measurement and equity bias, with faculty of color, women faculty, and other faculty from marginalized groups experiencing bias at higher rates. Research has also shown that student evaluations of teaching are subject to measurement bias, in that the results are more likely to have been shaped by discipline, student interest in the course, class level and difficulty, class meeting time, location, and setting, but not (generally speaking) actual instructor or course effectiveness.

Finally, how the data collected for student evaluation of teaching is analyzed and reported is inconsistent across the university and in some cases, analyses are inappropriate for the type of data collected (5-point Likert-type scales of ordinal data). This makes it difficult if not impossible to maintain consistency of interpretation of quantitative results from student evaluations within and across colleges.

-------

¹According to chapter 15.15 of the CBA, these instruments are also referred to as “student opinion surveys” or “student perceptions of teaching effectiveness.”

 

Charges:

• Reviewing the reliability and validity of the prompts required on all student evaluation instruments, suggesting revisions to the prompts if necessary, and determining if additional prompts are necessary to obtain a more reliable and valid assessment of teaching effectiveness at Cal Poly.
• Revisiting the criteria for procedures for conducting student evaluations to increase response rates and reduce incidences of bias, particularly negative bias toward people of color, women, and other minoritized populations in student feedback.
• Reviewing how both quantitative and qualitative data collected from student evaluations are provided to faculty, the analysis of the data, and how data are presented for review for retention, promotion, and tenure. The committee should also consider as part of their charges what data is appropriate for development of teaching effectiveness purposes and data appropriate for performance evaluation.
• Suggesting processes for disseminating results of student evaluations to Cal Poly students.  

 

The Ad Hoc Committee will provide a progress report to the executive committee at the end of Spring Quarter 2025 and provide a revised policy and resolution to replace AS-759-13 at the end of Fall Quarter 2025. 

 

A printable version of this committee's charges can be found here.

---

Meetings:

TBD

Committee Membership:

• One faculty member from each college. Colleges with governance committees are encouraged to nominate faculty from those committees. Faculty who have experience with evaluative instrument design, ordinal data analysis, and qualitative data analysis are encouraged to participate.
• One lecturer with a 12.12 appointment
• One member from the Faculty Affairs Committee
• One member from the Academic Senate Instruction Committee
• One representative from Academic Personnel (ex officio/non-voting)
• The Faculty Director of Program Improvement (ex officio/non-voting)
• An academic department head or chair
• One Associate Dean (ex officio/non-voting)
• One ASI representative
• One representative from CFA designated by CFA SLO chapter leadership.

 

A chair of this ad hoc committee will be elected by the members of this body.  

 

Note: Please click here to submit your statement of interest.

Membership

College / Division	Name	Department/Area	Office Phone	Email
BCSM	Anelise Sabbag	Statistics	(805) 756-5562	asabbag@calpoly.edu
CAED	Vacant
CAFES	Keri Schwab	Experience Industry Management	(805) 756-5113	keschwab@calpoly.edu
CENG	John Pan (Committee Chair)	Industrial and Manufacturing Engineering	(805) 756-2540	pan@calpoly.edu
CLA	Laura Cacciamani	Psychology & Child Development	(805) 756-2703	lcacciam@calpoly.edu
OCOB	Eduardo Zambrano	Economics	(805) 756-5327	ezambran@calpoly.edu
Lecturer	Vacant
Academic Department Head or Chair	Carrie A. Langner	Psychology & Child Development	(805) 756-2320	clangner@calpoly.edu
ASI	Jett Palmer	Student Representative	-	jpalme20@calpoly.edu
CFA	Silvia Marijuan	World Languages and Cultures	(805) 756-1212	smarijua@calpoly.edu
Faculty Affairs Committee Representative	Kylie Parrotta	Social Sciences	(805) 756-1240	parrotta@calpoly.edu
Instruction Committee Representative	James Bingaman	Agricultural Education & Communication	(805) 756-0662	bingaman@calpoly.edu
Academic Personnel (ex officio, non-voting)	Simone Aloisio	Academic Personnel	(805) 756-5237	saloisio@calpoly.edu
Associate Dean (ex officio, non-voting)	TBD
Faculty Director of Program Improvement (ex officio, non-voting)	Jean Lee	Academic Programs and Planning	(805) 756-3026	jlee473@calpoly.edu