The Development Process for a novel “Peer-Assessment Lecturer Survey” (PALS)

The Development Process for a novel “Peer-Assessment Lecturer Survey” (PALS)

(available on request by word format)

The peer-evaluation of an instructor can be approached from different perspectives.  In reviewing methods of evaluation, I found the social-cognitivist theory elucidated by Bandura (1986) to be helpful. It links the behaviourist approach, which emphasises the influence of the environment on our actions, and the cognitive approach, which emphasises the importance of cognition in mediating our learning and functioning. (Kaufman 2010)

For lecture evaluation, the behaviourist approach focuses on the presentation process, learning environment and instructor content.  This is best demonstrated by microteaching of skills of Allen (1969) and Passi (1976) seeing the instructor from a mechanical perspective.  For example, the microskill of “stimulus variation” in which an assessor may critique “using gestures to help convey extra meaning” or “at various times, the teacher was noted in the left, right, forward, and back of the teaching space” (Figure 1) (Allen 1969).  These principles are old-fashioned and mechanistic but useful.

The cognitivist approach shifts the emphasis to learner-centred content and outcomes where the substantive parts of evaluation occur internally within the minds of the instructors and students.  Lecture evaluation may utilize instructor processes such as Gagné’s nine instructional events (1985), the one-minute paper by Schwartz (Wilson 1986) or the course material and classroom observation checklists designed by Brent and Felder (2004).  For example, Gagné’s eighth instructional event of “assessing performance” (1985) helps the assessor gauge the success of achieving a stated instructional outcome.

Keeping the social-cognitivist theory of learning in mind, I designed a Peer-Assessment Lecturer Survey (PALS) which unites the requisite components of an instructor’s cognitive process, content organization and presentation behaviours into a simple checklist.  The PALS follows an instructor’s instructional event matrix (Figure 2) (Gagné 1985, Okey 1991) and utilizes a simple yes/no checklist to quickly tick off points and provide comments as it unfolds in real-time.  This procedural framework provides opportunities for both a rigid process checklist and as well as subjective, interpretive and content-based comments similar to Brent and Felder (2004.)  Instead of including numerous sub-checklists or Likert scales, a short tally of “positive” and “delta” points is used with cues that mentally prime the assessor to actively provide presentation and guidance comments on lecture skills (Allen 1969, Passi 1976).  Finally, the inclusion of questions from the one-minute paper (Wilson 1986) helps the assessor hone in on the most important points for peer-feedback.

Figure 1. Sample concrete Behaviourist skills on Movements and Gestures (Allen 1969)

Figure 2. Gagné’s Instructional Event and Learning Outcome Taxonomy Matrix

Bibliography

Allen, D.W. et.al. (1969) Micro-teaching – A Description. Stanford University Press

Bandura A (1986) Social Foundations of Thought and Action. A Social Cognitive Theory. Prentice-Hall, Englewood Cliffs, NJ.

Bloom, B.S. (1984) Taxonomy of educational objectives.  Published by Allyn and Bacon, Boston, MA.

Brent, R. Felder, R. (2004) A protocol for peer review of teaching Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition

Gagné, R.M. (1985) The conditions of learning and theory of instruction. Holt, Rinehart and Winston.

Kaufman, D.M. (2011). Ch 2 Teaching and learning in medical education: how theory can inform practice. In Swanick, T. 1^st ed. Understanding Medical Education: Evidence, Theory and Practice (Kindle Locations 1809-1810). Wiley. Kindle Edition.

Okey, J.R. (1991)  Procedures  of  Lesson  Design  Ch.  8   In lnstructional Design: Principles  and Application  2^nd edn. Edited by Leslie  J Briggs et al. Education  Technology  Publications pp192-208.

Passi, B.K. (1976) Becoming Better Teachers. Baroda : Centre for Advanced Study in Education, M. S. University of Baroda

Wilson, R.C. (1986) Improving Faculty Teaching: Effective Use of Student Evaluations and Consultants. Journal of Higher Education, 57 (2), pp. 196-211.

(prepared for a Medical Education assignment)

Sample.

Peer-Assessed Lecturer Survey (PALS)

Instructor:                                                                    Topic:

                                                                                                                                                                                    Notes & Comments

I. Gain Attention: Was your interest aroused?	[  ]  Yes       [  ]  No		When providing comments, make them: supportive, descriptive, specific, & behavioural.
II. Objectives were stated at the beginning and 1.     Clearly utilize active verbs from Bloom’s taxonomy 2.     Demonstrated what was expected of the Learner by the end of the session	[  ]  Yes       [  ]  No   [  ]  Yes       [  ]  No
III. Review of prerequisite knowledge 3.     Started at an appropriate challenge level 4.     Checked-in with audience to adjust & match needs 5.     Covered in less than 5 minutes	[  ]  Yes       [  ]  No   [  ]  Yes       [  ]  No   [  ]  Yes       [  ]  No
IV. ORAL Presentation skills Tally up positive and delta points ( ) on subjects like: ·         Voice: clarity, volume, energy, speed, um’s/er’s ·         Non-verbal cues: directive focusing (pointing/laser pointer), gestures/movement, & deliberate pauses	(+)	Δ
V. A-V & Guidance skills Tally up positive and delta points ( ) on subjects like: ·         Overhead/PowerPoint/Prezi, graphics, & videos ·         Used cases, story, analogy, examples, prompts & hints appropriately	(+)	Δ
VI. Facilitated Practice 6.     Opportunity provided for active involvement, student participation and/or practice	[  ]  Yes       [  ]  No
VII. Feedback 7.     Provided active feedback to audience 8.     Comments were supportive, descriptive, behavioural	[  ]  Yes       [  ]  No   [  ]  Yes       [  ]  No
VIII. Assessment of Objectives 9.     The instructor “closed the loop” on learning objectives with a post-instruction assessment 10.   Higher-order thinking questions were asked	[  ]  Yes       [  ]  No   [  ]  Yes       [  ]  No
IX. Further review after instruction 11.   Summary/outline provided 12.   Additional opportunities to master material (i.e. tools/homework) were provided for further practice	[  ]  Yes       [  ]  No   [  ]  Yes       [  ]  No

The most important take-home point was:

The muddiest point in the lecture was: