The Development
Process for a novel “Peer-Assessment Lecturer Survey” (PALS)
(available on request by word format)
The peer-evaluation
of an instructor can be approached from different perspectives. In reviewing methods of evaluation, I found the
social-cognitivist theory elucidated by Bandura (1986) to be helpful. It links the
behaviourist approach, which emphasises the influence of the environment on our
actions, and the cognitive approach, which emphasises the importance of
cognition in mediating our learning and functioning. (Kaufman 2010)
For
lecture evaluation, the behaviourist approach focuses on the
presentation process, learning environment and instructor content. This is best demonstrated by microteaching of
skills of Allen (1969) and Passi (1976) seeing the instructor from a mechanical
perspective. For example, the microskill
of “stimulus variation” in which an assessor may critique “using gestures to help
convey extra meaning” or “at various times, the teacher was noted in the left,
right, forward, and back of the teaching space” (Figure 1) (Allen 1969). These principles are old-fashioned and
mechanistic but useful.
The cognitivist
approach shifts the emphasis to learner-centred content and outcomes
where the substantive parts of evaluation occur internally within the minds of
the instructors and students. Lecture
evaluation may utilize instructor processes such as Gagné’s nine instructional
events (1985), the one-minute paper by Schwartz (Wilson 1986) or the course
material and classroom observation checklists designed by Brent and Felder
(2004). For example, Gagné’s eighth instructional
event of “assessing performance” (1985) helps the assessor gauge the success of
achieving a stated instructional outcome.
Keeping
the social-cognitivist theory of learning in mind, I designed a Peer-Assessment Lecturer Survey (PALS)
which unites the requisite components of an instructor’s cognitive process, content
organization and presentation behaviours into a simple checklist. The PALS follows an instructor’s instructional
event matrix (Figure 2) (Gagné 1985, Okey 1991) and utilizes a simple yes/no
checklist to quickly tick off points and provide comments as it unfolds in
real-time. This procedural framework provides
opportunities for both a rigid process checklist and as well as subjective,
interpretive and content-based comments similar to Brent and Felder (2004.) Instead of including numerous sub-checklists
or Likert scales, a short tally of “positive” and “delta” points is used with
cues that mentally prime the assessor to actively provide presentation and
guidance comments on lecture skills (Allen 1969, Passi 1976). Finally, the inclusion of questions from the
one-minute paper (Wilson 1986) helps the assessor hone in on the most important
points for peer-feedback.
Figure 1. Sample concrete Behaviourist
skills on Movements and Gestures (Allen 1969)
Figure 2. Gagné’s Instructional Event
and Learning Outcome Taxonomy Matrix
Bibliography
Allen, D.W. et.al. (1969) Micro-teaching – A Description. Stanford
University Press
Bandura A (1986) Social Foundations of Thought and Action. A Social Cognitive Theory.
Prentice-Hall, Englewood Cliffs, NJ.
Bloom, B.S. (1984) Taxonomy of educational objectives. Published by Allyn and Bacon, Boston, MA.
Brent, R. Felder, R. (2004) A protocol
for peer review of teaching Proceedings
of the 2004 American Society for Engineering Education Annual Conference &
Exposition
Gagné, R.M. (1985) The conditions of learning and theory of instruction. Holt, Rinehart and Winston.
Kaufman, D.M. (2011). Ch 2 Teaching
and learning in medical education: how theory can inform practice. In Swanick, T.
1st ed. Understanding Medical
Education: Evidence, Theory and Practice (Kindle Locations 1809-1810).
Wiley. Kindle Edition.
Okey, J.R. (1991) Procedures of
Lesson Design Ch. 8 In lnstructional Design: Principles and Application 2nd edn. Edited by Leslie J Briggs et al. Education Technology
Publications pp192-208.
Passi, B.K. (1976) Becoming Better Teachers. Baroda :
Centre for Advanced Study in Education, M. S. University of Baroda
Wilson, R.C. (1986) Improving Faculty Teaching: Effective Use of
Student Evaluations and Consultants. Journal of Higher Education, 57 (2),
pp. 196-211.
(prepared for a Medical Education assignment)
Sample.
Instructor: Topic:
Notes & Comments
I. Gain Attention:
Was your interest aroused?
|
[ ] Yes
[ ] No
|
When
providing comments, make them:
supportive,
descriptive, specific, & behavioural.
|
|
II. Objectives were
stated at the beginning and
1. Clearly
utilize active verbs from Bloom’s taxonomy
2. Demonstrated
what was expected of the Learner by the end of the session
|
[ ] Yes
[ ] No
[ ] Yes
[ ] No
|
||
III. Review
of prerequisite knowledge
3. Started
at an appropriate challenge level
4. Checked-in
with audience to adjust & match needs
5. Covered
in less than 5 minutes
|
[ ] Yes
[ ] No
[
] Yes [
] No
[ ] Yes
[ ] No
|
||
IV.
ORAL Presentation skills
Tally
up positive and delta points (
·
Voice: clarity, volume, energy, speed, um’s/er’s
·
Non-verbal cues: directive focusing (pointing/laser
pointer), gestures/movement, & deliberate pauses
|
(+)
|
Δ
|
|
V.
A-V & Guidance skills
Tally
up positive and delta points (
·
Overhead/PowerPoint/Prezi,
graphics, & videos
·
Used cases,
story, analogy, examples, prompts & hints appropriately
|
(+)
|
Δ
|
|
VI.
Facilitated Practice
6. Opportunity
provided for active involvement, student participation and/or practice
|
[
] Yes [
] No
|
||
VII. Feedback
7. Provided
active feedback to audience
8. Comments
were supportive, descriptive, behavioural
|
[ ] Yes
[ ] No
[
] Yes [
] No
|
||
VIII. Assessment
of Objectives
9. The
instructor “closed the loop” on learning objectives with a post-instruction
assessment
10. Higher-order
thinking questions were asked
|
[
] Yes [
] No
[ ] Yes
[ ] No
|
||
IX. Further review
after instruction
11. Summary/outline
provided
12. Additional
opportunities to master material (i.e. tools/homework) were provided for
further practice
|
[
] Yes [
] No
[
] Yes [
] No
|
||
The most important take-home point was:
The muddiest point in the lecture was:
Posts
