Embedding participatory assessment in Critical Intercultural Dialogue course

Authors

Cláudia Pato Carvalho, University of Coimbra
Cristiano Gianolla, University of Coimbra
Manuela Guilherme, University of Coimbra

Keywords:

student-centred learning, peer assessment, self-assessment, critical pedagogy

Summary:

This short paper analyses the pedagogical value of participatory student peer and self-assessment, as a form of critical pedagogy. It discusses feedback received from three cohorts of international students from 15 countries on the optional course Critical Intercultural Dialogue (CID), which is part of a master’s programme on Sociology, offered at the Faculty of Economics at the University of Coimbra. Students’ agency in the CID assessment process was expected to strengthen their commitment and participation in the course. Traditional assessment by professors was changed into three separate components, students were assessed by: (1) their teachers, (2) their peers and (3) themselves. Assuming that assessment is part of the teaching and learning process, students could not be excluded from the critical and participatory approach, which guided the course pedagogy. Therefore, students were invited to be an active and complementary part of the assessment process through peer and self-assessment, which counted between 9% and 12% of the overall assessment. Students in their feedback appreciated this approach and noticed how the assessment criteria fitted the course pedagogy.

The course Critical Intercultural Dialogue (CID) is based on a student-centred approach, which values student engagement and effective class participation. The methodological approach assumes that higher education may provide the ground for collaborative and critical learning. This space is characterised by reciprocal sharing of knowledge and a participatory approach to the study of intercultural dialogue, not only between teachers and students, but also among students. It does so by focusing on the understanding of cultures in different socio-political contexts.

 

CID was designed to elaborate on dialogic assessment, based on active student participation and autonomy, as ways to develop in a continuous learning process, embedded in a participatory, creative, and critical pedagogy, including peer and self-assessment as a formative process of self-improvement. In concrete terms, the challenge was to create assessment procedures that enhanced a holistic teaching and learning participatory approach for a broad range of international students. Furthermore, CID’s assessment procedures aim to critically engage with different perceptions of hierarchy between teachers and students, which are sometimes subtle but, as we believe, always influence student learning.

Critical Intercultural Dialogue’s pedagogy assumes that assessment is not merely a way for teachers to grade students (as requested by the educational system); it is also a central component of a dialogical approach to the teaching and learning process. This is especially valuable in relation to the international group of students who attend this course, in that they are strongly motivated to develop ‘critical (inter)cultural awareness’ (Guilherme and Menezes de Sousa 2019). This also impacts the dialogical procedure and context of the entire teaching and learning evaluation process. The teaching and learning process proposed in CID aims to comprehensively expand the cooperation of students and professors, and to share responsibility about the assessment process, which entails active listening to each other (Freire, 1970; 1998; Romão 2005).

 

Cognisant of the fact that we, as the three course lecturers, are full-time researchers whose teaching is a secondary activity, such a proposal of Freire’s problem-posing praxis responds to CID’s pedagogy because our teaching strategies are embedded in critical, dialogical and intercultural problem-posing activities that aim to instigate student engagement in research practice. Following this line of thought, CID teachers and students have promoted interculturality as a critical dialogue, hence, as “a permanent and active process of negotiation and interrelation, in which difference does not disappear” (Walsh 2018: 59).

 

At the beginning of the course, the students were requested to reflect on and contribute to the definition of the assessment criteria and to take them into account as they carried out coursework for each assessment component. CID’s assessment is structured in a way that it is sequential throughout the course and is based on successive and complementary feedback, aimed at allowing students to develop ownership of the teaching and learning process, improve their engagement and contribute to the course development. In this context, we embrace the two concepts (assessment and evaluation), as we use them during the process of the CID teaching and learning experience.

 

“Assessment” is an informative evaluation related to a situation when the students receive a quantitative grade, given by peers and/or by teachers.  “Evaluation” is a formative evaluation, and therefore qualitative, relating to contexts when the teachers and students receive qualitative feedback. These two evaluation methodologies interrelate in the case of the Critical Intercultural Dialogue Course since we needed to give a final grade and were simultaneously developing a participatory and dialogical pedagogy. The students’ first assignment was to choose one theme from a list provided to carry out a collaborative empirical study (group work). Presentations had to dialogue with the two preparatory readings of the respective lecture, provide contextual examples from the empirical case(s) selected by the group and further literature. Students were expected to show individual progression as well as the capacity to transform such progression into critical thinking through group collaboration (Collins and Delgado, 2018; Risager, 2013).

 

Presentations were followed by an open debate in the plenary where presenting students responded to questions and comments. The debate was opened to incorporate reflections on the assessment procedure and merits of the presenters. Students were also requested to make an individual presentation during the penultimate lecture of the course by choosing a keyword that the course had triggered them to analyse. Each student had 5 minutes to present the keyword and then a short, 5-minute debate followed. The keyword presentation could be related to the group presentation and final essay.

 

After each debate, the peer and self-assessment took place anonymously. Students were given 10 minutes to mark the presentation, using a form prepared for each session indicating the names of all the presenting students and a mark selector (scale 0-20). This form enabled both peer and self-assessment, as presenting students also assigned a mark for themselves. Before providing the link to the form, professors reminded students of the five assessment criteria, namely: dialogue with the topics of the lectures; quality of exposition, coherence, and capacity of synthesis; theoretical and empirical contribution; critical analytical contribution; and originality and innovation.

 

During the first and second cohort, CID assessment was divided into three interim assessment components: participation and engagement throughout the course (20%), oral presentation of group work and keywords (30%) and final essay (50%). Peer and self-assessment counted for 9% of the overall final mark, exclusively within the oral presentation of group work and keywords assessment. Therein, the marking breaks down as follows: self-assessment 15% (4.5% of the total mark) and peer assessment 15% (4.5% of the total mark). The remaining 70% for the participation and engagement assessment component (21% of the total mark) was provided by professors’ assessment.

 

In the third cohort, the assessment was divided into two components: student participation and engagement (40%) and final essay (60%). Peer and self-assessment counted for 12% of the final mark within the student participation and engagement component, broken down as follows: self-assessment 15% (6% of the total mark), peer assessment 15% (6% of the total mark), with the remaining 70% (28% of the total mark) again attributed to professors’ assessment. The increase of the weighting for peer and self-assessment was motivated by a positive experience with the method, with the objective to enhance it and expand student’s ownership of the assessment. (Collins and Delgado 2018; Risager 2013)

Critical Intercultural Dialogue is an optional course offered in English and held in the spring semester (February to July) at the University of Coimbra (Portugal), School of Economics, within the framework of the Master’s Programme in Sociology. It is divided into 14 three-hour, weekly lectures, granting 7.5 ECTS upon completion. Each lecture is divided into two parts, separated by a short interval of 10 minutes. The first part is led by a lecturer’s presentation and each lecture is structured to be highly interactive, practice-based learning (i.e. adapting methods like small group activities, research-debates, role-playing, instant reports, note-sharing, micro-papers, etc.). The second part of the lecture is dedicated to student-led research activities or pre-assigned student group presentations and a final open debate.

 

The student population has been the following: 12 students studied on the course in the academic year 2018-2019, 16 students in 2019-2020 and 8 students in 2020-2021. In total, we have registered seven home and 29 international students. Due to the pandemic, the course was offered to the second cohort as an online course, while the third cohort started learning remotely and later changed to a mixed learning environment, where students could choose whether to participate online or in-person.

 

The course is led by three core lecturers (the co-authors of this text), two Portuguese and one Italian-Portuguese with backgrounds in Education, Philosophy, Sociology and Political Science and from different generations. Guest lecturers are invited for specific topics each year. Whereas only Master’s students can attend the course among home students, both BA and MA mobility students are eligible to enrol. International students recurrently come from disciplines such as social sciences, management, and economics. The course was offered for the first time in the academic year 2018-2019 and has been redesigned each year, based on the class composition and the feedback received from previous students, in order to improve the quality of the teaching, learning process and the level of student participation.

The experience of peer and self-assessment in the course Critical Intercultural Dialogue has been evaluated via student feedback from three cohorts. At the end of the course, students provided feedback in two distinct ways, firstly, through oral feedback during class debates about the course content and teaching methods, and immediately after, by filling in an online feedback form designed by the course teachers. All students were asked to anonymously rate the course content, pedagogy, and assessment procedures and to explain their choices. We analysed the student feedback and, from our own perspective as course coordinators, we summarise the key points which the students repeatedly expressed below.

What follows is an overview of the feedback given by students in the three editions of the course, focusing on how students perceived the peer and self-assessment process.

 

Students from the first cohort (2018-2019) appreciated that the course dealt with a diversity of topics (racism, prejudice, intolerance, minorities, among others) from different angles, taught by guest lecturers in articulation with the in-situ professors. The students’ diverse backgrounds and, alongside the fact that the course includes a small/medium size group, provided excellent conditions for the course lecturers to make students feel included and that their opinion mattered for the course progression, as expressed through peer and self-assessment. This feeling was of paramount importance for positively impacting on student learning through continuous self-reflection, encouraging students’ disposition for reflexive, critical self-assessment, whilst providing a comfortable environment for dialogical peer-assessment. Moreover, the students praised the pedagogical approach. As mentioned by one of the students: “Since the course was based on participation and since it was possible to give feedback about the syllabus […] I evaluate pedagogy methods and approach very high”. This showed the pedagogical value of interconnecting assessment procedures with student participation and learning.

 

The second cohort (2019-2020) enrolled on the course during the pandemic. Overall, and apart from the pandemic restrictions, students similarly praised the pedagogical approach and teaching methods, saying they broadened students’ vision, improved their critical thinking skills and ability to see studied topics from different perspectives. Students also appreciated the fact that the lecturers did not introduce hierarchical relations with the students, which also resulted in students feeling more at ease and safer with peer and self- assessment. The guest speakers’ expertise on such diverse topics as racism, migration, prejudice, gender, culture or interculturalism were believed to open student minds and broaden their knowledge. Students thought this made it easier for them to weigh multiple perspectives whilst assessing different performances.

 

The class rule to mix students from different nationalities in all in-class activities and the fact that one of the main pedagogical approaches was group and peer discussion, both with in-situ teachers and invited guests (surprisingly a new approach for many students), were highlighted by the students as contributing to the course success: “I liked your pedagogical approach that you mixed people of different nationalities […]. And I liked that there was peer-assessment so that we could evaluate each other!”

In general, students mentioned that their capacity for intercultural dialogue was improved due to self- and peer-evaluation (please see above how we interrelated both assessment and evaluation activities). Students demanded more regular opportunities throughout the course to contribute to assessment, so that there was time for improvement. In some cases, students felt challenged by marking colleagues’ work during the peer-assessment process. Since students could join the course up until the fourth lecture, some students reported that the assessment method had not been sufficiently discussed with all the students at the beginning of the semester. Yet, in general, students said that their capacity for intercultural dialogue improved due to being engaged in self-assessment.

 

Student feedback from the third cohort (2020-2021) reveals an improvement in the course organisation. The non-hierarchical positioning of course teachers towards students was again appreciated, which we linked to our multiple efforts to encourage student participation. For some students this was a unique experience. “It was a completely new experience for me – I am not used to discussing anything in the lectures, in most of the other courses throughout my school years I was more listening and just accepting the information and here I could immediately react and say my opinion”.

In addition, students again referred to the benefits of learning from peers with different cultural backgrounds, appreciated the opportunity to learn via informal and continuous dialogue and valued the class dynamics allowing everyone to participate and share their different points of view in a secure environment. For some students this was a unique opportunity to share their opinion in an international environment where they could freely decide if they wanted to engage in debate or not.

The introduction of peer and self-assessment does not entail specific preconditions; it can be implemented in courses of different disciplines. It particularly fits courses that privilege a participatory approach and active engagement, and especially at an international level. The method may be improved by introducing a collective self-moderating mechanism, for example, a collective exercise through which students are requested to substantiate their assessment process with arguments that can convince their peers. This process of reflection on the assessment method would be another pedagogical component of the course. Such “qualitative” implementation could lead students to engage with assessment more actively and responsibly, fostering professional relationships with their peers. It could fine-tune peer-learning opportunities for students by comparing the quality of their work with that of their colleagues.

This paper discussed the use of peer and self-assessment when teaching a course on critical intercultural dialogue. It looked at student perceptions of being in the role of co-assessors of their peers’ and their own work. Student feedback collected in the three editions of the course suggests that the inclusion of different learning themes, the synergies between course leaders and guest lecturers and the horizontal teaching approach (as opposed to the hierarchical), stimulated student participation and facilitated self and peer-assessment. Students valued opportunities for continuous self-reflection and critical approaches towards each other’s work, their own work, and the course itself; ultimately broadening students’ vision, which stemmed from the course pedagogy. They thought these factors had positive impacts on student learning, enabling a dialogical peer-assessment process. Students believed that the assessment procedures were directly interconnected with student participation and learning.

  • Collins, H. and Delgado, I.A. (2018) ‘Critical cosmopolitanism in participant dialogue: Grappling with essentialism, negotiating difference and resisting closure’, Language and Intercultural Communication 19:6, pp. 541-554.

  • Freire, P. (1970) ‘Cultural action and conscientization’, Harvard Educational Review 40:3, pp. 452-477.

  • Freire, P. (1998) Teachers as cultural workers, Boulder, CO: Westview Press.

  • Guilherme, M.M. and Menezes de Souza, L.M. (eds.) (2019) Glocal languages and critical intercultural awareness: The South talks back, New York and Abingdon, UK: Routledge.

  • Risager, K. (2013) ‘Cultural awareness’, in M. Byram and A. Hu (eds.) Routledge encyclopaedia of language teaching and learning, London and New York: Routledge, pp. 181-184.

  • Romão, J.E. (2005) Avaliação dialógica: Desafios e perspectivas, São Paulo: Cortez Editora.

  • Walsh, C. (2018) ‘Interculturality and decoloniality’, in C. Walsh and W. Mignolo On decoloniality: Concepts, analytics, praxis, Durham: Duke University Press, pp. 57-80.