Evaluation – a necessary evil?
Quality assurance: how to meet the requirement
Since the famous Bologna Process, quality assurance has become a virtual sinecure which, according to its disciples, solves all problems, even where there are none to be solved.
The universities have always had various evaluation mechanisms which generally rely on collegiality or, at the very least, on the involvement of the university community: professors, lecturers, part-time lecturers, support staff, professionals, administrators and, of course, students.
“By replacing the traditional model for programme evaluation with a model in which the opinion of agencies, of external bodies, takes precedence over that of the university community in question, the government will weaken collegial structures. The European experience clearly illustrates the impact of such a transformation on the university community [9].”
In fact, this standardised form of evaluation reduces the process to a set of statistics that the universities might not share. All that just for the sake of “opening ourselves up to the world”, “ensuring a place at the top of the international league tables”, vying for a competitive position and an international profile on the world knowledge market.
The universities, in the manner of a factory producing machine parts or computers, should enter a “customers’” race in order, we are told, to democratise knowledge, although fundamentally it is nothing more than a vulgar obsession with profit.
In such an environment, it is the university’s image, the perception of the market, or, in other words, its position in a vast number of league tables rather than the quality of the education that is most important.
Classification
The evaluation of education and teachers is being raised more frequently throughout the world. Its aim, whether stated or not, is quality assurance: to improve the quality of education. It also aims to provide parents and students with an informed choice and give taxpayers real value for money. And to enable parents to send their children to the “best” school or university from a “scientific” shortlist. Evaluation also enables us to place the various establishments in neat columns, classified from “best to worst”.
These may appear to be the noblest of intentions, without any disastrous consequences. The truth, however, is quite different. The interpersonal nature of education does not easily lend itself to a traditional evaluation process. Failing to take this into consideration completely changes the fundamental rapport required in all forms of education.
From the Bologna Process [4], to the numerous evaluation mechanisms set up in colleges and universities, there are simply so many spin-off systems. Here are a few examples, mostly from Quebec, that illustrate the situation with regard to evaluation, mainly in universities.
Who evaluates whom? Evaluation of teachers by students
The evaluation of teachers and teaching is practised in all of Quebec’s universities. Although it takes various forms, it does meet basic criteria that are generally the same from one establishment to the next. At the University of Laval in Quebec, part-time lecturers are evaluated at the end of their course [5].
The evaluation, also called an appraisal, aims to “look at the teaching and training activities, and the education provided by the part-time lecturer”. Its goal is to correct, redirect, improve or adjust the teaching activities and performance of the part-time lecturer, where appropriate, in order to provide quality education [6].
The process requires the part-time lecturer to leave the class at the time of evaluation and that at least 60 per cent of the students enrolled on the course take part. An appraisal with a satisfaction rating below 80 per cent leads to measures being taken to correct the situation.
These measures range from a meeting with the department or faculty management to the loss of the PECC ( Profil d’engagement des chargés de cours). The PECC confirms that a part-time lecturer has the necessary skills to teach a particular. All evaluation reports are placed in the part-time lecturer’s file, which will be consulted when he/she applies for a new PECC.
Cost of internationalisation
The internationalisation of our universities to make them more competitive and of equal or greater value than others leads, together with the decline of collegiality, to the beginnings of global standardisation.
Some programmes will become stagnant or will disappear altogether; others will be very popular and will receive intensive financing. And we will have to accept the Anglicisation of some programmes. Here in Quebec, where Anglicisation is proceeding at breakneck speed, this drive towards internationalisation does not only raise questions of quality assurance and other evaluation processes, but also the delicate question of identity and the protection of the French language in Quebec and in North America.
Reinforcing, improving and enhancing existing methods of evaluation
The CSN believes that universities, or at least those in Quebec, already possess all the tools they need to monitor, evaluate and manage the quality of education. Everything is in place for a truly collegial administration.
Indeed, the university community, which actually ‘makes’ the university, works there, studies there, teaches there and lives there every day, is best placed to realise this.
There are numerous committees for programmes, departments, experts and improvements as well as the studies commission and the programme review board, the University Council or the Administration Council, etc., all of those bodies where the university community already rules.
There is no need, in our opinion, to make the processes more cumbersome. Although it may be necessary to revise certain methods of operating, some agreements relating to the representativeness of the community, or even to create other joint bodies, there is certainly no need to place our universities in the hands of “entities or independent members” wanting what is good for us… and making sure that we get it!
Finance as yardstick of quality
Back at the University of Laval, the recent election to the Dean’s Council in the spring of 2012 focused on the university’s place in international league tables, on “the university’s rise up the table over the last five years [10]”, on property development, on the leadership chairs wholly sponsored by the private sector, etc.
Scarcely a word was heard about education or teaching, about the importance of education in individual and collective emancipation, about the impact of culture and knowledge…
The same story applied at the University of Montreal where, according to the dean, Guy Breton, “the quality of a university can’t be measured by definitive qualitative criteria.” According to him, “quality” can only be measured by comparison, i.e. from a relative viewpoint.
It cannot be measured on the basis of the scientific or academic nature of the education and its content, but can be measured in accordance with the financial resources at the disposal of an establishment:“Quebec is not on another planet. The quality of education is something relative and, as with anything relative, it must be compared before it can be properly placed [11].”
Partial – or impartial – evaluation?
There is considerable pressure to measure and evaluate everything. While nobody opposes certain forms of evaluation and improvement, the types of measurement and the intended objectives pose a persistent problem.
When student bodies propose online evaluations available 24 hours a day during a period that can extend beyond the allocation of final grades, it is easy to understand the concerns of part-time lecturers, particularly when the teacher is a contract employee (increasingly the norm [7]).
Even when subject to formal agreements and proper management, there are loopholes in the appraisal and questions that arise from it. What about student objectivity, among other things? Are they evaluating the teaching or the teacher? Is the evaluation “impressionistic” rather than formative or summative? Should we create evaluation committees consisting of peers, students, administrators, didactic pedagogues, human resource representatives...?
The complexity of such a project is obvious and will resolve nothing without considering the intended objectives. Should we not take a long, hard look at these processes of evaluation, improvement, performance enhancement, etc., as it all seems to devolve from some paradigm shift in the fabric of the university?
Using a terminology that smacks of business and management-speak rather than the passing on of knowledge, is evaluation not being dragged into the world of commercialism and competitiveness?
At the Confederation Des Syndicats Nationaux (CSN), we believe that “the university must be evaluated in line with quality and performance criteria that are appropriate to all of its activities and accessibility to studies [8].”
In this regard, we can see that although there is nothing negative about the implementation of quality-assurance mechanisms in itself, the method used to evaluate a service or a product [1] risks affecting its very essence - its raison d’être- and even the goals of the service or product.
While it is fairly obvious how to evaluate an assembly line or the various stages in the production of a car or a washing machine, it is much more difficult to envisage the evaluation of something complex such as education.
Evaluating intangibles
Teaching is an action as well as a profession, and is a human and interpersonal relationship at heart. It calls for a wide range of the most subtle and abstract elements of human communication. The transmission of knowledge from the kindergarten to higher education relies on the most fundamental elements of interpersonal rapport: trust, complicity, mutual respect, collaboration, exchange, interaction.
How can we evaluate a “relationship of trust” between a teacher and a pupil? If there is freedom of education and if teaching is subjective, how can it be evaluated? How can we measure the degree of spontaneity or involvement on the part of a student?
If learning is a voluntary act, as US author Leo Buscaglia said, how can we evaluate such an act? If an appropriate measuring tool existed, would a teacher be objective enough and the best person to use it on his/her students... and vice versa?
There are, in summary, two main forces at work behind this obsession with evaluation: 1) the disengagement of governments from the financing of universities; 2) the resulting entry of the universities into the world of cutthroat competition.
It is this same mercantile madness that causes university presses to publish without counting, because what really matters is the number of books to the exclusion of all else (regardless of whether or not anyone reads them!). It is the number of publications that decides the ranking and, therefore, pole position.
In 1980, Cambridge University Press published 543 titles. In 2000, it published 2,376. Oxford University Press published 802 titles in 1980, increasing to 2,250 in 2000 [2]. Not only did university presses publish more than 31 million books, I would argue that this overproduction is intended to boost the universities’ ratings, to make them more attractive to professors and, of course, international students.
Perhaps we should refer to the main principle of the Magna charta universitatumqui [3], which insists on the “political, economical and ideological” independence of universities.
Footnotes
[1] See, among others, Vincent de Gaulejac, La société malade de la gestion, Seuil, 2009, in which he explains how whatever is being evaluated risks distancing itself from its raison d’être when it adapts to meet evaluation criteria. For example, a public service evaluated in this way will no longer be able to fulfil the purpose for which it was created after changing its characteristics to satisfy the evaluation criteria. De Gaulejac is also interested in the growing costs associated with this management obsession.
[2] Lindsay Waters, L’éclipse du savoir, Allia, 2008, p. 17(Original title: Enemies of Promise: Publishing, Perishing, and the Eclipse of Scholarship, 2004)
[3] http://www.magna-charta.org/library/userfiles/file/mc_french.pdf
[4] Further information: http://www.coe.int/t/dg4/highereducation/ehea2010/bolognapedestrians_fr.asp
[5] For each course until the end of the probationary period, then for every other course, or in line with other departmental procedures.
[6] Collective agreement by the Syndicat des chargées et chargés de cours de l’Université Laval(SCCCUL), 2007 – 2010, p. 39
[7] Part-time lecturers teach 50 per cent of courses in the first two-year ‘cycle’ in Quebec.
[8] Education platform for CSN : http://www.csn.qc.ca/web/csn/documentation
[9] Marie Blais, Assurance qualité : la réingénierie de l’université québécoise continue, Vie économique, 2012, p. 8
[10] Denis Brière, current dean of the University of Laval, in Contact, autumn 2012, vol. 27. no. 1, p. 5
[11]Éric Martin in: http://www.iris-recherche.qc.ca/blogue/le-spectre-du-sous-financement