This is rather surprising, in particular when one considers that many European quality agencies have been in operation for 20-plus years. There was a kind of wave of new quality assurance bodies established in the 1990s, with organisations being founded at almost the same time in the North-Western and Central-Eastern parts of Europe.
Joint projects and support from various international organisations were vital for this capacity building and for professional advancement. Over the years, the professionalism of quality assurance staff has grown and we now have a separate category of quality managers, even if this is not yet a profession in its own right.
All this time there has been mutual learning and experience sharing through regional networks such as the European Association for Quality Assurance in Higher Education or ENQA and the Central and Eastern European Network of Quality Assurance Agencies in Higher Education or CEENQA, both established in 2000.
So we already have some experience. Moreover, there is a shared vision about how things should be done which has been consolidated into a European model called the ESG – the Standards and Guidelines for Quality Assurance in the European Higher Education Area.
The ESG is something that all stakeholders in Europe – from national authorities, higher education institutions and quality agencies to student organisations and business partners – have agreed and it has been in force for 12 years now.
What makes for quality education?
Over time, the sophistication and variety of instruments used to measure quality has increased. Many agencies have tried several different types of procedures – single programme evaluations, clustered evaluations, audits of quality systems, departmental reviews and institutional reviews, to name a few.
It is natural to assume that each shift was preceded by analysis of what the previous work yielded. This is what quality agencies are expected to do – to engage in thematic analysis of procedures and national and international higher education systems. However, external quality agency reviews, carried out by ENQA, demonstrate that this is still a challenge for quality assurance agencies.
The puzzling thing is that we not only have great difficulty assessing quality higher education, but agreeing on what it is in the first place.
As both practice suggests and research shows, different stakeholders have different views about what quality means for them. Does it mean excellence, meeting threshold standards, transformative potential, fitness for purpose, transparency and accountability? These are just a few of the possible definitions.
This lack of agreement has far-reaching consequences because when public authorities set the agenda and seek to satisfy public demands and sometimes their own political interests, they can be prone to changing policies and definitions about what quality agencies should be measuring and in what proportions.
As a result, we have constant changes in procedures, criteria and indicators. In the last ENQA members’ forum, almost half of the agencies indicated they were undergoing changes of procedures or of their own organisational structures.
Latest trends are to measure quality by greater focus on teaching and learning experiences, institutional risks and the employability of graduates. What we evaluate depends upon what we value.
Some things are more difficult to measure
Other stakeholders, such as international organisations, also have their own policy goals and preferences. For example, the European Commission, in its revised modernisation agenda, talks about the growing importance of civic society and the third mission of universities, especially in the face of recent challenges like massive migration and clashes of cultures.
But there is one pitfall: not everything that could be counted counts and not everything that counts can be counted. Measuring quality of teaching and learning experiences and assessing institutional risks present difficult challenges. But pressure on metrics and comparisons, including international ones, is high.
There is one aspect which I find quite difficult to acknowledge, coming as I do from a quality assurance agency background. It is that, besides procedures that produce quick assessments of educational provision, it is hard to measure the long-term impact of external quality assurance. This is because higher education is a complex phenomenon, with multiple purposes and multiple actors. It is not easy to deal with such complexity.
Quality, to quote the ESG, is mainly a result of the interaction between teachers, students and the institutional learning environment. Thus, quality assurance agencies always play a secondary role in achieving quality.
Yet what has the greatest impact, based on what quality assurance agencies say and some surveys of higher education institutions show, comes from slow, steady work to build institutional quality cultures through regular reviews and interaction between internal and external quality assurance.
Quite often neither politicians nor university leaders have the time to do this in their haste to demonstrate the difference made during their time in office.
To sum up, external quality assurance depends a lot on what element of what higher education is for – research, employment, personal development or societal advancement – is more dominant at any given time. It is contingent upon national context, but there are international trends that can be observed. The expectations of stakeholders are rising all the time, but we see ever greater stratification of quality and challenges remain.
Aurelija Valeikiene is deputy director of SKVC, the Centre for Quality Assessment in Higher Education – Lithuanian ENIC-NARIC centre and external quality assurance agency.
The present essay is reprinted from the University World News (Issue 00483 dated 19 November 2017), which being a high quality international newspaper and website provides a global view on higher education.