Notes on: forde-leaves, N., Walton, J., Tann, K. (2023) A framework for understanding assessment practice in higher education. Assessment and Evaluationin Higher Educuation. 48 (8) 1076--91. DOI: 10.1080/02602938.2023.2169659

Dave Harris

Conflicts and assessment arise from conflicting demands on the curriculum and there are three terms in assessment discourse such as 'autonomy, logic and the basis of success'. They apply LCT to develop a new holistic framework which will analyse assessment practice cross discipline institution and geographical boundaries.

There are different purposes of HE globally, and segmentation. There have been calls to rethink assessment theory and practice, although relatively little discussion about generative mechanisms or conceptual bases, a general under theorising. It is still a 'cottage industry… Attempts to enhance assessment practice built on sand' (1076), with 'empirical saturation'. We need meta theories to get at principles. LCT as in Maton will expose dichotomies and trace them in assessment practices and lead to 'socially contextualised idealised positions' (1077).

Assessment practices serve competing purposes — certification, progress, transfer, accountability, supporting learning. These reflect 'an outward pull' from external constituencies which affect university autonomy '(Shay). These in turn affect notions of authentic assessment. There are also regulatory bodies of various kinds. The broadest call might arise from 'neoliberal accountability and audit culture' (2).

However, assessment is a social practice undertaken by multiple actor groups and they will reveal mechanisms of contestation. There may be three basic paradigms — 'assessment of learning(AOL), assessment for learning AFL, and assessment as learning AAL', but these are used inconsistently, and there may be other principles such as learning oriented assessment or sustainable assessment, informed by 'socio-constructivist approaches '(Shay)

AOL is associated with certification and accreditation, summative assessment, while the others aligned with progressive transformative and formative assessment, although they also engender Instrumentalism and something called '"criteria compliance"' [apparently associated with Torrance 2007— teaching to the test?]. Forms are usually traditional such as examinations or essays. Other approaches tend to emphasise employability skills, authentic assessment. The arguments are often seen as dichotomies, and there are calls for paradigmatic shifts, often in the form of 'a formative – good – summative – bad sentiment' (3) These are not helpful.

They use LCT to reveal the organising principles of practice as legitimation codes, '"organising principles and practices, dispositions and contexts"' (Maton). They are interested in the language of legitimation, using the LCT dimensions of 'autonomy, semantics and specialisation'. This will help examine the autonomy of assessment at the macro level, logic at the meso level and successive assessment at the micro level, the level of assessment tasks. Each dimensional will generate legitimation codes which will be stronger or weaker. Together there are four code modalities as in table 1 [note that one of them is 'rhizomatic code', a matter of semantic gravity and semantic density.

They have generated a framework for an assessment enquiry, as a three-dimensional space enabling them to map specific practices within the wider space. They are interested in assessment in HE and they have gathered some empirical data.

This data comes in field work gathered as part of a doctoral project, semistructured interviews with 28 academics teaching accounting and business management in an elite university. They gathered data on assessment practice at the micro level and also assessment in a social context, the 'macro higher education landscape and the meso level of the institution and relevant disciplinary practices' (4). They did this via Zoom. They selected participants trying to get a representative sample in terms of gender, seniority and breadth of research and teaching activities. They ended with 46% female, 61% non-senior, length of experience range from 2 to 20 years, several disciplines were  represented, outward pull was particularly important for business schools.

In terms of autonomy, the issue is the insulation between assessment practices and its constituents, 'actors, ideas, artefacts' and the way they are related together for example in 'ways of working and beliefs' (5) it is possible to define positional autonomy (PA) as a measure of relations between constituents within context and those positioned in other contexts, and this can be used to understand boundaries between actors inside HE and those outside. universities are fairly autonomous in terms of the field of economic production and so tend to attract plus marks. Principles relating from outside HE provide weaker relational autonomy [but not weaker boundaries or influences?]: We can see 'the specificity or distinctiveness of knowledge practices'[but not what causes them in practice?]. The authors claim that we can go on to dissociate issues of power from knowledge practices after disaggregation, leading to a distinction between agency and control an assessment.

This will also lead to the notion of academic freedom. Strong relations between actors in HE so strong insulation from outside actors, autonomy. However we need 'an external language of description or a "translation device" (Maton and Howard 2018) to interpret' their comments. The intention is to describe not make moral judgements [translation is just the same old problem of interpreting empirical data to illustrate theoretical categories, examples are given in table 2]. There is a lovely bourdieuvian display as a result in figure 2 to show relative dimensions of educationally and non-educationally driven assessments

They detect 'sovereign codes' to describe academic assessments, referring to internal control and educational practices or principles, controlled by academics or disciplinary communities intending to provide academic education, liberal humanist ideas, traditional education, 'may align with learning for interest is derived from an inner academic habitus' (6) [illustrated with quotes]. 'Exotic codes' by contrast refer to imposed assessments, legitimate according to outside controls or influences and principles, for example those imposed by professional or other bodies or the market, where assessment is seen as guaranteeing some return for investment. 'Introjected codes'can also be seen as 'cooperative assessments' influenced from outside, for example industry  but also with educational significance such as skills development — assessment is influenced by real-life. 'Projected codes' are managed assessments relating to internal influences but oriented to outside purposes, including pragmatic ones, like self-preservation — examples here include grade inflation or indications of efficiency [nice example in  the quotes about giving everybody 60s and 70s to avoid remarking failures (7)]

For semantics, there is density and gravity, to understand the relations between theory and practice all the conceptual and contextual. The greater the conceptual coherence the greater the semantic density, the condensation of meaning within symbols, the integration of concepts. Semantic gravity refers to 'contextual coherence', the relation of meaning tricks context, such as practice or the workplace.  Assessment cannot be both. They define gravity as the extent to which assessments are relevant to the real world or are rooted in practice, including embedded in specific situations cases are examples. Nice tables again.

Semantic density can be interpreted in terms of language and task, whether there is disciplinary language or whether it relies on everyday interpretations, whether the assessment is integrated and holistic, whether it is disparate and discreet. At last 'rhizomatic codes' demand abstraction and complexity, focusing on theoretical disciplinary knowledge and complex theory that spans modules and syllabi, with 'densely constructed concepts' (9). There is no application to real life necessarily. Traditional modes emphasising the written word are the most common means of achieving such theoretical pursuits [emphasised in a quote]. 'Prosaic codes' are the opposite found in practical assignments that are concrete less dense, based on practical knowledge free of formal theories and legitimated according to alignment with actual practices.  'The worldly code' is found in professional assessments, concrete yet still academically applicable complex, as in problem-based learning. 'Rarefied codes' are generic assessments not tied to context but focused on 'soft, generic skills or basic everyday terminology' such as communication and teamwork skills. Together there can be diversity.

Specialisation focuses on what is legitimate knowledge and who is a legitimate knower, epistemic and social dimensions which may not work together. They form the basis of what is valued and maybe contested for example in claims that social constructivist knowledges have replaced disciplinary ones.

These issues are often represented in assessment rubrics. Underperformance may be down to 'a lack of transparency and tacit understanding (O'Donovan, Price and Rust 2004), obfuscated by low levels of assessment literacy' (10) but this can also be seen as a matter of different languages of legitimation code clashes, where students focus on, say knowledge and teachers emphasise social relations [more likely the other way about]. Assessments can either value the dispositions and attributes of the actor, or of the knowledge.

'Knowledge codes' refer to content, knowledge of principles and procedures rather than the attributes of the individual. 'Knower codes' often refer to the individual and their dispositions or attributes and may lead to 'dialogic assessments' (11) and socio-constructivist perspectives, student reflections rather than experts.

Those that refer to elite codes are called 'genuine assessments' which 'valorise both knowledge and knowing'. There are also mixed forms where assessment can be designed, for example in the co-construction of assessment criteria to 'allow for no engagement with knowledge', and the example is 'continuous reflective blogs that complement the focus on accumulation of specialised knowledge and personal reflection'.

'Relativist codes' lead to 'performative assessments' which emphasise neither knowledge nor individuals and '"anything goes" and anyone can learn, and trainability is the issue, although suspicions of spoon feeding and rewarding attendance also lurk here.

Together, it's possible to map an entire framework [and that's what they do, coding interview extracts as they go. They do not try to divide things into extreme positions and they want to emphasise complexities].

They are jolly pleased with the results, and think that their dimensions can help identify the influences on assessment, such as autonomy vs profane concepts, or competing paradigms autonomy, different semantic dimensions of assessment, and combinations of specialisations. Overall, it shows the role of theory and assessment research and produces productive dialogue and reconciles polarised positions [not really  — those is a place for everything]. It does show struggle between educators themselves and for others including students, who might be helped by being introduced to the 'competing languages of legitimation' (14).