Notes on: forde-leaves, N., Walton, J.,
Tann, K. (2023) A framework for
understanding assessment practice in
higher education. Assessment and
Evaluationin Higher Educuation. 48 (8)
1076--91. DOI:
10.1080/02602938.2023.2169659
Dave Harris
Conflicts and assessment arise from
conflicting demands on the curriculum and
there are three terms in assessment
discourse such as 'autonomy, logic and the
basis of success'. They apply LCT to
develop a new holistic framework which
will analyse assessment practice cross
discipline institution and geographical
boundaries.
There are different purposes of HE
globally, and segmentation. There have
been calls to rethink assessment theory
and practice, although relatively little
discussion about generative mechanisms or
conceptual bases, a general under
theorising. It is still a 'cottage
industry… Attempts to enhance assessment
practice built on sand' (1076), with
'empirical saturation'. We need meta
theories to get at principles. LCT as in
Maton will expose dichotomies and trace
them in assessment practices and lead to
'socially contextualised idealised
positions' (1077).
Assessment practices serve competing
purposes — certification, progress,
transfer, accountability, supporting
learning. These reflect 'an outward pull'
from external constituencies which affect
university autonomy '(Shay). These in turn
affect notions of authentic assessment.
There are also regulatory bodies of
various kinds. The broadest call might
arise from 'neoliberal accountability and
audit culture' (2).
However, assessment is a social practice
undertaken by multiple actor groups and
they will reveal mechanisms of
contestation. There may be three basic
paradigms — 'assessment of learning(AOL),
assessment for learning AFL, and
assessment as learning AAL', but these are
used inconsistently, and there may be
other principles such as learning oriented
assessment or sustainable assessment,
informed by 'socio-constructivist
approaches '(Shay)
AOL is associated with certification and
accreditation, summative assessment, while
the others aligned with progressive
transformative and formative assessment,
although they also engender
Instrumentalism and something called
'"criteria compliance"' [apparently
associated with Torrance 2007— teaching to
the test?]. Forms are usually traditional
such as examinations or essays. Other
approaches tend to emphasise employability
skills, authentic assessment. The
arguments are often seen as dichotomies,
and there are calls for paradigmatic
shifts, often in the form of 'a formative
– good – summative – bad sentiment' (3)
These are not helpful.
They use LCT to reveal the organising
principles of practice as legitimation
codes, '"organising principles and
practices, dispositions and contexts"'
(Maton). They are interested in the
language of legitimation, using the LCT
dimensions of 'autonomy, semantics and
specialisation'. This will help examine
the autonomy of assessment at the macro
level, logic at the meso level and
successive assessment at the micro level,
the level of assessment tasks. Each
dimensional will generate legitimation
codes which will be stronger or weaker.
Together there are four code modalities as
in table 1 [note that one of them is
'rhizomatic code', a matter of semantic
gravity and semantic density.
They have generated a framework for an
assessment enquiry, as a three-dimensional
space enabling them to map specific
practices within the wider space. They are
interested in assessment in HE and they
have gathered some empirical data.
This data comes in field work gathered as
part of a doctoral project, semistructured
interviews with 28 academics teaching
accounting and business management in an
elite university. They gathered data on
assessment practice at the micro level and
also assessment in a social context, the
'macro higher education landscape and the
meso level of the institution and relevant
disciplinary practices' (4). They did this
via Zoom. They selected participants
trying to get a representative sample in
terms of gender, seniority and breadth of
research and teaching activities. They
ended with 46% female, 61% non-senior,
length of experience range from 2 to 20
years, several disciplines were
represented, outward pull was particularly
important for business schools.
In terms of autonomy, the
issue is the insulation between assessment
practices and its constituents, 'actors,
ideas, artefacts' and the way they are
related together for example in 'ways of
working and beliefs' (5) it is possible to
define positional autonomy (PA) as a
measure of relations between constituents
within context and those positioned in
other contexts, and this can be used to
understand boundaries between actors
inside HE and those outside. universities
are fairly autonomous in terms of the
field of economic production and so tend
to attract plus marks. Principles relating
from outside HE provide weaker relational
autonomy [but not weaker boundaries or
influences?]: We can see 'the specificity
or distinctiveness of knowledge
practices'[but not what causes them in
practice?]. The authors claim that we can
go on to dissociate issues of power from
knowledge practices after disaggregation,
leading to a distinction between agency
and control an assessment.
This will also lead to the notion of
academic freedom. Strong relations between
actors in HE so strong insulation from
outside actors, autonomy. However we need
'an external language of description or a
"translation device" (Maton and Howard
2018) to interpret' their comments. The
intention is to describe not make moral
judgements [translation is just the same
old problem of interpreting empirical data
to illustrate theoretical categories,
examples are given in table 2]. There is a
lovely bourdieuvian display as a result in
figure 2 to show relative dimensions of
educationally and non-educationally driven
assessments
They detect 'sovereign codes' to describe
academic assessments, referring to
internal control and educational practices
or principles, controlled by academics or
disciplinary communities intending to
provide academic education, liberal
humanist ideas, traditional education,
'may align with learning for interest is
derived from an inner academic habitus'
(6) [illustrated with quotes]. 'Exotic
codes' by contrast refer to imposed
assessments, legitimate according to
outside controls or influences and
principles, for example those imposed by
professional or other bodies or the
market, where assessment is seen as
guaranteeing some return for investment.
'Introjected codes'can also be seen as
'cooperative assessments' influenced from
outside, for example industry but
also with educational significance such as
skills development — assessment is
influenced by real-life. 'Projected codes'
are managed assessments relating to
internal influences but oriented to
outside purposes, including pragmatic
ones, like self-preservation — examples
here include grade inflation or
indications of efficiency [nice example
in the quotes about giving everybody
60s and 70s to avoid remarking failures
(7)]
For semantics, there is density
and gravity, to understand the relations
between theory and practice all the
conceptual and contextual. The greater the
conceptual coherence the greater the
semantic density, the condensation of
meaning within symbols, the integration of
concepts. Semantic gravity refers to
'contextual coherence', the relation of
meaning tricks context, such as practice
or the workplace. Assessment cannot
be both. They define gravity as the extent
to which assessments are relevant to the
real world or are rooted in practice,
including embedded in specific situations
cases are examples. Nice tables again.
Semantic density can be interpreted in
terms of language and task, whether there
is disciplinary language or whether it
relies on everyday interpretations,
whether the assessment is integrated and
holistic, whether it is disparate and
discreet. At last 'rhizomatic codes'
demand abstraction and complexity,
focusing on theoretical disciplinary
knowledge and complex theory that spans
modules and syllabi, with 'densely
constructed concepts' (9). There is no
application to real life necessarily.
Traditional modes emphasising the written
word are the most common means of
achieving such theoretical pursuits
[emphasised in a quote]. 'Prosaic codes'
are the opposite found in practical
assignments that are concrete less dense,
based on practical knowledge free of
formal theories and legitimated according
to alignment with actual practices.
'The worldly code' is found in
professional assessments, concrete yet
still academically applicable complex, as
in problem-based learning. 'Rarefied
codes' are generic assessments not tied to
context but focused on 'soft, generic
skills or basic everyday terminology' such
as communication and teamwork skills.
Together there can be diversity.
Specialisation focuses on what is
legitimate knowledge and who is a
legitimate knower, epistemic and social
dimensions which may not work together.
They form the basis of what is valued and
maybe contested for example in claims that
social constructivist knowledges have
replaced disciplinary ones.
These issues are often represented in
assessment rubrics. Underperformance may
be down to 'a lack of transparency and
tacit understanding (O'Donovan, Price and
Rust 2004), obfuscated by low levels of
assessment literacy' (10) but this can
also be seen as a matter of different
languages of legitimation code clashes,
where students focus on, say knowledge and
teachers emphasise social relations [more
likely the other way about]. Assessments
can either value the dispositions and
attributes of the actor, or of the
knowledge.
'Knowledge codes' refer to content,
knowledge of principles and procedures
rather than the attributes of the
individual. 'Knower codes' often refer to
the individual and their dispositions or
attributes and may lead to 'dialogic
assessments' (11) and socio-constructivist
perspectives, student reflections rather
than experts.
Those that refer to elite codes are called
'genuine assessments' which 'valorise both
knowledge and knowing'. There are also
mixed forms where assessment can be
designed, for example in the
co-construction of assessment criteria to
'allow for no engagement with knowledge',
and the example is 'continuous reflective
blogs that complement the focus on
accumulation of specialised knowledge and
personal reflection'.
'Relativist codes' lead to 'performative
assessments' which emphasise neither
knowledge nor individuals and '"anything
goes" and anyone can learn, and
trainability is the issue, although
suspicions of spoon feeding and rewarding
attendance also lurk here.
Together, it's possible to map an entire
framework [and that's what they do, coding
interview extracts as they go. They do not
try to divide things into extreme
positions and they want to emphasise
complexities].
They are jolly pleased with the results,
and think that their dimensions can help
identify the influences on assessment,
such as autonomy vs profane concepts, or
competing paradigms autonomy, different
semantic dimensions of assessment, and
combinations of specialisations. Overall,
it shows the role of theory and assessment
research and produces productive dialogue
and reconciles polarised positions [not
really — those is a place for
everything]. It does show struggle between
educators themselves and for others
including students, who might be helped by
being introduced to the 'competing
languages of legitimation' (14).
|
|
|