IAN GILHESPY
University College of St. Mark and St. John
Derriford Road
Plymouth
England
PL6 8BH
E-mail: Igilhespy@marjon.ac.uk
 

Please note that this paper is a draft version of an article recently published in the International Journal of Arts Management (Fall, 1999). The final version is copyrighted to the journal.
 

Introduction

This paper is one of the outcomes of a research project which was designed in order to investigate  whether  methods of performance measurement are appropriate and sensitive mechanisms  to evaluate the performance of arts and cultural organisations.

The principal aim of this paper is to present some of the conclusions of a research project, specifically to present a  model of a performance measurement system for application in cultural organisations.  The contention of the paper is that the objectives of cultural organisations  may be rendered down into a number of strategic options that may be pursued in relation to one another. And further that the achievement of these strategic  options may be  measured with a degree of sensitivity.
Given the confines of the presentation of a paper it is not possible to report on the research process itself. Suffice to say that the performance measures presented in the paper have been tested( except where clearly stated). The derivation of the measures and of the constituent parts of the policy matrices  are to be the subject of a future publication.  This current paper focuses on some of the conclusions of the research.

The research project was supported by Southern Arts, the regional arts board for the South of England and included literature reviews, large-scale quantitative research using a postal questionnaire into the economic importance and impact of the arts, small-scale qualitative research using a semi-structured interview schedule with key professionals in the management of arts organisations in the South of England and  further  quantitative research into the rationales of their organisations and further quantitative research with the same key professionals into the development and application of specific measures of performance of cultural organisations.

Context
Lane (1993) has written that the rapid expansion of the public sector in the 1960s and 1970s implied a basic change in the relationship between the public and the private spheres. This rapid expansion meant that ' a considerable portion of the resources of society is mobilised by national or local governments and allocated by these bodies to various programmes, thus reducing the scope for markets'(Lane,1993, pp190). Following this period, the  1980s and  1990s have witnessed  a period of resurgence for economic liberalism  in Britain and elsewhere in the West with a strong political commitment to 'markets and competitive individualism' (Hutton,1995).

This period  has seen the re-examination of some of the functions of the state(Knox et al:1992).  This re-eamination is not of the relative merits of a planned economy contra a market economy but, first,  of how to strike a balance between the public and private sectors, or as Lane(1993, pp190) puts it ' what types of goods and services are to be allocated by means of the budget and through markets....and are there possibilities for mixing public and private modes of interaction'?  Second, this re-examination concerned the evaluation of public sector programmes.  It is the latter element of this re-examination which is the main focus of this paper.

This re-examination produced government policy  to improve methods of accountability  in public expenditure(H.M. Treasury, 1983).  The basic elements of these methods were that the objectives of any activity in public life should be identified and defined and that appropriate means should be developed by which the achievement of these objectives could be measured.  This type of scrutiny is known as performance measurement.

The reason put forward for monitoring or measuring the performance of an organisation which receives public money  is to provide information about whether the organisation is achieving what it is supposed to achieve(H.M.Treasury,1983).  To this end it became government policy that monitoring systems ( i.e. sets of performance measures) should be developed that are appropriate in the  evaluation of the achievement of an organisations  objectives(Audit Commission:1986).  Monitoring, therefore, was developed as a process which provided data with which  evaluation could be made. The output of the monitoring processes,  as developed in the early  1980s (Carter:1991),  was an analysis  of the success of the organisation in the fulfilment of its' objectives. Importantly, in the early 1980s objectives were to be defined by the providers rather than the consumers, a point I shall discuss later in the paper.

In Britain, with  the influence of the National Audit Commission  and the Chartered Institute of Public Finance and Accountability (C.I.P.F.A.), the triumvirate of economy, efficiency and effectiveness of service - known as the Three 'E's-  have become  regarded as the fundamentals of monitoring(Jackson and Palmer:1988). These terms are very significant in this paper to the extent that the  rather specific meanings they entertain require some elucidation.

First, I shall define efficiency.  Efficiency, broadly,  has two interrelated uses in this paper.  The first use of the term efficiency is a technical one drawn from the literature on performance measurement and  considers the best possible relationships between inputs and outputs( Gratton and Taylor,1992).  This use of the term states that  efficiency is  output divided by resources consumed or, alternatively, that  efficiency is a measure of unit cost - the cost of inputs divided by the output (Jackson and Palmer,1988).

The second of these uses has been neatly summarised by Towse (1993,pp.3) as follows:

The social efficiency case for intervention in the market is basically that free market forces cannot work to provide the socially optimum outcome.

In the course of my research I found it necessary to introduce a fourth 'E' into the analysis, 'E' for equity.
The term equity in this paper is mainly about the distribution of opportunities to  benefit from artistic activity. To a lesser extent it is also about the distribution of who bears the costs.  This distribution of opportunities takes a number of forms.  The first dimension of distribution is social ( intersocial equity). This dimension is concerned with the distribution of opportunities amongst different socio-economic groups, between the genders and amongst ethnic groupings.  This is in part a question of access, that is to say, availability of opportunities to partake in the arts.  But it is also a question of educational and cultural capital. The sociologist Pierre Bourdieu (1985) has written extensively on this. The second dimension is regional ( interspatial equity).  Formulae exist that are designed to ensure that health and educational provision are equal across the country ( Towse, 1993). No such formula exists for the provision of the arts because no such policy exists for the arts(Evans,1995). In fact, it has been policy to concentrate activities into certain areas(Mulgan and Worpole,1986).  There is a third dimension of equity noted  by  Bennett(1992) which is to do with equity over time(intergenerational equity).  This dimension of equity has relevance to the arts in so far as one of the reasons for supporting artistic and cultural activities with public money is so that  they can be enjoyed by future generations.  This could be called cultural sustainability. I shall use the terms outlined above in relation to the specific performance indicators that appear later in the paper.

Models and Modelling
A model may be understood as  a simplified version of reality (Wilson et al,1992).  The model proposed in this paper is a model for the evaluation of the performance of cultural organisations.  The design of the model involves the  invention and application of a common rating system  for cultural organisations where sets of objectives - artistic, social, financial, externalities for instance - must be synthesised into one system of measurement.

The model is intended to be one that has a number of attributes. The model should :

i) incorporate the measurement of a wide range of strategic objectives;
ii) be  sensitive to  conflicts in the statement of strategic objectives with  the use of  weighting  methods;
iii) combine elements of economy, efficiency and effectiveness(including  equity);
iv) be testable.

In the second instance the design of the model is to take account of the  expectations for performance that  external stakeholder groups may entertain (including those that may not have been consulted in the formulation of policy of specific cultural organisations). The external stakeholder groups in focus are the government agencies involved in funding and accountability procedures.

A critical assessment of the model  using the criteria of parsimony, useability, sensitivity and appropriateness is to be the subject of a separate paper.

Strategy and Cultural Organisations
Cultural organisations make choices.  It was found in the research that these choices are made in a variety of ways.  For some the choices are  strategic in so far as they are made in relation to formal policy ( i.e. with mission statements, for instance, and/or statements of objectives) which may or may not have been made with negotiation with other stakeholders.  For other organisations in the study the choices were not strategic but guided by such factors as inspiration, intuition, common sense and circumstance.

The selection of the base data for the model needs to be made in accordance with the  choices available to cultural organisations( whether these choices are explicit at the level of policy or at the level of practice). This includes the choices that are less amenable to quantitative measurement as well as those that are more amenable.  The selection of the base data, therefore, will become a matrix of choices available to cultural organisations.  This matrix is a policy matrix .  The initial part of this discussion will focus on the strategic choices available to the managers of cultural organisations. Other stakeholder groups will be considered later in the paper.

Strategic Choices for Management
In Table 1 there is a policy matrix. The choice of policy objectives that make up the matrix is a parsimonious one.  The matrix could be a very long one. The composition of the matrix  is, nevertheless,  intended to be one that represents the range of policy objectives  available to the managers of cultural organisations.

Later in the paper a similar policy matrix is presented for external stakeholders  but one which includes prestige, quality of life,  economic impact and employment impact.

The policy objectives are shown in alphabetical order and will be discussed in that order.
_____________________________________________
Table 1 A Policy Matrix  For Managers - The 'Top Ten'
_____________________________________________
Policy Objective
Access Maximisation
Attendance Maximisation
Diversity/Multiculturalism
Economy Maximisation
Education
Excellence
Innovation
Revenue Maximisation
Service Quality Maximisation
Social Cohesion
_____________________________________________
The policy objectives featured in Table 1 will be discussed in turn each with  a table including  lists of  performance indicators  that may be of potential use in the evaluation of the achievement of each of the policy objectives.  The performance indicators may be used to assist the evaluation. They are are not the evaluation itself. This includes those policy objectives that are more amenable to quantitative analysis than others.

Access Maximisation
Access emerged in the course of the research as a key social objective for arts organisations meaning  - either or both - access for everyone to the arts and access for those social groups assumed to be less well represented than other groups in the audiences/visitors. The implementation of a concessionary pricing structure is one means of operationalising this objective although   this may be more to do with a welfare framework of provision than an analysis of visitor patterns(Evans,1995)and willingness to pay (Stabler,1994).  In Britain there have been policies for cultural and leisure provision that emphasise the need for opportunities  for those social groups which are financially less able to provide for themselves, usually students, the unemployed and the elderly(Coalter,1988).  Interspatial - as opposed to intersocial - access may apply if the organisation wishes to serve either its' local or regional population specifically.

The programming of forms of outreach work(using the term broadly to mean work in prisons, retirement homes, rural areas  or anywhere that is not the usual site of the organisation)  is another means of putting this objective into action as a means of reaching 'new users'. It is a policy which fits the social efficiency argument introduced earlier very well. Performance indicators for outreach work in terms of access are shown in Table 2.  The possible outcomes of outreach work are discussed under  'Social Cohesion'.

There may be other means of promoting the attendance of new users which lie outside of the artistic programme such as in advertising or service quality.

The commitment of resources to the pursuit of access maximisation is a strategic choice which, ipso facto, means those resources are not spent on the pursuit of other strategic choices such as  revenue maximisation or attendance maximisation. Strategic choices may be in conflict although some will complement one another. It will be true of the 'merit good' objectives that revenue maximisation is curtailed and, probably, attendance maximisation as well.  The term merit good  denotes products, services and opportunities which are meritorious in their  own right and need to be supplied( by governments, usually) even if as Cwi(1982) has noted this 'merit' has different connotations in different countries according to the degree of consensus about the extent to which the arts are one of the 'necessary ingredients of a good society'( Robbins,1963,p.58 quoted by Austen-Smith,1994). This may be expressed by an estimate of lost revenue  and lost attendance.  Likewise  promotion expenditure for new users may be in conflict with economy maximisation.
 _____________________________________________
Table 2. Access Maximisation
_____________________________________________
 

Performance Indicator	What is being  measured?
O.A.P./Total Attendance	Effectiveness/Equity
Students/Total Attendance	Effectiveness/Equity
Unemployed/Total Attendance	Effectiveness/Equity
Children/Total Attendance	Effectiveness/Equity
Concessions/Total Attendance	Effectiveness/Equity
New Visitors/Total Attendance	Effectiveness/Equity
(Using Social Surveys)
Outreach Costs/Programme Costs	Efficiency
Outreach Costs/G.O.E.`	Efficieny
Outreach Attendance*/Total  Attendance	Effectiveness/Equity
Local Users/T.A.^	Effectiveness/Equity
RegionalUsers/T.A.	Effectiveness/Equity
Estimate of Lost Revenue	Effectiveness/Equity
Estimate of Lost Attendance	Effectiveness/Equity

_____________________________________________
`G.O.E. = Gross Operating Expenditure T.A.^ = Total Attendance
*If appropriate
_____________________________________________
Attendance Maximisation
Attendance maximisation is something of an unwritten policy objective.  By this I mean that it may be assumed that attendance maximisation is a goal until it becomes contrary to other strategic objectives including not only those objectives that relate to  merit good  and market failure ( in this instance market failure relates back to the definition of social efficiency made earlier where free markets do not provide the socially optimal outcome)  but also revenue maximisation, service quality maximisation  and economy  maximisation in certain circumstances. Attendance maximisation can be measured against targets or over time. This is discussed after each of the policy objectives has been covered.
_____________________________________________
Table 3.Attendance Maximisation
_____________________________________________
 
 

Performance Indicator	What is being  measured?
Total Attendance	Effectiveness
Total Attendance/G.O.E.*	Efficiency
Subscription/T.A.	Efficiency/Effectiveness
T.A./Capacity	Efficiency
Expected Attendance/Actual Attendance	Economy

_____________________________________________
Diversity/Multiculturalism
The policy objective of diversity in the research data related to both the range of artistic activities and, perhaps more importantly, to the provision of cultural opportunities which would not be supplied either by commercial producers of culture -  therefore relating to  the 'market failure' argument defined earlier -  or by the traditional funded arts sector, the 'elite' arts.

The policy objective of diversity   means   giving a cultural voice to those social groups who would otherwise go unheard such as the disabled, for example, and to forms of participatory arts activity  where the boundaries of production and consumption would be blurred.

As an artistic objective multiculturalism may be regarded as a particular form of diversity but with, perhaps,  the more explicitly political motive of providing   representation to  ethnic minorities for cultural expression.

The social benefits of multiculturalism are discussed under social cohesion.

There are limits to the desirability of diversity in the artistic programme.  Exceeding those limits  may  lead to a lack of focus for the organisation and/or a failure to pursue artistic quality in chosen activities.  Thus, there may be a strategic trade-off between diversity and quality/excellence.

This strategic choice, as with other artistic choices, may involve a reduction in attendance and revenue as well as a decrease in expenditure on  other artistic and social objectives.

It may be possible to estimate loss of attendance and revenue due to the pursuit of diversity.
_____________________________________________
Table 4.Diversity/Multiculturalism
_____________________________________________
 
 

Performance Indicator	What is being  measured?
Range of Programme	Effectiveness
Expenditure on Multiculturalism/Programme Costs	Effectiveness
Expenditure on Multiculturalism/Gross Operating Expenditure	Effectiveness
Estimate of Lost Revenue	Effectiveness
Estimate of Lost Attendance	Effectiveness

_____________________________________________
Economy Maximisation
Economy maximisation has the status as a policy objective rather like attendance maximisation.  It should be assumed that organisations wish to pursue this objective unless there are reasons for not doing so i.e. if it is  contrary to other objectives.
_____________________________________________
Table 5.Economy Maximisation
_____________________________________________
 

Performance Indicator	What is being  measured?
Publicity Costs/Turnover	Economy
Premises  Costs/Turnover	Economy
Admin.Costs/Turnover	Economy
Programme Costs/Turnover	Economy
Salaries/Gross Expenditure	Economy
Planned Costs/Actual Costs	Economy

_______________________________
Education
The term education as a policy objective has a wide range of meanings and the application of the term is at the discretion of the management of the centre.  Education may include the provision of classes, workshops and possibly outreach provision.  The educational dimension of the work of some cultural organisations  may also go beyond these formal confines into the development of life skills and social confidence more generally, for instance, in the operation of user panels.   Education is about the re-distribution of cultural capital which, it is assumed, is distributed unequally. Thus, education as a policy objective relates to citizenship where citizenship is defined as:

...that set of practices( juridical, political, economic, and cultural) which define a person as a competent member of society, and which as a consequence shape the flow of resources to persons or social groups.(Turner, 1993, pp.2)

Education is also about the creation of future audiences for the arts although the benefits of this may not be returned to the organisation doing the investment.  Education is also the rationale  behind the conservation activities of some cultural organisations, a form of cultural sustainability  where existing cultural opportunities are maintained for future generations.  This may be referred to as the pursuit of intergenerational equity.
_____________________________________________
Table 6.Education
_____________________________________________
 
 

Performance Indicator	What is being  measured?
Educational Expenditure/Programme Costs	Effectiveness
Educational Expenditure/ Gross Operating Expenditure	Effectiveness
Outreach Costs/Programme Costs	Efficiency
Outreach Costs/G.O.E.`	Efficiency
Estimate of Lost Revenue	Effectiveness
Estimate of Lost Attendance	Effectiveness

 _____________________________________________
Excellence/Quality
The setting and assessing of standards of excellence/quality are at this stage - I am concerned at the present with a model for application by arts managers - to be made by arts professionals themselves.  This is to be an intersubjective process i.e. one to be negotiated amongst the management team. This negotiation could be informed by calculations of any possible loss of revenue and attendance  engendered by the programming of artistic activity which is intended to be of high quality.
_____________________________________________
Table 7.Excellence/Quality
_____________________________________________
 

Performance Indicator	What is being  measured?
Estimate of Lost Revenue	Effectiveness
Estimate of Lost Attendance	Effectiveness
Quality Ratings (Using Social Surveys)	Effectiveness
Willingness to Pay Rating (Using Social Surveys)	Effectiveness

_____________________________________________
Innovation
The assessment of the provision for opportunities for innovation is - rather like the policy objective of excellence  - to be made by the arts managers themselves.   The policy objective of innovation  has similarities to that of conservation in so far as the beneficiaries may be future generations.  Innovation is to cultural organisations what research and development is to a number of commercial organisations, an investment for which the dividend is not known.  However, the cultural organisation differs from its commercial counterpart in that the dividend, if paid, may not be paid back to the organisation that made the investment.

The artistic benefits of innovation - and possible social benefits - are incalculable as they can only become known in the uncertain future.  This means that although the immediate cost of innovation may be estimated it  is impossible to provide complete information on all the possible benefits of artistic activity.  The future demand conditions for cultural activity may not be predictable with any degree of certainty.
_____________________________________________
Table 8. Innovation
_____________________________________________
 

Performance Indicator	What is being  measured?
Estimate of Lost Revenue	Effectiveness
Estimate of Lost Attendance	Effectiveness
New companies in Yearly Programme	Effectiveness

_____________________________________________
Revenue Maximisation
The policy objective of revenue maximisation is a strategic choice made relative to other policy objectives. By this I mean that it should be assumed of an organisation - rather like attendance and economy maximisation.  This policy incorporates  all forms of revenue including business and public support.
_____________________________________________
Table 9. Revenue Maximisation
_____________________________________________
 

Performance Indicator	What is being  measured?
Direct Income/G.O.E	Economy
Box Office Income/Turnover	Economy
Sales Revenue/Turnover	Economy
Business Support/Turnover	Economy
Box Office Income/T.A	Efficiency
Sales Revenue/T.A.	Efficiency
Administration Costs/T.A	Efficiency
Gross Expenditure/T.A.	Efficiency

_____________________________________________
Service Quality Maximisation
It was noted in the data analysis  for this research that the provision of cultural opportunity is the provision of a complex good.  The service quality features of this complexity were highlighted.  In the first instance it would be for managers to estimate the achievement of this policy objective.

Service quality features  may be amenable to analysis using  surveys of customer satisfaction.

In Table 10 I have rendered a potentially long list into four key features with examples of each.

A second form of performance indicator for service quality - like artistic quality - would be the willingness of the customer to pay more for an increase in quality.
_____________________________________________
Table 10. Service Quality Maximisation
_____________________________________________
 

Performance Indicator	What is being  measured?
Quality of Building Design: Foyer; Bars; Restaurant; Car Parks; Seating; Sightlines.    Effectiveness Quality of Sales Products: Food; Drink; Shop Sales.	Effectiveness
Quality of Service Elements: Efficiency of Staff; Friendliness of Staff; Friendliness of Volunteers; Cleanliness of Building	Effectiveness
General Features: Atmosphere Sense of Community	Effectiveness
Willingness to Pay Rating (Using Social Surveys)	Effectiveness

_____________________________________________
Social Cohesion
Social cohesion is one of the more intangible policy objectives that the management of a cultural organisation may choose to pursue. It is intended that the policy objective incorporates several elements:  the objective of community is one element - although notoriously difficult to define-  and that of multiculturalism(in this instance multiculturalism as a social rather than artistic objective).   It is a policy objective that relates to the sense of 'publicness'  -  that is a sense of collectivity and involvement to counter feelings of atomisation and individuation -   to cultural democracy and to the encouragement of understanding amongst a range of social groups.
_____________________________________________
Table 11 Social Cohesion
_____________________________________________
 

Performance Indicator	What is being  measured?
Expenditure on community-oriented activities/Gross Expenditure	Efficiency
Attendance of community-oriented activities/Total Attendance	Effectiveness/Equity
Outreach Expenditure/Gross Expenditure	Efficiency
Outreach Attendance/Total  Attendance	Effectiveness/Equity
Attendance by Ethnic Minorities/Total Attendance (Using Social Surveys)	Effectiveness/Equity
O.A.P./Total Attendance	Effectiveness/Equity
Students/Total Attendance	Effectiveness/Equity
Unemployed/Total Attendance	Effectiveness/Equity
Children/Total Attendance	Effectiveness/Equity
Concessions/Total Attendanc	Effectiveness/Equity
Attendance of User Panels	Effectiveness/Equity
Number of Volunteers	Effectiveness/Equity

_____________________________________________
I shall now go on to examine the  question  of whether it is possible to  put together a common rating system which may synthesise  the values of all of the policy objectives
.
The Weighting of Strategic Choices
Two features of the policy matrix need to be noted:

i) the strategic choices available may be in conflict with each other;
ii)  the strategic choices have a relative importance to each other.

Therefore,iii) the strategic choices will require weighting.

The relative importance of the policy objectives within the plan may be represented in the form of a weighting.   The weighting  is a sort of numerical proxy for the significance that the management of a centre put on their strategic choices.
The relative importance that individual organisations assign to the policy choices available to them is likely to be a complex one. If the organisation has formulated a policy document then the policy choices may be relatively clear. The assignment of weightings would require considerable  judgement. The weightings should be expressed as decimal points. The decimal points need to add up to one.  This is shown in Table 12.

Using the performance indicators for guidance the management of a centre would then award a score for the achievement of each of the policy objectives.  This would  again be done with a numerical proxy.  The scores would then be recalculated  according to their weighting.

The score would then be compared against a target score and, in due time, against past performance. The score would be considered in terms of the effects that the external environment has had  on the organisation in the relevant period including those of competitor organisations( if appropriate). This is part of the process of strategic management on which there is not space in this paper to elaborate.
_____________________________________________
Table 12.  The Weighting Game
_____________________________________________
 

Policy Objective	Weighting	Score	Weighted Score
Access Maximisation	x	y	xy
Attendance Maximisation	x	y	xy
Economy Maximisation	x	y	xy
Diversity	x	y	xy
Education	x	y	xy
Excellence	x	y	xy
Innovation	x	y	xy
Revenue Maximisation	x	y	xy
Service Quality Maxim	x	y	xy
Social Cohesion	x	y	xy

 

_____________________________________________ Total 1.00 - xy(total) _____________________________________________ Compare total 'xy' to 'a'(target score). Consider in relation to  'b'(uncontrollable factors). _____________________________________________ 'x','y' and'xy' The discussion of  'x','y' and'xy' rests on one key  assumption drawn from economics. This is the assumption that the purpose of public support(i.e.funding) is to alter the  behaviour of an organisation(Towse,1993), not merely subsidise existing behaviour.   The relevant concept here is that of additionality(Evans,1995). The meaning of additionality is the that output of an organisation will be changed with the provision of public support and may be understood as a sort of surplus. This surplus would not be produced without the public support. In Table 12 the 'x' factor relates to the inputs of the cultural organisation. The allocation of inputs is made according to the policy objectives of the organisation.   The allocation of inputs is set by the strategic choices of the organisation(  controllable factors ).  The estimation of 'x' is at the discretion of the management of the organisation.  To assist the calculation of the 'x' factor the management of an organisation may wish to estimate any loss in revenue incurred in the pursuit of its' policy objectives.  The estimation of revenue loss is not a precise process but one that depends on the professional management of  organisations and their understanding of the demand conditions for the cultural activities in their area.  The estimation of the  loss in revenue could assist in the calculation of the relative importance  of the policy objectives in relation to one another.  This process of estimation may be less problematical  for those organisations  which are trading organisations  as opposed to those organisations with a tradition of free entry where there may be no index of cost recovery available. There may, of course, be a number of performance centres within the organisation  with distinct responsibilities  for the pursuit of different objectives.  The 'y' factor relates to the outputs of the cultural organisation. The outputs are the benefits for the consumers(the public) receive.  The calculation of 'y' is also an estimation.    The 'y' factor - or score - is  an estimation of the achievement of each of the policy objectives that the organisation has chosen to pursue.  The score is an estimation - rather than a precise figure -  as a result of the intangibility of some of the policy objectives of cultural organisations, that  performance indicators do not measure the outputs precisely and because of the effects of the external environment( the uncontrollable factors the effects of which a rational organisation will try to limit).  The 'xy' calculation is an expression of the relationship between inputs and outputs.    The total figure 'xy'  is an estimation of the achievement of an organisation in relation to its' objectives.  The  'xy' estimations would then be compared to  target figures that were set by the organisation including the total figure. The differences between the total figures and the target figures would be considered both in terms of the impact of the external environment and the performance of management itself in the achievement of its objectives.  The efficiency of management in achieving its objectives is assessed.  A model of a performance measurement system may be represented in the form of a series of stages and as a flow diagram.  _____________________________________________ Table 13.A Model of a Performance Measurement System for  Application in Cultural Organisations (in Stages) _____________________________________________ 1. Definition of Purpose: Establish mission for organisation. This should be done in negotiation with  relevant external stakeholders. 2. Setting the Objectives: Determine policy objectives. Again, this should be done in negotiation with relevant external stakeholders.  3. Environmental Analysis: Identify possible constraints on achieving objectives. A form of risk assessment in which conditions which are beyond the decision-makers control are recognised including the possible behaviour of competitors. 4. Strategic Choice(i): Make strategic choices. Choices are made to achieve the policy objectives that have been set  and weighed(the 'x' factor) in the policy matrix. 5. Strategic Choice(ii): Choose indicators for measurement of performance. Relevant performance indicators(dials) are decided upon which will measure the achievement of the policy objectives. 6. Strategic Choice(iii): Set targets for performance. Expectations are set for the achievement of the objectives that have been chosen(setting the dials). 7. Strategic Evaluation:  Evaluate the achievement of objectives. i) An analysis is undertaken of whether expectations have been achieved, underachieved or exceeded for  each of the policy objectives(reading the dials).  ii) The overall  figure is compared to the target figure. This is an overall assessment of achievement. ii) The overall assessment of achievement is evaluated in relation to  influence of the external environment.  iv) An overall assessment of the performance of management may be made.  8. Strategic Control  Monitor information for  feedback  for strategic choice. The evaluation of outputs produces information that may be used  to  fine-tune the  re-distribution of inputs.  _____________________________________________ A further output of the process may be the production of information that may be used for advocacy purposes with external stakeholders. _____________________________________________ _____________________________________________ Figure 1. A Model of a Performance Measurement System  for Application in Cultural Organisations (in diagrammatic form) _____________________________________________ _____________________________________________ Organisations have a number of stakeholders which have an interest of some sort in the operation of the facilities. In the introductory remarks for this paper it was noted that the design of the model would,  in the first instance,  be made for the internal use of an organisation. The issue arises as to  which stakeholders interests are best represented by the performance measurement process being presented here.  Benefits and Disbenefits to the Internal Stakeholders of the Organisation The potential benefits for a cultural organisation undertaking this process may be understood in respect of the contribution of the process to improved management and the provision of information for advocacy purposes. The potential benefits include: i)  an increase in the clarity of policy objectives; ii)  an increase in the visibility of the use of public support; iii)  an increase in the congruence between policy objectives and the strategic choices made to  implement the policy objectives, which is to say  in the allocation of resources; iv)  an increase in the visibility of 'additionality';  v)  an improvement in the use in management information; vi)  a commitment is made to the professionalism of arts managers; vii)  the information may be used for advocacy purposes. This would take place when the achievement of  objectives  by cultural organisations becomes explicit to government organisations.  The potential disbenefits for a cultural organisation undertaking this process may be: i)  an increase in planning which may not appear to be  cost-effective; ii)  a reluctance to be precise about policy objectives; iii)  an increase in information for scrutiny that may be used against the organisation  by an external     stakeholder and, thus, a possible decrease in the freedom to manage; iv)  an increase in information for scrutiny that may be used against cultural organisations relative to other forms of spending as a means of achieving   governmental policy objectives; v)  given the imprecision in the measurement of some of the outputs a suspicion that the process may  be of limited value; vi)  a commitment to the professionalism of a group may lead to the maximisation of their own     interests  i.e. an overemphasis on effectiveness criteria above economy and efficiency criteria.  A Performance Measurement Model for External Stakeholder  Groups There are other groups which have an interest  - or a stakeholding - in the performance of cultural organisations with different incentives to measure.  I shall now go on  to address the interests of the external stakeholder groups, or, at least, those government agencies involved in funding and with a responsibility for accountability.  A number of attributes for a model of a performance measurement system were noted earlier in the paper. In order to accommodate the interests of external stakeholders the following attribute needs to be added. The model should be : i) responsive to the interests of external stakeholder groups. To achieve this requires the addition of new elements to the policy  matrix( See Table 14).  The new elements are 'Economic Importance and Impact', 'Prestige' and 'Quality of Life'(Publicness). _____________________________________________ Table14.A Policy Matrix  For  External Stakeholders _____________________________________________ Policy Objective Access Maximisation Attendance Maximisation Diversity/Multiculturalism Economic Importance and Impact Economy Maximisation Education Excellence Innovation Prestige Revenue Maximisation Quality of Life(Publicness) Service Quality Maximisation Social Cohesion  _____________________________________________ Economic Importance and Impact By the late 1980s a commercial rationale for the support of the arts by the state had grown in significance(Myerscough,1988). This rationale was based upon the economic externalities of the arts. These externalities may be used as performance indicators.  These indicators were relevant to the aim expressed by both the Arts Council of Great Britain and the Southern Arts Board  to expand the arts economy.  At the civic level investment has taken place in cultural events and infrastructure not only or the expansion of the arts economy itself but also for the impacts that cultural investment is purported to have on local economies more generally. This form of investment has become an element of inter-urban competition for business relocation and tourism. Investment in culture has become a means of urban regeneration(Bianchini and Parkinson,1993) although indices of urban regeneration were not within the framework of the research. _____________________________________________ Table  15. Economic Importance and Impact _____________________________________________   Performance Indicator What is being  measured? Size of Arts Economy  Effectiveness Impact of Arts Economy  Effectiveness Direct Employment  Effectiveness Indirect Employment  Effectiveness Induced Employment  Effectiveness Revenue related to Cultural  Tourism Effectiveness Business Relocation (using social surveys) Effectiveness Employment Relocation Effectiveness _____________________________________________ Prestige The state, at both  national and local levels, may commit expenditure on cultural activities and institutions for reasons of civic and/or national prestige.  Civic and national prestige is both an end in itself( a form of social externality and a means to other ends (in the form of some of economic externalities noted above). _____________________________________________ Table 16. Prestige _____________________________________________   Performance Indicator What is being  measured? Civic Prestige  (using social surveys) Effectiveness National Prestige  (using social surveys) Effectiveness _____________________________________________ Quality of Life There are constraints on the capacity of users and non-users to express their (dis)satisfaction with public facilities(Mayston,1985).  The reasons for this are, first, to do with mobility.   By  this  Mayston (1985)meant that the consumers of public facilities are unlikely to move house in order to seek an improvement in the quality of provision. Second, the ballot box  is a general expression of preference of policy-makers or as Jan-Erik Lane(1993,pp.197) puts it: The value of goods and services produced(by public activities) does not show up in the national accounts, simply because the demand for these goods and services is not revealed in standard prices. The willingness to pay shows up in the election process, which does not indicate  the marginal value of various goods and services very adequately.  These constraints, I would argue, increase the need for the accountability of public services.  One procedure  is to survey  public attitudes towards public spending on the arts and  include members of the public who do not use cultural facilities directly.  This is significant if one adopts the principle that the public should judge the use of public money, what Peacock(1994)refers to as the 'publicness' of arts expenditure.   The responses in such surveys ( Myerscough,1988 and R.S.G.B.:1991) are positive towards spending on the arts.  Investment in city re-imaging may result in improvements in the quality of  public and private spaces, improvements which may be welcomed by the local citizens( Gonzales,1993 on Bilboa in Bianchini and Parkinson,1993). Indicators of customer satisfaction were not within the framework of the research.  _____________________________________________ Table 17. Quality of Life _____________________________________________   Performance Indicator What is being  measured? Willingness of Local Population  to pay for local arts activity (Using Social Surveys)  Effectiveness Appearance of the arts in quality  of life surveys (Using Social Surveys) Effectiveness _____________________________________________ Benefits and Disbenefits to the External Stakeholders The potential benefits for external stakeholders undertaking this process are in some ways like those at the organisational level in that the process may provide information for management and for advocacy purposes. The process may also facilitate comparison. The potential benefits include: i)  an increase in the clarity of policy objectives; ii)  an increase in the visibility of the execution of the expenditure of support.  This would work given the following paths of accountability and regulation;  local authorities to the Audit Commission/central government; regional authorities to the Audit Commission/central government; regional arts boards to the Arts Council; Arts Council to the Department of National Heritage; iii)  an increase in the congruence between policy objectives and the strategic choices made to implement the policy objectives, which is to say in the allocation of resources; iv)  an increase in the visibility of 'additionality';  v)  an improvement in the use in management information; vi)  the information may be used for advocacy purposes; vii)  the information may be used at higher levels of government to compare amongst different forms of    spending to achieve general governmental policy objectives; vii)  comparison amongst organisations may become possible. The potential disbenefits for the external stakeholder vary according the character of the stakeholder but may include: i)  an increase in planning which may not appear to be  cost-effective; ii)  an increase in accusations of 'back-seat driving' from those at an operational(organisational) level; iii)  an increase in information for scrutiny that may be used against the organisation  by an external     stakeholder and, thus, a possible decrease in the freedom of the body to manage its own affairs (i.e. a breaking of the arms length principle from ministerial level to the Arts Council); iv)  given the imprecision in the measurement of some of the outputs a suspicion that the process may     be of limited value; v)  with arts professionals at the agency level(in Britain these are known as Regional Arts Boards) and at the organisational level a cabal of vested interests could arise which could work against the interests of the public; vi)  a distraction from the 'real business' of the arts; vii)  comparison amongst organisations may become possible.  Ownership and Responsibility in the Performance Measurement Process The initiative in Britain in the early 1980s to develop methods of performance measurement in the public sector was driven by central government. This raises a significant  point of discussion about the relationship between the central administration of government and locally constituted ways of life or to put it another way between central control and local autonomy.   Bauman(1992) has written  on the emergence of the modern state. His argument is that the emergence of the modern state depended on what he refers to as 'universalistic ambitions'.  Such ambitions involve a tendency to 'uniform administrative principles'. This tendency implies  - necessarily -   a move away from that which is not uniform or as he puts it: Popular, locally administered ways of life were now constituted, from the perspective of universalistic ambitions, as retrograde and backward looking, a residue of a different social order to be left behind; as imperfect, immature stages...as grounded in superstition or error...(Bauman,1992, pp11) Bauman's historical observations have contemporary resonances.  Mechanisms of accountability as defined by central government are assumed to  be appropriate even for those organisations which are in some way distant from central government. There is some sort of self-fulfilling prophecy at work. If organisations do not practice forms of management procedure deigned to be virtuous by central government then they are assumed to be poorly managed.  My contention in this paper is that the  model presented in Table 13 and Figure 1 is able to accommodate the maintenance of local distinctiveness.  The policy matrix  for organisations(in Table 1.) has an empirical basis  in the expression of   the objectives of cultural organisations as made by their managers.  This means that the organisations may be  being judged 'in their own terms'.  The interests of the internal stakeholders of management would, therefore, well served by the model system. This is where the particular distinctiveness of cultural organisations may be maintained, where they may retain their 'organic' character to use a term used both by interviewees in the study and by the French economist Lipietz(1992,p.61) when stressing the need to maintain forms of grassroots democracy, for 'communities more centred on themselves.'  There are, of course, other stakeholder groups to be considered. And with public services one of these has to be the public themselves. In their document 'Hitting Local Targets  - The Public Value of Public Services' (1997,p.3) the Public Management Foundation declare that "Public Services are the principal means through which citizens transact with their government, outside the ballot box. " In this document - and with its predecessor 'The Glue that Binds'(1996) - the demand is made  of the need to undertake  surveys of satisfaction with public services. Research undertaken by the  Public  Management Foundation(1997, p.3) of  perceptions of satisfaction with the public services of health, policing and education  found that " many( of members of the public) believe that local managers should be free of any central target to set service standards which they feel best serve local needs." It is possible that this finding  may be  transferable to cultural organisations.   If so, then the model proposed in this paper , with its emphasis on the bespoke rather than the prescriptive, would facilitate  this. There remains, however, something to be said about the character of the transaction between the cultural organisation and the public.  In measuring the economic importance and impact of the arts the significance of the user  is  measured according to how much s/he spends at arts events and facilities. This method can measure how inputs may be turned into throughputs and into economic impacts the functions of the arts economy may be measured according to the criteria of economy and efficiency.  However, the method is not designed to measure the value of the involvement of the user in any other way than financial. The user is strictly a consumer.  This would seem to undermine the relationship between organisation and user if that relationship is supposed to be an expression of citizenship.  The involvement of the user in arts facilities is often  that of an active participant, creative and contributory in the process of cultural production. Many of the performance indicators presented in this paper  conceptualise the user  according to a series of social and geographical  characteristics and frequency of use. As such they may again measure the efficiency of arts organisations and, to some degree, effectiveness of service.  Stewart and Walsh (1992) have pointed to what they refer to as the inadequate language of consumerism when employing that language in the  discussion of the provision of public services.  The argument has some application here.   The conceptual framework of the three 'e's  - even though it includes  'effectiveness of service'  -  is only partially  adequate in conceptualising the complex inter-relationships between the arts organisations and their  users.  I would contend that the three 'e's framework , as developed in the 1980s and into the 1990s, has adopted the language of consumerism.  The  notion of consumers is adequately entertained within the parameters of effectiveness of service when that consumer is understood as a satsified or disssatisfied customer. However, there is an essence of 'publicness'  that the language of consumerism cannot distil.  There are whole host of answers for what the arts do for us as social beings. The key answers to emerge in this research project are that they give us a sense of collectivity and involvement to counter feelings of atomisation and individuation - a sense of publicness. They give us means of expression and means of developing our identities.   The contribution that cultural organisations can make to the  public sphere may be made in terms  associated with political liberalism. The language of citizenship is more loquacious here than the language of consumerism.  It is my contention that the model presented in this paper begins to address this problem by recognising the social  benefits  that may be produced by the investment of public monies into cultural organisations.  The significance of  the social benefits of policy objectives such as access maximisation and social cohesion is recognised   even though those benefits are less amenable to measurement. References Bauman,Z.(1992) Intimations of Postmodernity. London. Routledge Bennett, G. (1992) Dilemmas. Earthscan Bianchini, F. and Parkinson,M.(1993) Cultural Policy and Urban Regeneration. Manchester Manchester University Press. Bourdieu,P.(1985) Distinction: A Social Critique of the Judgement of Taste. London. Routledge and Kegan Paul Coalter, F. Long and Duffield,B.(1988) Recreational Welfare. Avebury. Cwi,D.(1982)'Merit Good or Market Failure: Justifying and Analysing Public Support for the Arts. '. From 'Public Policy and the Arts.' ed.Mulcahy and Swain. Boulder.Westview Press. Evans,G(1995) The National Lottery and Planning for the Arts. Reassertion of the Hegemonies. London C.E.L.T.S. Gonzales, J.(1993)  Bilbao:Culture , Citizenship and Quality of Life in Bianchini, F. and Parkinson,M.(1993) Cultural Policy and Urban Regeneration. Manchester University Press. Gratton, C, and Taylor,P.(1992) Economics of Leisure Services Management.  London. I.L.A.M./Longman H.M. Treasury, (1983) Financial Management in Government Departments H.M.S.O.  Hutton,W.(1995) The State We're in . Jonothan Cape. Jackson, P. and Palmer, A.(1988) Public Sector Performance Assessment : Getting the Concepts Right. Ed. Beeton, D. Public Finance Foundation  Knox,C. Sugden,J.and Tomlinson,A.(1992) Setting The Scene for Leisure in the 1990s. Leisure Studies Publication no.46 Lipietz, A.(1992) Towards a New Economic Order. Oxford. Polity Press Mayston, D.(1985) Non-Profit Performance Indicators in the Public Sector  Financial Accountability and Management. (1)1. Mulgan, G. and Worpole,K.(1986) Saturday Night of Sunday Morning. From Arts to Industry - New Forms of Cultural Policy. London. Comedia Myerscough,J.(1988) The Economic Importance of the Arts. London.P.S.I. Peacock,A.(1994) 'The Design and Operation of Public Funding of the Arts'from Peacock,A. and Rizzo,I. eds. Cultural Economics and Cultural Policies. Kluwer Academic Press  Public Management Foundation(1996) The Glue that Binds London. P.M.F. Public Management Foundation(1997) Hitting Local Targets - The Public Value of Public Services. London. P.M.F. Robbins,Lord(1963) 'Art and the State', in Politics and Economics:Essays in Political Economy. London. Macmillan quoted by Austen-Smith,D.(1963)'On Justifying Subsidies to the Performing Arts' reprinted in the Journal of Cultural Economics,Vol.18. No.3 1994. Kluwer Academic Publishers Schuster, J. Mark Davidson (1987)' Making Compromises to make Comparisons in Cross - national Arts Policy Research.'  . Journal of Cultural Economics. Vol.11. No1. Dec. Stabler, M and Ravenscroft,N.1994 The Economic Evaluation of Output in Public Leisure Services' Journal of Leisure Studies Vol 13 Stewart, J. and Walsh, K. (1992) 'Change in the Management of Public Services'. Public Administration.  Towse,R.(1993) How is Subsidy to the Arts actually Supposed to Work. Paper presented at UFSIA University of Antwerp:  Turner,B. (1993) Citizenship and Social Theory. London. Sage Wilson,R.Gilligan,C & Pearson,D.(1992) Strategic Marketing Management. Planning, Implementation and Control. Oxford. Butterworth Heinemann.   back to IG page