Published on Academia, 2008
______________________________________________
Original title: Avant le Déluge: Heterodox Economists Respond to UK Enquiry On Economics Research, Six Months Before the 2008 Crash
Introduction
These documents are a chronicle, and a record, of the submission that the UK-based Association for Heterodox Economics (AHE) made to the UK’s ‘Quality Assurance Authority’ (QAA) in response to a request for views on the ‘Research Assessment Exercise’ (RAE) which is the framework for awarding grants to economics researchers. The documents shed clear light on the reason that the economics profession completely failed to anticipate or understand the crash, highlighting the urgency for a ground-up reform of the way economics is taught, researched, and practiced. The submission was made in February 2008. No response was received and there is no indication that any of its recommendations were either implemented or even influenced the QAA’s procedures.
Six months later, on September 15th2008, Lehman Brothers collapsed, and the global financial crisis ensued.
Table of contents
Letter from Fred Lee to the AHE, 18 January 2008
Appendix 2: extracts from subject benchmarking statements
Appendix 3: demands of the post-autistic economics movement
Summary of the submission
The submission presented, and argued for, three key conclusions
Conclusion 1
The peer review system as at present applied within economics is unfit for purpose. It has cramped creativity and innovation, restricted the discipline’s ability to respond to change, lowered the quality of its output, and thereby damaged its reputation.An appropriate system for scientific advance must reward criticism, innovation, and hence difference from the thinking which is conventional at any given time. The present system in our experience lacks instruments to promote any of these, and works predominantly to reward subjectivity and conformity.
Conclusion 2
An assessment system based almost exclusively on peer-reviewed publication in ranked journals has interacted in a negative way with the discipline’s conception of itself and the standards of judgment that these erect.The hallmarks of this conception are described by the PAE statement in appendix 3 and the economists’ declaration in appendix 1.
Economics, our consultation establishes, is widely recognised to organise around the two principles of monotheoreticism, the idea that the subject has arrived at, or is in the process of arriving at, a unique set of valid ideas; and the prescriptive approach: the idea that the criterion of ‘good’ economics is the reproduction of a fixed set of accepted methods, concepts and procedures.
Extracts from other QAA subject benchmarks, reproduced in appendix 2, show that these principles are remote from those of all other subjects, which are organised around pluralism, the idea that a variety of concepts, ideas and approaches are required to arrive at truth, and around a particular object of study (government, spatial relations, etc), not a particular method of studying it.
Conclusion 3
Economics is attracting a critical and disbelieving response from the general and the educated public, from peer disciplines, and students.If evidence showed that the above processes are in fact selecting for genuinely and objectively excellent results, there would be grounds for maintaining them in their present form. [However], the extent of unease with the outputs of the economics profession is unparalleled in other disciplines.
The background
The UK’s Quality Assurance Authority is charged by the government with overseeing how academic subjects are taught and researched. In 2006 it had already conducted a review of teaching leading to so-called ‘benchmarks’ as guidelines for curriculum design and implementation.
In 2008 it turned its attention to research, as part of the general review of research funding known collectively as the ‘Research Assessment Exercise’ or RAE. The documents here record the process of submission and the submission itself
Letter from Fred Lee to the AHE, 18 January 2008
18 January 2008
UK Colleagues, AHE Members, and Others,
The ESRC in conjunction with the Royal Economic Society and the Conference of Heads of University Departments of Economics is undertaking an International Benchmarking Review of UK Economics. The remit of the review is as follows:
The Economic and Social Research Council has embarked on a series of international benchmarking reviews across the social sciences. This review has been established by the ESRC in partnership with the professional body for Economics in the UK, the Royal Economic Society, and in consultation with the Conference of Heads of University Departments of Economics. The object is to benchmark the current position of UK Economics research against the best done world-wide, highlighting strengths and weaknesses as appropriate.
The review report will probably suggest actions for the profession, funders and users of research. Whilst there may be recommendations for the ESRC, the outcomes will concern, and will be considered by, social science funders generally. For its part, the ESRC will use the review results to inform its work to sustain the health of social science disciplines, and to feed into its strategic planning process.
The International Benchmarking Review of UK Economics is managed by a steering group (chaired by Professor Sir John Vickers) that is comprised of senior academics, research funders and research users. The review will be undertaken by an International Panel which will visit the UK for a few days, to coincide with the RES Annual Conference in March 2008. The Panel consists of six leading international academics, chaired by Professor Elhanan Helpman, Harvard. They will meet with a number of UK scholars, study current UK research and assess the standing of Economics research in the UK against agreed criteria. After the fieldwork, the Panel will compile a report that summarises their assessment of current UK Economics research.
In order to provide context for the Panel, a briefing pack will be provided prior to their visit to the UK. This will include demographic and funding statistics, bibliometrics, the results of a survey of non-academic stakeholders, submissions from Heads of Departments of UK Economics and an overview of research quality in the principal sub-disciplines of UK Economics.
Although the ESRC initially overlooked the Association for Heterodox Economics, through the alertness and work of Andy Denis, Andrew Mearman, and Alan Freeman a letter on behalf of the AHE and its members was sent to the ESRC manager of the Review requesting that the AHE be allowed to make a submission:
The Association for Heterodox Economics is a British economics organization that is distinct from the Royal Economics Society. It is committed to open dialogue in economics and progress towards that goal.
We have become aware of the current international benchmarking review of UK Economics. This is an issue on which we have made submissions before.
We should like more information about the current process. Our view is that because of the nature of our members’ views, which tend to be marginalised in debates in economics, they would most likely not be represented by the Royal Economics Society or in the submissions from UK Heads of Departments of Economics. We would welcome the opportunity to participate in the current international benchmarking review process. Could you let us know if you would be willing to accept a submission from the Association that dealt with UK Economics in an International Context and the Health of the Discipline?
The manager of the Review responded positively:
Your association is very welcome to make a submission to the Review Panel – we will include this in its briefing materials. You may wish to limit your submission to no more than 2-3 sides of A4 as the Panel will have a very large number of papers to read through. We will need to receive you submission by mid-February in order to include this in the briefing pack for panel members which we are planning on sending out one month prior to their visit.
Therefore the AHE is now soliciting brief comments from all heterodox economists in the UK and elsewhere that fall under the following headings:
UK Economics in an International Context– your impressions of the strengths and weaknesses of the UK economic research area set within an international context.
Health of the Discipline– your views of the health of the discipline, including training and capacity issues.
Please send your brief and succinct comments to the AHE Coordinator, Alan Freeman ([email protected]) by February 7thso that he can collate and submit them to the AHE Coordinating Committee which is meeting on February 9th. Anybody wishing to assist Alan (and Andrew and Andy) in drawing up the submission, please contact Alan at the above e-mail address.
For the International Benchmarking Review of UK Social Anthropology: http://www.esrcsocietytoday.ac.uk/ESRCInfoCentre/Images/Ant_Book_A-web_tcm6-16599.pdf
For theInternational Benchmarking Review of UK Politics and International Studies: http://www.esrc.ac.uk/ESRCInfoCentre/Images/P_IBR-Final_Report_tcm6-23426.pdf
The AHE Submission
Introduction
This submission, on behalf of the UK Association for Heterodox Economics (AHE, www.hetecon.net), is based on consultations with the substantial worldwide community of practicing economists who define themselves either as heterodox or as pluralist – or, as is common, both. This includes
- the AHE, now in its tenth year and representing some 200 practicing economists. Submissions to our conference, held annually since 1998, have risen to 150, showing the growing interest in critical and non-mainstream approaches;
- the International Confederation for Pluralist Economics (ICAPE, icape.org) which represents over 30 organisations worldwide and was founded in 1993;
- the Post-Autistic Economics (PAE, paecon.net) movement, formed by French students in 2000 to request “a pluralism of approaches, adapted to the complexity of the objects and to the uncertainty surrounding most of the big questions in economics”. The movement provoked a high-level ministerial enquiry and its journal, in its 43rdissue, has 9590 subscribers in 150 countries.
This note draws on the appended statement on pluralism in economics, signed by 44 leading economists worldwide including three Nobel Prize-winners and published in the American Economic Review in 1992.
It also draws on the AHE’s note on the QAA subject benchmarking statement, deposited at http://mpra.ub.uni-muenchen.de/6832/1/MPRA_paper_6832.pdf. Whilst research and curriculum development are administratively distinct, our general view is that there is a close practical and conceptual relation between the two.
Finally this note is informed by recognised empirical research into the effects of the present RAE, and by critical discussion on methodology and assessment within economics. The representative paper cited at the end of this submission, which we invite the panel to consider and respond to, contains a detailed bibliography.
Key points
There is an emerging consensus around three principal points:
- The peer review system as at present applied within economics is unfit for purpose. It has cramped creativity and innovation, restricted the discipline’s ability to respond to change, lowered the quality of its output, and thereby damaged its reputation.An appropriate system for scientific advance must reward criticism, innovation, and hence difference from the thinking which is conventional at any given time. The present system in our experience lacks instruments to promote any of these, and works predominantly to reward subjectivity and conformity.
The world changes rapidly; economics has repeatedly been obliged to revitalise and transform its ideas in response. The present system has damaged its capacity to do so, and we hope the review will place at the forefront the question: where will the next generation of innovative economists come from?
The system has yet to respond to the challenge thrown down, sixteen years ago, by 44 leading economists worldwide including 3 Nobel Laureates, which called for “a new spirit of pluralism in economics, involving critical conversation and tolerant communication between different approaches.”
- An assessment system based almost exclusively on peer-reviewed publication in ranked journals has interacted in a negative way with the discipline’s conception of itself and the standards of judgment that these erect.The hallmarks of this conception are described by the PAE statement in appendix 3 and the economists’ declaration in appendix 1.
Economics, our consultation establishes, is widely recognised to organise around the two principles of monotheoreticism, the idea that the subject has arrived at, or is in the process of arriving at, a unique set of valid ideas; and the prescriptive approach: the ideathat the criterion of ‘good’ economics is the reproduction of a fixed set of accepted methods, concepts and procedures.
Extracts from other QAA subject benchmarks, reproduced in appendix 2, show that these principles are remote from those of all other subjects, which are organised around pluralism, the idea that a variety of concepts, ideas and approaches are required to arrive at truth, and around a particular object of study (government, spatial relations, etc), not a particular method of studying it.
Regardless of the (debated) merits of journal-based assessment as it applies to other subjects, it has particularly detrimental effects in economics:
- It selects for conformity and penalises creativity. In evaluating journal submissions, prescriptive standard – all too often reducing to the ability to reproduce the subjective prejudice of the reviewers – dominates over critical thinking and innovation. An assessment system dominated by ranked journal submission reproduces and amplifies this tendency.
- The system has provoked the ‘diamond list’. The frequency of denials that this exists are matched only by the extent of recognition that it does, and this is hardly surprising: it is an entirely predictable result of an evasive strategic response: peer-review journals have proliferated to fill the gap in the market for peer- reviewed outlets. Research panels respond to the oversupply of ‘research output’ by creating a hidden extra standard – journal rankings. This nebulous concept of quality is all the more dangerous for being unacknowledged and, consequently, both subjective and opaque.
- This in turn provokes a narrowing of the academic base of the profession. The strategy adopted by most departments to maximise success in the RAE is to concentrate recruitment on those they deem most likely to score in the ‘diamond list’ which is almost exclusively mainstream. As a result, teaching of alternative or critical approaches is being severely reduced – as, of course, is the number of students capable of critical or innovative thinking.
- Economics is attracting a critical and disbelieving response from the general and the educated public, from peer disciplines, and students. If evidence showed that the above processes are in fact selecting for genuinely and objectively excellent results, there would be grounds for maintaining them in their present form. The extent of unease with the outputs of the economics profession is unparalleled in other disciplines. This unease is visible in:
- increasing but largely unrecognised interest in approaches or subject areas marginalized by the profession including feminist economics, the new geography, the new labour economics, sustainable economics and cultural economics;
- a growing reappraisal of erroneously discarded and discredited paradigms including Keynesian, Institutional, Evolutionary, and Marxian approaches;
- the growing tendency for key contributions to economics outside the discipline, not least from the study of business, geography, urban economics, the environment, sociology and psychology, and a growing feeling that work with or within other disciplines is more likely to add to human knowledge even though it will not be rewarded by the research exercise;
- critiques from authoritative commentators outside the profession, such as George Soros (only the most prominent in a significant list);
- dissent from the profession’s own recognised authorities, for example Stiglitz or Krugmann (and extending to Nobel prize winners);
- recognition of the validity of this dissent is provided by the recent, well-known, and severely critical independent audits of the International Monetary Fund and the World Bank. It is noteworthy that the principal criticism of the World Bank’s prescriptions was its failure to consider alternative possibilities to its own analysis.
- dissatisfaction from the consumers of economics. The very fact that NGOs and students are both organised into what amounts to a permanent movement of public dissent from official economics is a unique phenomenon among the social sciences to which the research assessment process has to our knowledge made no response. This dissent however extends to a growing number of governments and the business community itself.
- growing difficulties in attracting quality students to the study of economics as such.
Unease is also manifested evident in the growing volume of research activity displaced by the RAE into other fields. Lee (2007) notes that “Out of the 94 British universities and other institutions of higher education that offered undergraduate and post-graduate courses and degrees in economics circa 2000 only 41 entered the 2001 RAE…To account in part for the 53 non-submissions, the Royal Economic Society asserted that weaker economic departments submitted to the Management and Business Studies unit of assessment (Royal Economic Society).”
However as Lee notes, no argument or evidence was offered to support the RES’s assertion. It is noteworthy that the RAE panel for Business and Management Studies felt ‘the quality of the research submitted was high on the whole, with a substantial portion of at least national standing and a significant fraction of international standing’.
In effect, the RES’s defence rests on a unilateral claim which goes to the heart of the problem with the discipline: it rests on the unjustified and entirely self-referential assertion that the standards of economics are higher than those of its sister disciplines. We consider this defence unacceptable. The growing weight of public dissatisfaction should be ample evidence that a belief in the superiority of economics is shared nowhere outside the profession itself. The time has come for economics to put its house in order.
Alternatives
Solutions may lie in two directions: reform of the general preconceptions of the discipline, which falls beyond the remit of the present exercise, and a reconstruction of assessment and benchmarking systems to correct for the inbuilt defects of these preconceptions.
A variety of options in the second direction have been proposed, and we think should all be considered, including for example alternative, and explicit, evidence of research impact such as impact or citation indexes, and so on.
There is a difficulty with all such methods of assessment: in any institution dominated by prescriptive and monotheoretic paradigms, pressure for conformity will tend to dominate all selection criteria. This should be expected to show up to a greater or lesser degree in any ranked system based on journal publication and citation, although any alternative to the present would represent an improvement.
A somewhat more radical approach, would be to introduce criteria for the evaluation of research that positively select for pluralism, that is, for diversity and divergence from the mainstream.
There are strong grounds for including such alternative rankings among the criteria used to evaluate research performance. It is certainly reasonable, and in our view necessary, that departments should be required to show evidence that submissions to the RAE have been made in a variety of fields and from a variety of standpoints. Two measures that would re-enforce the impact of alternative assessment criteria are:
- to provide explicit additional assessment criteria, indicative of pluralism and a heterodox presence, as a required assurance of research recognition.
- to provide a variety of routes to bid for recognition for research excellence, which would include submissions based on the presence of heterodox, innovative, or critical standpoints, and on evidence of pluralist practice.
At the very least there should be alternative routes to achieving an RAE standing sufficient to attract funding which do not place departments under the debilitating yoke of conformity. Departments should be offered the serious option of submissions based on a ‘pluralism’ criterion, for which there should be stated criteria including variety of journals, range of disciplines engaged with, and so on.
Ideally, pluralism should be a required criterion of excellence. Top, that is 5*, funding should not be obtainable without clear evidence of diversity and the encouragement of heterodox and critical approaches, including variety of journals, staff members, conference activity, and taught perspectives.
The basic principle involved is highlighted by the 1992 declaration, which notes that: “Economists will advocate free competition, but will not practice it in the marketplace of ideas”. If economics believes what it preaches, then logically it is inefficient and restrictive to seek to erect a single regulatory standard applying uniformly to all economics research. Monotheoretic research, whilst recognized as making a valid contribution to knowledge, should be recognized as an insufficient basis for institutional excellence; and there should be genuine competition of standards, enforced by avariety of evaluation routes. The bottom line is that no department, and no individual, should be forced to demonstrate excellence by submitting to views they do not agree with.
Appendix 1
A Plea For a Pluralistic and Rigorous Economics
This appeal appeared in The American Economic Review, Vol. 82, No. 2, p xxv (1992)
“We the undersigned are concerned with the threat to economic science posed by intellectual monopoly. Economists today enforce a monopoly of method or core assumptions, often defended on no better ground that it constitutes the ‘mainstream’. Economists will advocate free competition, but will not practice it in the marketplace of ideas.
“Consequently, we call for a new spirit of pluralism in economics, involving critical conversation and tolerant communication between different approaches. Such pluralism should not undermine the standards of rigor; an economics that requires itself to face all the arguments will be a more, not a less, rigorous science.
“We believe that the new pluralism should be reflected in the character of scientific debate, in the range of contributions in its journals, and in the training and hiring of economists.”
| Abramovitz, Moses Arthur, W. Brian Axelrod, Robert Blaug, Mark Boulding, Kenneth Cowling, Keith Cyert, Richard M. Davidson, Paul Day, Richard Deane, Phyllis Denison, Edward Desai, Meghnad Freeman, Christopher Frey, Bruno Furubotn, Eirik | Galbraith, J.K. Georgescu-Roegen, N. Goodwin, Richard Grainger, Clive W.J. Grandmont, Jean-Michel Harcourt, Geoffrey Heilbroner, Robert Hirschman, Albert Kindleberger, Charles Kornai, Janos Laidler, David Leibenstein, Harvey Matthews, R.C.O. Mayer, Thomas Minsky, Hyman | Modigliani, Franco* Nelson, Richard Olson, Mancur Pasinetti, Luigi Perlman, Mark Rothschild, Kurt Samuelson, Paul* Shubik, Martin Simon, Herbert* Spanos, Aris Tinbergen, Jan* Tsuru, Shigeto Vickers, Douglas Weintraub, Roy [Nobel Laureates = *] |
Appendix 2: extracts from subject benchmarking statements
Theology
Given that constant new development has been the characteristic of the field of TRS since the latter half of the twentieth century, both in the UK and elsewhere, it is vital that any definition of the subject does not constrain future innovation… Much of the excitement of the discipline lies in its contested nature.
Accounting
[K]nowledge and understanding of some of the alternative technical languages and practices of accounting (for example, alternative recognition rules and valuation bases, accounting rules followed in other socio-economic domains, alternative managerial accounting approaches to control and decision-making)Politics
The scope of politics and international relations is broad, the boundaries often being contested or in movement. Perhaps in no other academic discipline are the subject matter and approaches so much in contention and in flux. This contributes to the challenging yet captivating nature of the discipline. The present state of the discipline is the result of curiosity, free inquiry and debate and its future will be driven by the same forces. It is therefore not the intention of this section to lay out a ‘national curriculum’ for politics and international relations.
Earth sciences
[M]ost tuition has an holistic, multidisciplinary and interdisciplinary approach…It is stressed that the examples which follow should not be taken as prescriptive but are presented to illustrate the variation in emphasis from subject areas which can be described as natural sciences-based to those characterised as more social sciences or humanities-based.History
We have seen our task as the following: to lay out criteria for judging the suitability and adequacy of single-honours degree courses in history; to do this in a way that is as specific as possible without undermining the principle that there are many different suitable and adequate ways of constructing and making available the great richness and diversity of history; to do it in a way that recognises also the need for adaptability to new academic developments in the field, and innovations in course structures and teaching methods.
Geography
The breadth of geography means that many of its core constituents can be approached through a number of routes, and so any attempts at prescription must be discarded; institutions offering degree programmes in geography must be free to decide upon the details of content and organisation.
Appendix 3: demands of the post-autistic economics movement
The following are extracted from the manifesto, published in 2000, of the French post-autistic student movement (http://www.paecon.net/HistoryPAE.htm)
Most of us have chosen to study economics so as to acquire a deep understanding of the economic phenomena with which the citizens of today are confronted. But the teaching that is offered, that is to say for the most part neoclassical theory or approaches derived from it, does not generally answer this expectation…The empirical side (historical facts, functioning of institutions, study of the behaviorsand strategies of the agents) is almost nonexistent.
Out of all the approaches to economic questions that exist, generally only one is presented to us. This approach is supposed to explain everything by means of a purely axiomatic process, as if this were THE economic truth. We do not accept this dogmatism. We want a pluralism of approaches, adapted to the complexity of the objects and to the uncertainty surrounding most of the big questions in economics (unemployment, inequalities, the place of financial markets, the advantages and disadvantages of free-trade, globalization, economic development, etc.)
References
Lee, F.S., 2007. ‘The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities’. Camb. J. Econ., 31(2), p.309-325.
*****
Click on the title to read as PDF version: Avant le deluge: Heterodox economists respond to UK enquiry on economic research, six months before the 2008 crash:
