Archive for June 25th, 2009

Update: First time visitors, check out some of the recent or top posts in the right-hand column. If you like what you read, you can subscribe by RSS in the upper right hand corner.

It’s been too long since I’ve been meaning to post this…

A devoted reader of this blog emailed me two articles recently; the first (pdf), by Therese Grijalva and Clifford Nowell, was published last year in the Southern Economic Journal. It endeavors to rank PhD programs in economics overall and by field. They essentially assign a productivity number to each faculty member based on the number of journal articles published and the quality of the journal in which they publish them.

The quality scores are based on two different methodologies, but ulimately boil down to citations. Their results aren’t earth-shattering, with many of the usualy suspects atop the list. A number of more marginal schools earn high marks in various subfields. My alma mater, the University of Notre Dame, is ranked 88th overall and misses the top 20 in all 18 subfields.

The second article, which is still in draft form (and I don’t know if I’m at liberty to publish it on this website), attempts to address the bias towards neoclassical economics in analyses like Grijalva and Nowell’s. They aim towards a “quality-equality” measure of economic programs, while still taking the bibliometric approach seen in other papers. They directly address G & N’s approach and write,

Instead our concerns are with two interrelated issues:  the assumption that in economics, scientific knowledge is homogeneous to which any quality index can be unambiguously applied and the limited coverage and partiality of the SSCI impact factor scores even when restricted to North American, Western European and English language journals.  Economics is about explaining the provisioning process, the real economic activities that connect the individual with goods and services, or more succinctly, economics is defined as the science of the provisioning process. As a field or discipline of scientific study, it consists of two distinctly different theoretical approaches to analyzing and delineating the provisioning process:  neoclassical or mainstream economics and heterodox economics (Lee, 2009a, 2009b).  Although they contest each other’s theoretical analysis, both mainstream and heterodox economics adhere to the discipline’s goal of producing scientific knowledge regarding the provisioning process.  But what constitutes scientific knowledge and its quality is determined by the scientific practices within the two sub-disciplines in economics.  Therefore, a quality index utilized for mainstream economics is not necessarily appropriate for identifying quality research in heterodox economics. Consequently, for a quality index to be used in an even handed way to rank departments in terms of the quality of research, it needs to be a synthesis of the separate ‘indexes’ used in the two sub-disciplines.

Of course, this concern is a direct affront to many neoclassical economists, who don’t consider much of the work of heterodox economists to be as rigorous as their own.

The other concern is that the citation-based quality ranking excludes six of the eleven well known heterodox journals, reducing productivity values for departments with faculty who publish in them. Thus, the authors create a heterodox quality index and combine it with the G & N quality index at parity.

Their approach, not surprisingly, gives the largest boost to heterodox programs like UMass Amherst and UMKC. What is interesting is that the big dogs, like Harvard, Chicago, etc. are still the top departments in their metric. UMass Amherst makes it up to 33.

In discussing the relevance of their findings, however, they choose to bring up Notre Dame and its department split. They observe that the ND department, in their ranking, is 92, while the exiled department, ECOP, is actually 74. Further, when adjusting for size instead of aggregate productivity, the exiled department actually is ranked 25 overall. Thus, they say,

In this case, the claims of the Dean and the chair are not at all supported and their decision to exile the heterodox economists essentially dismantled a better department and replaced it with one of a lesser rank…

The Notre Dame case dramatically illustrates how bibliometric (and peer review) based methods can be misused to silence dissenting voices and to render invisible heterodox ideas and departments in a contested discipline such as economics.  This paper does not disagree with the use of bibliometric methods to rank departments (and journals); but what is objected to is their misuse in the name of science and objectivity….  It is not that doctoral programs with a heterodox presence are better than programs without, but they are also not inferior to them—just different but equal.

This sort of argument isn’t what neoclassical crusaders want to hear. Nevertheless, a more open and rigorous debate is needed about what is and is not economics, so valid approaches are not unfairly crowded out.

Update: While electing not to provide the text of the draft article I discuss, I forgot to provide the title and authors. The title is “Ranking Economics Departments In A Contested Discipline:  A Bibliometric Approach To Quality Equality Among Theoretically Distinct Sub-Disciplines”.

Two of this second paper’s authors wrote the first article, Therese Grijalva and Clifford Nowell. The third author is Frederic Lee. This may clear up some confusion, as I seemed to pit the two against one another. It’s important to keep in mind, though, that the bibliometric style approach is consistent in both.

Read Full Post »