More wierd review literature in this guest post from Climatequotes.
———————–
IPCC coordinating lead author’s own paper falsely cited in AR4
Yesterday I posted about a contributing author’s own paper being incorrectly cited in the AR4. However, I have now found something worse than that: a Coordinating Lead Author’s own paper has been falsely cited.
Recently I posted about an interesting claim and comment made in the AR4. In the Second Order Draft (SOD), the following claim appears (page 43, lines 34-36):
Relatively few NGOs are directly accountable to members in the same way that governments are to voters or businesses are to shareholders, raising further questions about the extent to which their claims to the mantle of civil society are justified.
The claim had no citation, which led an expert reviewer to make the following comment, Comment 12-189 (page 78):
Seems a bold claim – can you substantiate it? ENGOs in most cases are supported through financial contributions from individuals as well as from foundation funding. They are also governed through Boards that in principle represent their consituency. If you leave this, you need to provide a citation.
(Jan Corfee-Morlot, University College London (on
leave from OECD))
The reviewer thinks that this claim is bold, and seems to imply it is false. He ends “If you leave this, you need to provide a citation.” The writing team responds:
Noted. Will look for references.
In the final report, here is how the same claim is presented, Section 12.2.3.3 Civil society of WG III:
Relatively few NGOs are directly accountable to members in the same way that governments are to voters or businesses are to shareholders, raising further questions about the extent to which their claims to the mantle of civil society are justified (Najam, 1996).
Notice it is the same as the SOD, except for the (Najam, 1996). The authors apparently looked for and found a citation. That citation was not in the First or Second drafts, and not mentioned in the reviewer’s comments. It seemed to have appeared out of the blue in the final report. Here is the reference:
Najam, A., 1996: Understanding the Third Sector: Revisiting the prince, the merchant and the citizen. Nonprofit Management and Leadership, 7(2), pp. 203-19.
That’s Adil Najam. He was the Coordinating Lead Author for this chapter. Maybe the paper didn’t come out of the blue after all.
On my previous post, I did not have access to the paper. I have now obtained it. After reading the paper, I do not see how it supports the claim which it is the citation for. I’ll walk you through how I came to this conclusion.
The IPCC claim is that “few NGOs are directly accountable to members”, which raises “further questions about the extent to which their claims to the mantle of civil society are justified”. They are questioning both the accountability of NGOs to their members, and the NGOs claims that they represent ‘civil society’. In order for their Najam citation to match their claim, it must question both NGO’s accountability to its members and its claim to represent civil society. I couldn’t find it doing either.
First, I need to clarify how the paper defines certain sectors. It talks about three sectors, the State, the Market, and Voluntary Association groups. NGOs are in the Voluntary Association sector.
On the claim of accountability, the paper only talks briefly. In Table 1, pg. 11, they list the three different sectors (state, market, and voluntary association [read NGO]), with certain characteristics. One of the characteristics is Accountability. The state is considered accountable to the electorate, the market is accountable to investors, and voluntary associations are accountable to constituency. Constituency meaning, members. There is no asterisk explaining that the sector isn’t directly accountable to its members like the other sectors.
The only other mention of accountability occurs on page twelve:
In this conception the rationale for what is sometimes called the third sector-but which Uphoff (1996, p. 24) views as a “middle sector” thaz “operates best between the public and private sectors”- comes not from certain organizations refusing to fit into the mold of the established institutional templates of society but rather from “demands by citizens for accountability from the prince and the merchant” (Gordenker and Weiss, 1996, p. 19).
This is stating that the third sector demands accountability from the other two sectors. Nothing to do with the IPCC claim.
I cannot find anything else that relates to the accountability claim. If you can, please let me know about it.
The second claim is that there are “questions about the extent to which their [NGOs] claims to the mantle of civil society are justified”. This paper does mention civil society several times, but it doesn’t question the NGOs claim as being a part of it (except in one case where they consider some NGOs to be state actors, a different question entirely). Actually, the ‘voluntary association sector’ where NGOs belong is linked to civil society several times. On page two:
Despite all of the above, there has been no dearth of attempts to define what is variously referred to as civil society or the nonprofit, the nongovernment, the voluntary, the independent, the charitable, the philanthropic, the associational, and the third sector.
Another case occurs on page ten:
The third-the voluntary association sector-is most concerned with the articulation and actualization of particular social visions, through the shared normative values of its patrons, members, and clients; it represents the interests of the minority (or of marginalized interests) and operates in the realm of civil society.
That is pretty clear. It operates in the realm of civil society.
Also, in that same Table 1. from before, we see this connection again. Another characteristic they describe of the three sectors is ‘Social Model’. The state sector is political society, the market sector is market society, and the voluntary association sector is civil society. Interestingly enough, when this characteristic reaches the ‘source’ column, it reads “Suggested by the author“.
The author himself suggested labeling voluntary associations as a part of civil society.
If you can tell me why I am reading this incorrectly, please let me know. I will update my post to reflect the fact that I am wrong, the IPCC is right, and I can’t read. However, I feel after reading the paper in depth, the IPCC is the one who misrepresented the paper and made an error. (Najam 1996) is cited validly in another section of the AR4, but I do not know why it was cited here.
I am not saying the IPCC’s questioning that NGOs aren’t accountable and aren’t a part of civil society isn’t valid. The claim may very well be accurate. It also may not be. The claim is pretty unimportant and not really related to climate change. My issue is with the process. A claim without a reference in the SOD was challenged by a reviewer, the IPCC writing team said they would look for references, then a paper written by one of the two coordinating lead authors of the chapter appears in the final report as the citation for the previous claim, and the paper doesn’t appear to support the claim.
I welcome Mr. Najam to explain this, perhaps it is a misunderstanding or I simply read the paper wrong. I have contacted him but have not yet received a response.
the Air Vent 
I think we will be flooded with many more inappropriately cited (citing a peer-reviewed paper for a particular statement, which is not in that paper) references in the IPCC reports in the coming weeks. It is more commonplace than one would like.
#1 It appears to be very commonplace for the IPCC report unfortunately.
Jeff,
Are you suggesting that the broader issue–from the IPCC perspective–is conflict of interest? Reply: The post is from climatequotes
An IPCC process exists.
That process includes independent review.
…Everything seems to work to this point.
Review occurs comments are made and documented.
…Still things seem to be working.
Someone or someones review the comments and post a response.
…Still according to Hoyle.
Now it goes off the rails.
When implementing recommendations in the comments someone (the Lead Author?) seeks a supporting citation he/they choose a paper written by a lead author. To make matters worse, the paper seems to be falsely cited.
…So here is where we lose transparency. Person or persons not named, but who we can deduce must be part of the lead author’s team clearly exercise questionable judgement in the application of a citation.
If this person is the lead author we have a clear conflict of interest. Even if the lead author has recused himself from the decision making process, there is no disclosure of such.
If I’m reading this correctly, there are several issues at play here with respect to the process other than the false citing of a paper (as egregious as that would be if true). There is:
1. Lack of Transparency when final decisions are made. This includes failure to document etc.
2. No mechanism to prevent real or perceived conflicts of interest and all the disclosure procedures that entails
3.The ability to inappropriately cite without further review.
To answer your question, I don’t think your misrepresenting the paper.
The real problem is the sheer amount of references to be checked. Many papers are not accesible for most people, because they are behind the infamous “pay-wall”. This precludes the inspection of the validity of the citations by the “Jo-Doe and Anne-Doe”, which is a shame for any reasonable human being. I want to be able to verify every single claim, citation, whatever. And do not come with the rebuttal that I am not qualified to judge these papers. After all, the authors of most of these papers have been paid by my taxation.
Jeff,
My apologies had to deal with a seriously skateboard scraped twelve year old and missed the guest post intro. Came back to my computer and the headline was what was in frame.