Some have claimed that Cook et al. found a consensus on the dangers of climate change (Kammen, 2013) or on the need for climate policy (Davey, 2013). They investigated neither. Even some of the authors of the paper misrepresent its findings (Nuccitelli, 2014, Friedman, 2014, Henderson, 2014).
Cook et al. took a sample of the academic literature and rated its contents. The raters were recruited through a partisan website (Cook et al., 2013) and frequently communicated with each other (Duarte, 2014). Their sample is not representative of the literature (Tol, 2014a). The sample was padded with large numbers of irrelevant papers (Tol, 2014a). For example, a paper on photovoltaics in Kenya (Acker and Kammen, 1996) was taken as evidence that climate change is caused by humans as was a paper on the coverage of climate change on US TV (Boykoff, 2008). Three-quarters of the "endorsing" abstracts offer no evidence either way (Tol, 2014a). Their attempt to validate the data failed (Tol, 2014a). An attempt to replicate part of the data failed too (Legates et al., 2013). The data show inexplicable patterns (Tol, 2014a) while the consensus rate suffers from confirmation bias (Cook et al., 2014a, Tol, 2014b).
in sum, one of the most visible climate papers of recent years is not sound. Whereas previous critique could be interpreted as a lack of competence (Tol, 2014a), the later data release suggests that Cook et al., perhaps inadvertently, worked towards a given answer. This reflects badly on the authors, referees, editors and publisher. It also weakens the activists and politicians who cite Cook et al. in support of their position.
BTW, I'll be forever grateful to Richard Tol for coining the phrase "sunk benefit." (I assume he coined it.) He mentions it here.