There is but little time in my world right now: I’m moving cities and decluttering on a manic scale. Let me therefore point you at someone else’s stuff again. Every now and then I have a bit of a rumble about peer review here, because although it is like democracy probably the least worst system it has undeniable weaknesses, as it relies on a professional detachment that we can’t all always manage and on editors knowing who the ideal reviewer for any given work should be, which makes it vulnerable to whom-you-know network constriction. Now, I don’t have an alternative to this, but I did recently read this, linked to in comments to one of Ben Goldacre’s recent posts at Bad Science, which proposes one. Specifically:
Now, OK, he’s talking science, and the actual model-building the article’s mainly about strikes me as being uselessly simplistic and normalising, but this is still interesting. Aside from the presence of money in the system and the lead time to print there still isn’t that much difference at the publishing end between sciences and humanities, I think. I could imagine such a system in operation for history, although I can’t imagine us getting to a single clearing-house server: who would run it? (Don’t say Google! But how much would they love to do it? It might make Google Scholar useful!) We would have competing publishers or national academies for a long time. Who would control access, then? But if, for example, all research funded by the British Arts and Humanities Research Council were required to be submitted to such a server, then I bet scouting that server would be worth doing. And so on. The other question is that of judging articles etc. on saleability. Exciting and trendy topics would maybe reach a market more readily than grunt-work shifting data or editing. Of course, you could argue that they already do; that’s at least one meaning of trendy, isn’t it? I imagine that there would always be a market for journals that wanted to publish off the mainstream. But even that work would be easier to find this way. So I think the biggest question for me with such a system would be the same one I often ask: who keeps the gates? But if gatekeepers could be agreed upon to everyone’s advantage, do you think this would be better than what we have, or would it completely erode quality? Would we still need peer review behind it? Or could we rely on editor’s discretion? Would it open things up? Or dilute them uselessly? Your thoughts would be welcomed while I try and source more cardboard boxes and wonder if I really need all the books…
1 October 2010 at 4:45
*ponders* I’m not that familiar with the publishing process (should, uh, get onto that. I have at least one draft article I should kick into shape soon…), but I’m under the impression that the peer review process – rejections, particularly – gives the author a chance to edit up a piece and (in theory) improve it? This server-based idea would doom your article to remain in the form you first put it out in, which might be great in the interests of not having your personal brilliance watered down, or it might suck, if your personal brilliance isn’t as brilliant as you thought it was.
1 October 2010 at 9:37
Well, I guess that it leaves the rôle with the editors. I can imagine, for example, the editor of Journal of Antipodean Medieval Studies (should there be such a thing) contacting you and saying “Dear Ms Eccentric, we really like your piece on $SERVER on the twittiness of Arthurian heroes and would like to publish it, but we wonder if you were aware of this manuscript in which Cei is actually less twitty than that, also we’re concerned that there isn’t enough slash, could you revise accordingly and send me an updated version?” As with the rest of the process, the editor is here doing stuff that otherwise reviewers would do and he or she just pass on. Many do this anyway, as an extra layer of checking and also because they have their eye on how the journal operates more than the reviewers do.
When you do get round to sending stuff out, this is an essential guide.
1 October 2010 at 13:51
Oooh, a link! Thank you :D
1 October 2010 at 11:02
The immediate problem I see is that scientific research (at least physics) is much more peaked/focused than historical research. As in, everyone is piling into the hot topics in their particular field and competing to solve the same problems. In contrast, historical research, though it has some trendy topics, is far more about finding your particular niche and then exploring it (possibly more akin to a scientific field like zoology, where you choose your species and stick to it, even when lemurs are no longer cool). There’s far less repetition or extension of others’ work: you have your own bishop and you know more about him than anyone else in the world (whereas you are unlikely to know more about Pluto than anyone else in the world).
When you add to that that historians aren’t required to show all their data, you end up with it being extremely difficult to assess the quality of a particular piece of historical work safely if you’re not an expert in that specific field, because you don’t know the key question: is there some vital sources that the author has not mentioned which contradicts their argument completely? It’s the unknown unknowns that are the problems. If someone says that a sixteenth century document in an archive says X, you are unlikely to go and check whether it actually says not-X. And whereas I can tell a plausible seminar paper about Vandal Africa from one with an unlikely argument, I can’t easily tell whether the plausible paper may nevertheless be based on a completely wrong dating of some key text. (In contrast, one of the reasons that Carolingian scholarship is both so much fun and so terrifying is that there a number of us who basically know a fair bit about each other’s texts, so we can point out the omissions and contradictions in each others’ arguments. The downside is grinding trench warfare over the dating of the Vita Karoli).
So I don’t see how the editors of, say, Early Medieval Europe (and that’s a far more focused journal than many historical ones) could accurately spot on their own which were the very good papers on seventh century Byzantium and tenth century Pictland and Carolingian liturgy, as opposed to the ones which were just good (though they could probably spot the rubbish ones).
1 October 2010 at 18:26
All grants that now get American federal dollars are required to make their work available through PubMed, which has been a searchable database of all published science for at least 20 years. The number of open access journals in science is growing fast. I read recently that PLOS One is now among the highest publishers of science articles. (PLOS = Public Library of Science)
2 October 2010 at 2:07
[...] This post was mentioned on Twitter by nicola griffith, BioMed Central. BioMed Central said: Reading: 'On peer review and an alternative' http://ow.ly/2MNed #oa #peerreview [...]
13 October 2010 at 2:48
This is very interesting. I see the problems Magistra does. I wish more humanities journals were open access; there are social science organisms in Chile and Brazil that do a lot of this for those fields.
The thing I would be nostalgic for in a different system (not that I think the current one isn’t without the problems already noted) is that character of journals. There’s just something about each one, they have a certain tradition and editorial board, and that makes certain types of people submit certain types of work to each one, and so they have their personalities; I come from the days when one still looked at them on paper, so every article came in the context of a whole issue or bound volume, and I still find it important to experience that…
13 October 2010 at 13:43
I agree with all of this, really. In the UK and Western Europe (and I guess the US also) humanities journals have found it better to join big subscription packages than to risk open access. I also like handling a new journal issue as if it’s a magazine and contains up-to-the-minute historical gossip. One knows, intellectually, that a lot of the stuff in there has been ground out over years, but it’s when it’s in print, in my head, that it’s new. Preprint online access is messing with this model and I don’t like it.
The character of journals is also true, but I’m less happy about that. It means that one has to learn still another set of unwritten rules about where something should be sent and who will be kind to it in order to succeed, whereas one would hope the quality of the material was the first thing. But the wishes of the readership to know what they’re subscribing to is a factor too, I suppose, not all supply but some demand also.
15 April 2011 at 16:57
[...] united blogosphere. Various bloggers, including literature scholars and archæologists and, I suppose, I, have pondered how we might take academic work to the Internet for instant response, critique and a [...]