UC San Diego New-Media Expert Pushes Peer Review into the 21st Century
San Diego, CA, March 21, 2008 -- The populist blog could be the future of academic publishing. In a daring attempt to rewrite – or at least edit – the rules of scholarly peer-review publishing, a professor at the University of California, San Diego this week posted the last chapter sections from his upcoming book on a web log – opening it up to comments from anyone, anywhere.
|
Communication professor Noah Wardrip-Fruin posted the manuscript of "Expressive Processing: Digital Fictions, Computer Games and Software Studies" in daily installments, averaging 2,000 words, over a ten-week period on Grand Text Auto. The GTxA blog is co-authored by Wardrip-Fruin and five other artists, game designers and academics. Its blend of commentary and high-minded discussion on videogame culture and new media attracts roughly 35,000 unique visitors each month.
"I wanted to experiment – and take an existing academic blog, where people are already talking about the kinds of things you care about, and make them part of the peer review process," said the UCSD academic. "My editor at MIT Press was very supportive of doing something like this as a way of exploring the space, but he was skeptical that I would learn anything much of value. He was also annoyed that it was positioned as a competition between blog-based and traditional peer review."
As his editor, Doug Sery, told the Chronicle of Higher Education, "the reputation of MIT Press, or any good academic press, is based on a peer review model." So even as he was giving Wardrip-Fruin the green light to post eight chapters of the manuscript on the blog, Sery sent the same chapters to three peer experts who would submit blind reviews.
|
"My argument for blog-based peer review is that it's a good thing to do in an existing online community, where people who know a lot about the subject (inside and outside the academy) are already part of the community," said Wardrip-Fruin.
"I was impressed at how important conversation turned out to be, although that should have been expected," continued Wardrip-Fruin. "In one of the first postings, someone said he felt there was something wrong with the way I was discussing The Sims™. I responded that I could probably just massage a couple of sentences and fix it. Then somebody else came in a few paragraphs later and said the problem was more with the way I connected The Sims to Chris Crawford's idea of process [vs. data] intensity. That was insightful, and I probably wouldn't have discovered that in the one-way, one-step response that you get from traditional peer review."
During the 10-week experiment, hundreds of comments were attached to the manuscript on the blog, primarily from roughly 60 core commenters.
Before the experiment began, the author feared "a lot of noise" from comments that would not really move the conversation forward. "As it turns out, there has been very low noise, and I think that has to do with the fact that we're doing it within this existing academic community that has already set the standards for what it means to comment on the Grand Text Auto blog," explained Wardrip-Fruin. "So I've got the best of both worlds: a relatively low noise threshold, but a lot more voices than I would have gotten otherwise."
|
Another benefit, according to Wardrip-Fruin, was the variety of people submitting comments: "It was important to get the views of people in the videogame industry for example, who would never participate in a traditional academic peer review."
"Expressive Publishing" explores the world of videogames, game culture and interactive fiction through the lens of an emerging critical field called 'software studies'. Wardrip-Fruin is co-director of a new Software Studies Initiative based in the UCSD division of the California Institute for Telecommunications and Information Technology (Calit2) and supported by the Center for Research in Computing and the Arts (CRCA), which hosts the GTxA blog.
|
Specifically, he says, the reviewers on the blog felt that they lost sight of the overarching themes of the book, and suggested that the broad argument – now primarily detailed in the first chapter – be threaded through the individual chapters more.
The traditional reviews reached the same conclusion, but for different reasons. "All three of the blind peer reviewers had a very clear sense of the overall argument of the book," said Wardrip-Fruin. "But they strongly disliked the first chapter, which lays out the argument in a compressed, rapid-fire way, depending on readers to get through one challenging chapter and then remember it for the rest of the book." The blind peer review comments therefore also point toward presenting the main body of the argument gradually, over the course of the book's chapters, rather than compressing so much of it into the first chapter.
The author believes that blog-based peer review will never completely take the place of the traditional kind. But, he argues, the blog format works well "in an existing online community where people who know a lot about the subject are already part of the community." The blog-based review is particularly useful, Wardrip-Fruin adds, in fields where non-academics such as game developers, industry researchers and community historians can offer insights that might never occur to an academic.
In those cases, he argues, blog-based and traditional peer review can be complementary and valuable.
"Frankly, I hope I won't be the last person to do this," concluded Wardrip-Fruin. "I'm hoping we'll see more experiments of this sort once we release the software for other people to use. When that happens, we'll hopefully be able to write up some recommendations for when this kind of peer review can be a powerful addition to traditional peer review, and what sort of audiences we think should pursue it and how."
Until then, the UC San Diego professor can pride himself on making a believer of one important reviewer who left at least one comment on the blog. That reviewer: Doug Sery – his editor at MIT Press.
Related Links
Grand Text Auto
Noah Wardrip-Fruin Website
Institute for the Future of the Book
Software Studies @ UCSD
California Institute for Telecommunications and Information Technology
Center for Research in Computing and the Arts
Media Contacts
Media Contact: Doug Ramsey, 858-822-5825, dramsey@ucsd.edu