Replication: enough of the cheap talk

Most economists agree that replicating other's work is worthwhile. But it's just "cheap talk", according to Daniel Hamermesh:

The cost of pure and scientific replication in economic research has diminished over the past forty years. At the same time our verbal nods toward the need for replication have increased... Despite the declining costs and despite our expressions of preferences, both pure and scientific replications are very rare in leading journals; and even within-study scientific replication is unusual.

Our expressions of preference are cheap talk. The profession provides few incentives for most active economists to produce replications of others’ research, and similarly few to increase the believability of one’s own research by testing ideas on multiple sets of data. Any demand for replication must arise from the profession as a whole, as intermediated by the actions and decisions of the editors of the most visible professional journals.

What is to be done?

Editors need to take the lead by providing sufficient incentives for top-flight authors of empirical work to engage in pure and scientific replication, and by insisting on within-study scientific replication in many more articles. Without these changes occasional paeans to the virtues of replication are as likely to enhance the scientific soundness of empirical research in economics as programmes that urge abstinence are to reduce teenagers’ sexual activity.

These quotes come from his new IZA discussion paper, Replication in Economics, forthcoming in the Canadian Journal of Economics. These issues matter, so it's good to see someone of Hamermesh's stature raising them.

(See also my previous post: Does empirical economics follow the scientific method?)