Notes

  1. A very common example for this type of research data is the German Socio-Economic Panel Study (SOEP): http://www.diw.de/en/diw_02.c.221178.en/about_soep.html
  2. A prominent example for irreproducible/incorrect research in the recent past that massively influenced policy-makers is the case of Rogoff and Reinert which leads to my controversial discussions in the community. (see http://www.bloomberg.com/news/2013-04-28/refereeing-the-reinhart-rogoff-debate.html).
  3. In economic journals, two types of data policies can be distinguished: A replication policy requires authors to provide data and code to would-be replicators. In contrast a data availability policy mandates authors to provide data and code to the journal. The journal provides this information to would-be replicators by attaching the data and code to the article (often in “supplementary information”). The terms replication policy and data availability policy have been defined in: McCullough, McGeary, & Harrison (2008).
  4. A useful attempt has been implemented by the NEREUS-Network in the course of the “Network of European Economists Online” (NEEO) project (http://www.economistsonline.org/home): NEEO had a runtime from 2007 till 2010. In this project existing research resources such as RePEc (http://www.repec.org) and new content of excellence from over 1000 top economics scholars, are made available through the Economists Online portal. Though the portal provides bibliographic information for more than 900,000 research articles and access to several thousand full-texts, there are only 142 datasets (and even a much smaller number of datasets accompanied by the code of computation) available in the NEEO Dataverse (http://dvn.iq.harvard.edu/dvn/dv/NEEO).
  5. The sole publication-related data archive we were able to find in the course of the project is currently available at the ICPSR (Inter-university Consortium for Political and Social Research): http://www.icpsr.umich.edu/icpsrweb/deposit/pra/index.jsp
  6. http://www.edawax.de
  7. Results of this work package are available at http://www.edawax.de/wp-content/uploads/2013/01/EN-EDaWaX-Online-Survey-Hosting-Options_blog.pdf
  8. For additional information on the EDaWaX project please visit the project web blog: www.edawax.de
  9. Data archive of the AER: http://www.aeaweb.org/aer/contents/index.php
  10. For the background of this recommendation I also want to refer to the case of Oberholzer-Gee/Strumpf vs. Liebowitz. http://regulation2point0.org/wp-content/uploads/downloads/2010/05/The-Oberholzer-Gee-Strumpf-File-Sharing-Instrument-Fails-the-Laugh-Test.pdf
  11. To determine the average impact factor and ranking for groups of journals with different data policies, the following numbers of journals have been included in our analyses:
    • For analyzing the ranking within the JCR 21 journals equipped with a data availability policy (72.4% of all journals with such a policy in our sample), 11 journals equipped with a replication policy (100% of all journals with such a policy in our sample) and 74 journals without a data policy (73.3% of all journals with such a policy in our sample) have been included.
    • For the analyses regarding the Handelsblatt ranking 28 journals with a data • availability policy (96.6%), 11 journals (100%) with a replication policy and 96 journals without a data policy (95.1%) were included.
    • For the analyses regarding the ranking of BHM, 26 journals with a data availability policy (89.7%), 11 journals (100%) with a replication policy and 100 journals without a data policy (99%) were included.
    The journals not included could not be used for these calculations, because they were not listed.
  12. In the course of our analysis we found a case where the data policy is available in the printed edition only (German Economic Review). For other journals we were able to find a data archive, but not a data policy (e.g. Jahrbücher Nationalökonomie und Statistik, Journal of Financial Economics, The Federal Reserve Bank of St. Louis Review). These cases were not included into the analysis of the data availability policies but were categorized as journals without data policy.
  13. The original wording of all data availability and replication policies we found in the course of our analyses is available on the project blog: http://www.edawax.de/wp-content/uploads/2012/07/Data_Policies_WP2.pdf
  14. The remaining journals did not request authors to submit datasets because they are focussed on experimental data. In these journals providing of data was optional.
  15. An example for a data archive in economic scholarly journals (here the data archive of the American Economic Review) is available here: http://www.aeaweb.org/aer/contents/index.php
  16. Readers interested in Dataverse should visit the Dataverse homepage (www.thedata.org) for more information. An interesting overlook on Dataverse was provided by Mercè Crosas, Director of Product Development, in a workshop on Persistent Identifiers in Berlin at May the 8th 2012. Her presentation is available here: http://www.ratswd.de/ver/docs_PID_2012/Crosas_PID2012.pdf
  17. A good example for a dataset and code in Dataverse is available here: http://dvn.iq.harvard.edu/dvn/dv/arzheimer/faces/study/StudyPage.xhtml?globalId=hdl:1902.1/12092&studyListingIndex=0_e757de6b960f442ef22a63c6b03a
  18. Beside the JAE, also the Journal of Economic and Social Measurement and the International Journal of Research in Marketing own a replication section. In addition the Public Finance Review published a call for replication studies in 2010 (Burman, Reed, & Alm, 2010).
  19. The information system PANGAEA (http://www.pangaea.de) is operated as an open-access library aimed at archiving, publishing and distributing georeferenced data from earth system research. Inter alia PANGAEA was able to conclude an agreement with Elsevier with the result that the research data used within an article and available at PANGAEA is shown on the website of the research article published in an Elsevier Journal.
  20. Dryad is a discipline-specific research data repository for the basic and applied biosciences: www.datadryad.org.
  21. The homepage of da|ra is http://www.da-ra.de/en/home/
  22. Further Information on Datacite, that is part of the international DOI-Foundation (IDF) may be obtained on the website http://datacite.org/
  23. http://www.gesis.org
  24. For Germany, the RatSWD (German Data Forum: http://www.ratswd.de) has formulated some criteria for research data centres. For obtaining certification from the German Data Forum they have to fulfil some requirements, listed on the webpage http://www.ratswd.de/download/publikationen_rat/RatSWD_FDZCriteria.pdf
  25. An online-survey among German and European research data centres supports this finding. For more information on the results of this survey, please visit http://www.edawax.de/wp-contentt/uploads/2013/01/EN-EDaWaX-Online-Survey-Hosting-Options_blog.pdf
  26. Examples of software solutions comprise Nesstar (http://www.nesstar.com/), Dataverse (http://thedata.org/) or CKAN (http://ckan.org/) exemplarily
  27. Some information on Mantra can be obtained via the website of the University of Edinburgh: http://www.ed.ac.uk/schools-departments/information-services/about/organisation/edl/data-library-projects/mantra