|Year : 2016 | Volume
| Issue : 20 | Page : 2491-2496
The Content of Statistical Requirements for Authors in Biomedical Research Journals
Tian-Yi Liu1, Si-Yu Cai1, Xiao-Lu Nie2, Ya-Qi Lyu1, Xiao-Xia Peng3, Guo-Shuang Feng2
1 Department of Epidemiology and Biostatistics, School of Public Health, Capital Medical University, Beijing 100069, China
2 Center of Clinical Epidemiology and Evidence-based Medicine, Beijing Children's Hospital, Capital Medical University, Beijing 100045, China
3 Department of Epidemiology and Biostatistics, School of Public Health, Capital Medical University, Beijing 100069; Center of Clinical Epidemiology and Evidence-based Medicine, Beijing Children's Hospital, Capital Medical University, Beijing 100045, China
|Date of Submission||07-Jun-2016|
|Date of Web Publication||10-Oct-2016|
Department of Epidemiology and Biostatistics, School of Public Health, Capital Medical University, Beijing 100069
Source of Support: None, Conflict of Interest: None
Background: Robust statistical designing, sound statistical analysis, and standardized presentation are important to enhance the quality and transparency of biomedical research. This systematic review was conducted to summarize the statistical reporting requirements introduced by biomedical research journals with an impact factor of 10 or above so that researchers are able to give statistical issues' serious considerations not only at the stage of data analysis but also at the stage of methodological design.
Methods: Detailed statistical instructions for authors were downloaded from the homepage of each of the included journals or obtained from the editors directly via email. Then, we described the types and numbers of statistical guidelines introduced by different press groups. Items of statistical reporting guideline as well as particular requirements were summarized in frequency, which were grouped into design, method of analysis, and presentation, respectively. Finally, updated statistical guidelines and particular requirements for improvement were summed up.
Results: Totally, 21 of 23 press groups introduced at least one statistical guideline. More than half of press groups can update their statistical instruction for authors gradually relative to issues of new statistical reporting guidelines. In addition, 16 press groups, covering 44 journals, address particular statistical requirements. The most of the particular requirements focused on the performance of statistical analysis and transparency in statistical reporting, including "address issues relevant to research design, including participant flow diagram, eligibility criteria, and sample size estimation," and "statistical methods and the reasons."
Conclusions: Statistical requirements for authors are becoming increasingly perfected. Statistical requirements for authors remind researchers that they should make sufficient consideration not only in regards to statistical methods during the research design, but also standardized statistical reporting, which would be beneficial in providing stronger evidence and making a greater critical appraisal of evidence more accessible.
Keywords: Biomedical Research Journal; Statistical Reporting Requirement; Systematic Review
|How to cite this article:|
Liu TY, Cai SY, Nie XL, Lyu YQ, Peng XX, Feng GS. The Content of Statistical Requirements for Authors in Biomedical Research Journals. Chin Med J 2016;129:2491-6
|How to cite this URL:|
Liu TY, Cai SY, Nie XL, Lyu YQ, Peng XX, Feng GS. The Content of Statistical Requirements for Authors in Biomedical Research Journals. Chin Med J [serial online] 2016 [cited 2017 Oct 24];129:2491-6. Available from: http://www.cmj.org/text.asp?2016/129/20/2491/191822
| Introduction|| |
Statistics is the study of the collection, analysis, interpretation, presentation, and organization of data. Robust statistical designing, sound statistical analysis, and standardized presentation are important to enhance the quality and transparency of biomedical research. Therefore, most medical journals introduce their statistical reporting guideline so that researchers give serious considerations to statistical issues not only at the stage of data analysis but also at the stage of methodological design. In addition, statistical advice offered in the instructions for authors can help editors, peer reviewers, and readers evaluate the risk of bias and statistical validity.
The International Committee of Medical Journal Editors (ICMJEs) published a set of recommendations for the conduct, reporting, editing, and publication of scholarly work on manuscripts being prepared for submission to medical journals since 1979. However, the guideline for presenting and writing statistical issues was not included until 1988. At the same time, Guidelines for Statistical Reporting in Articles for Medical Journals, Amplifications and Explanations was published. Moreover, in 2013, the National Institutes of Health (NIH), as an organization of the nation's medical research agency, addressed Principles and Guidelines for Reporting Preclinical Research (NIHPG) to enhance the reproducibility of biomedical study., In 2015, the guideline of Statistical Analyses and Methods in the Published Literature (SAMPL) was published. Adhering to basic principles for reporting statistical analyses addressed by ICMJE, SAMPL puts further emphasis on how to report basic statistical methods and results with more details.
Moreover, statistical analyses and reporting are also closely related to the design and activities of the research itself. Besides specified guideline for statistical reporting, a series of guidelines were developed to enhance the quality and transparency of health research by EQUATOR network, mainly including the Consolidated Standards Of Reporting Trials (CONSORT), the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE), the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), the Standards for Reporting of Diagnostic Accuracy (STARD), and the Reporting of In vivo Experiments for Animal Research (ARRIVE). These guidelines provide recommendations for reporting based on the design type of biomedical studies with intention to address all quality issues in medical publishing.
Furthermore, different biomedical journals have distinctive requirements for statistical reporting besides the above-authorized guidelines. However, more requirements result in more researchers not knowing the appropriate journal for submission. We aim to refine common issues among different requirements introduced by the biomedical journals by a systematic review so that the statistical reporting guidelines can be easier to understand and apply.
| Methods|| |
Biomedical journals with impact factors (IFs) of 10 or above selected from the journal list issued by 2013 Journal Citation Reports® were included in the study. Only biomedical journals were included in the study. In addition, the review journals were excluded from the study due to few statistical requirements for authors in these journals. For those journals without clear identifier of statistical requirements, we decided whether it could be included after looking through its website or querying to its editor by email.
Searching statistical instruction for authors
To get the authority information of the detailed statistical instruction for authors, we linked into the homepage of each included journal and downloaded the instruction for authors. For some journals without clear statistical requirements in the instruction for authors, we contacted the editors by emails to make sure whether there were other requirements for authors about statistical reporting. To ensure that all of statistical instructions could be collected, the list of journals which used ICMJE, NIHPG, and CONSORT were checked by looking through the website of ICMJE (http://www.icmje.org/journals-following-the-icmje-recommendations/), NIHPG (http://www.nih.gov/about/endorsing-jounals.htm), and CONSORT (http://www.consort-statement.org/about-consort/consort-endosement/consort-endosers---journals/).
A data extract form was predesigned referring to Guidelines for Statistical Reporting in Articles for Medical Journals, Amplifications and Explanations published on Annals of Emergency Medicine, and SAMPL., The last version of data abstract form was used by two reviewers (Tian-Yi Liu and Si-Yu Cai) independently after pilot trial. The extracted data included journal's IF, and whether the journal following the ICMJE, NIHPG, SAMPL, CONSORT, STROBE, PRISMA, STARD, ARRIVE, or any particular statistical requirement. Meanwhile, all details of statistical requirements presented in the instruction for authors were abstracted. Data management was performed using Microsoft Office Excel 2007 (Excel version 14, Microsoft Corp.).
First, we described the types and numbers of statistical guidelines introduced by different press groups. The improvement and update of statistical instructions for authors were summarized by years. After that, we also summarized the particular statistical requirements in frequency, and all of requirements were grouped into three categories, i.e., design, method of analysis, and presentation.
| Results|| |
In the 2013 Journal Citation Reports® , there were 174 journals with IF of 10 and above. We excluded 71 journals, those were not biomedical journals (n = 103). Thirty-nine review journals were also excluded from this study and 64 medical research journals were included in this systematic review. In addition, Chinese Medical Journal was included considering as one of the most influential journals in China. In summary, 65 original journals, belonging to 23 press groups, were screened to abstract statistical requirements in instruction for authors [Supplementary Table 1] [Additional file 1]. Of 65 journals, 63 addressed statistical instructions. Only two biomedical journals did not introduce any statistical reporting guideline for authors [Supplementary Figure 1] [Additional file 2].
|Figure 1: Improvements and supplements of statistical requirements addressed by different published guidelines.|
Click here to view
[Table 1] shows the common items of statistical requirements issued by different reporting guidelines for medical research. The most common issues included "outcomes and estimation (confidence interval [CI])," "statistical methods explain," "ancillary analyses methods (subgroup)," "research design explanation," and "participants inclusion and exclusion criteria." In addition, the emphasized items involved in specified design types were different in each reporting guideline. For example, item of "Blinding methods" was required by NIHPG, CONSORT (for randomized controlled trial [RCT]), and STARD (for diagnostic test).
|Table 1: Common items of statistical requirements issued by different reporting guidelines for biomedical research|
Click here to view
The improvement and update of statistical instructions for authors are summarized in [Figure 1]. ICMJE statistical guideline published in 1988 listed 15 statements to provide basic reporting tips for preparing manuscripts, including "describe statistical methods, report exact P value and measurement error or uncertainty (CI), describe randomization and blinding," and others. Considering the progress of clinical research, a series of reporting statements have been issued by EQUATOR network since 1996 which substantially expanded requirements referring to more commonly used methods, including subgroup analysis, adjusted analysis, sensitivity analysis, and dealing with missing data. At the same time, NIHPG, focusing on the performance of statistical analysis and transparency in reporting, was published to enhance the reproduction and internal validity of research. The latest published SAMPL included more detailed requirement according to the methods, such as association analysis, correlation analysis, survival analysis, and Bayesian analysis.
The particular requirements addressed by 44 journals were summed up as frequency of issues relevant to design [Figure 2], method of analysis [Figure 3], and presentation [Figure 4]. When the particular requirement was addressed by a greater number of journals, it was deemed as more important. [Figure 2] shows issues relevant to research design, including participant flow diagram, eligibility criteria, and sample size estimation. [Figure 3] highlights that novel methods should be explained as possible as in detail and standard works reference when explaining the method of analysis. On the other hand, advanced analysis methods, including multivariable analysis, Bayesian analysis, longitudinal analysis, and Cox model, were emphasized by more journals compared with ICMJE guidelines. As shown in [Figure 4], reporting actual outcomes and exact P value was emphasized by many journals.
|Figure 2: Frequency of particular statistical issues relevant to research design.|
Click here to view
|Figure 3: Frequency of particular statistical issues relevant to method of analysis.|
Click here to view
|Figure 4: Frequency of particular statistical issues relevant to presentation.|
Click here to view
| Discussion|| |
It was shown that complying with reporting guidelines strictly can improve the quality and reliability of the research effectively. Moher et al. compared the quality score for reports of RCTs published in 1994 (pre-CONSORT) and in 1998 (post-CONSORT), which showed that the use of the CONSORT statement is associated with improvements in the quality of reports of RCTs. In 2014, Choi et al. conducted a systematic review to assess the endorsement of reporting guidelines in traditional medicine journals in Korea. They suggested that it is necessary for traditional medicine journals to adopt reporting guidelines because better reporting is likely to influence the quality as well as the effect of future research.
A majority of biomedical journals introduce statistical reporting guideline issued by NIHPG for authors, which focuses on the interpretation and repetition of biomedical experiments. Compared with ICMJE and NIHPG, those focus on not only statistical reporting but also research design, SAMPL guideline issues statistical requirements in details referring to different statistical analysis methods to help authors understand the factors of better statistical reporting.
On the other hand, a series of reporting guidelines issued by EQUATOR network are based on different types of research. Therefore, specified statistical requirements in each research design are different. For example, CONSORT statement focuses on sufficient randomization, estimation of sample size, dataset used in statistical analysis (i.e., intention-to-treat or per-protocol), and subgroup analysis. STROBE statement mentions statistical method used to control confounding factors, explain missing data, and solve matching strategy, and methods used to examine subgroups and interactions. STARD statement for diagnostic test emphasizes reporting methods for estimating or comparing diagnostic accuracy, dealing with indeterminate index test or reference standard results, any analysis of variability in diagnostic accuracy, and distinguishing the prespecified from exploratory.
The contents of guidelines have been revised again and again with adding explanation and elaborations to refine the usability as well as to amplify the range of application., This study demonstrates that besides 7 statistical reporting guidelines issued by NIHPG, ICMJE, and EQUATOR, 68% of journals with IF of 10 or above still introduce some particular statistical requirements for authors. The most common statistical requirements remind us that authors should consider the statistical requirements at the beginning of research design rather than only at the stage of statistical analysis. As Ronald said, "to consult the statistician after an experiment is finished is often merely to ask him to conduct a postmortem examination. He can perhaps say what the experiment died of." It is worthy for authors to note that statistical requirements are required more with the improvements of research methods, especially the rapid development of clinical research.
Schriger et al. analyzed the methodological and statistical general contents addressed in instructions for authors offered by better quality medical journals and analyzed the length of journals' instructions for authors. Compared with the year 2006, there is prominent improvement referring to statistical requirements for authors at present. First, more guidelines have been published by different organizations and also are adopted by more biomedical journals up to now. In 2006, only 22 of 166 journals mentioned CONSORT, 4 of 166 addressed other guidelines such as Quality of Reporting of Meta-analyses (QUOROM) or STARD. Moreover, few journals provided requirements about statistical methods and reporting. Nowadays, the types of reporting guidelines introduced by journals are increasing obviously. Second, some statements have been updated and refined timely. For example, the QUOROM statement was updated into PRISMA, which contains more details about reporting systematic review and meta-analysis. Third, a higher number of journals pay more attention to methodology issues and provide particular advice correspondingly to their instruction for authors. Compared with Schriger et al.'s study, we focused on specific content referring to methodological and statistical reporting requirements, which could be of benefit to authors to achieve completeness of statistical reporting.
Furthermore, the improvement and update of statistical requirements indicate the rapid growth of research method and statistical analysis. In recent decades, advanced statistical requirements are applied, especially in cohort study with large-size samples or observation studies to identify the relative risk, and to describe characteristics of population; for example, dealing with missing data and outliers, estimation of sample size, Bayesian analysis, multivariable analysis, Cox models, and longitudinal analyses. On the other hand, there also are timely supplements in terms of particular requirements.
As for Chinese Medical Journal, one of the most influential journals issued by the Chinese Medical Association covers a broad range of fields of biomedical research in China. It also recommends 6 kinds of statistical guidelines and mentions particular requirements. However, statistics errors in biomedical journals remained commonly despite that many editors and authors made a substantial effort. Therefore, explanation and elaboration of statistical reporting guidelines, aiming to improve the compliance of reporting guidelines and to enhance the quality of researches sequentially, is imperative.,,,
However, the study still has some limitations. Considering the academic authority, we focused on the biomedical journals with IF of 10 or above. However, it was likely that some journals with higher impact on specific areas were missed. On the other hand, journals on public health, especially on epidemiology and biostatistics, those focusing on the breakthrough of methodology, were also missed. In fact, many press groups publish several journals, including journals on epidemiology and biostatistics with <10 of IF. Journals issued by one press group usually use the same reporting guideline for medical research.
In summary, statistical requirements for authors are becoming increasingly perfected. All these improvements could be contributed to the push of evidence-based medicine, which focuses on the critical appraisal quality of existing researches and the standardization of reporting. Furthermore, recent statistical requirements for authors remind that researchers should pay more attention to the research design and statistical reporting that would be benefit to provide stronger evidence for the development of biomedicine.
Supplementary information is linked to the online version of the paper on the Chinese Medical Journal website.
We are very grateful to all those editors who responded to the consulting E-mail: Adam Lipkin (Nature Neuroscience), Andrea Dulberger (Molecular Cell), Claire Sanders (Nature Methods), Danielle Loughlin (Trends in Cell Biology), Lawrence Latido (Elsevier Researcher Support), Miranda Dubner (Nature Biotechnology), Nicholas Perry (Cell Metabolism, Structure, Chemistry and Biology), Rebecca Schwarzlose (Trends in Cognitive Sciences), Robin Wasley (Cell Stem Cell), and Tony Ogden (Cell Host and Microbe and Immunity). We also owe gratitude to Ali Abbas for helping to polish the language.
Financial support and sponsorship
This study was supported by the grant from the Beijing Natural Science Foundation (No. 7162061). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Conflicts of interest
There are no conflicts of interest.
| References|| |
Dodge Y. The Oxford Dictionary of Statistical Terms. USA: Oxford University Press; 2003.
Bailar JC 3 rd
, Mosteller F. Guidelines for statistical reporting in articles for medical journals. Amplifications and explanations. Ann Intern Med 1988;108:266-73.
Moher D, Avey M, Antes G, Altman DG. The National Institutes of Health and guidance for reporting preclinical research. BMC Med 2015;13:34. doi: 10.1186/s12916-015-0321-8.
Collins FS, Tabak LA. Policy: NIH plans to enhance reproducibility. Nature 2014 30;505:612-3.
Lang TA, Altman DG. Basic statistical reporting for articles published in biomedical journals: The statistical analyses and methods in the published literature or the SAMPL guidelines. Int J Nurs Stud 2015;52:5-9. doi: 10.1016/j.ijnurstu.2014.09.006.
Group ICMJE. Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. Available from: http://www.icmje.com
. [Last accessed on 2014 May 10].
Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al.
CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. Int J Surg 2012;10:28-55. doi: 10.1016/j.ijsu.2011.10.001.
Von EE, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al
. Strengthening the reporting of observational studies in epidemiology (StroBE) statement: Guidelines for reporting observational studies. Int J Surg 2014;12:1495-9. doi: 10.1016/j.ijsu.2014.07.013.
Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Int J Surg 2010;8:336-41. doi: 10.1016/j.ijsu.2010.02.007.
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al.
The STARD statement for reporting studies of diagnostic accuracy: Explanation and elaboration. Ann Intern Med 2003;138:1-12. doi: 10.7326/0003-4819-138-1-200301070-00012-w1.
Kilkenny C, Browne W, Cuthill IC, Emerson M, Altman DG; National Centre for the Replacement, Refinement and Reduction of Amimals in Research. Animal research: Reporting in vivo
experiments - The ARRIVE guidelines. J Cereb Blood Flow Metab 2011;31:991-3. doi: 10.1038/jcbfm.2010.220.
Altman DG. Statistical reviewing for medical journals. Stat Med 1998;17:2661-74. doi: 10.1002/(SICI)1097-0258(19981215) 17:23%3C2661:AID-SIM33%3E3.3.CO;2-2.
Moher D, Jones A, Lepage L; CONSORT Group (Consolidated Standards for Reporting of Trials). Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA 2001;285:1992-5. doi: 10.1001/jama.285.15.1992.
Choi J, Jun JH, Kang BK, Kim KH, Lee MS. Endorsement for improving the quality of reports on randomized controlled trials of traditional medicine journals in Korea: A systematic review. Trials 2014;15:429-42. doi: 10.1186/1745-6215-15-429.
Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al.
CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:e1-37. doi: 10.1016/j.jclinepi.2010.03.004.
Ronald F. Presidential Address to the First Indian Statistical Congress. Sankhya 1938;4:14-7.
Schriger DL, Arora S, Altman DG. The content of medical journal Instructions for authors. Ann Emerg Med 2006;48:743-9. doi: 10.1016/j.annemergmed.2006.03.028.
He J, Jin Z, Yu D. Statistical reporting in Chinese biomedical journals. Lancet 2009;373:2091-3. doi: 10.1016/S0140-6736(09)60867-9.
Lin F, Han Y, Fang YA. Application status and analysis of statistical method in published literature (in Chinese). Chin J Nucl Med Mol Imaging 2014;34:401-4.
Dai XQ. Common statistical analysis misuses of qualitative data in medical journals (in Chinese). Chin J Emerg Med 2014;23:359-60.
Liu HQ, Fang JQ. Pay attention to statistical issues of medical clinical trials - Developing statistics reporting checklist. Chin Med J 2007;87:2446-8.
[Figure 1], [Figure 2], [Figure 3], [Figure 4]