|Year : 2011 | Volume
| Issue : 2 | Page : 85-89
Quality of reporting statistics in two Indian pharmacology journals
Jaykaran, Preeti Yadav
Department of Pharmacology, Govt. Medical College, Surat, Gujarat - 395 001, India
|Date of Web Publication||6-Jun-2011|
Department of Pharmacology, Govt. Medical College, Surat, Gujarat - 395 001
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Objective: To evaluate the reporting of the statistical methods in articles published in two Indian pharmacology journals. Materials and Methods: All original articles published since 2002 were downloaded from the journals' (Indian Journal of Pharmacology (IJP) and Indian Journal of Physiology and Pharmacology (IJPP)) website. These articles were evaluated on the basis of appropriateness of descriptive statistics and inferential statistics. Descriptive statistics was evaluated on the basis of reporting of method of description and central tendencies. Inferential statistics was evaluated on the basis of fulfilling of assumption of statistical methods and appropriateness of statistical tests. Values are described as frequencies, percentage, and 95% confidence interval (CI) around the percentages. Results: Inappropriate descriptive statistics was observed in 150 (78.1%, 95% CI 71.7-83.3%) articles. Most common reason for this inappropriate descriptive statistics was use of mean ± SEM at the place of "mean (SD)" or "mean ± SD." Most common statistical method used was one-way ANOVA (58.4%). Information regarding checking of assumption of statistical test was mentioned in only two articles. Inappropriate statistical test was observed in 61 (31.7%, 95% CI 25.6-38.6%) articles. Most common reason for inappropriate statistical test was the use of two group test for three or more groups. Conclusion: Articles published in two Indian pharmacology journals are not devoid of statistical errors.
Keywords: Inappropriate statistics, Indian Journals, Pharmacology
|How to cite this article:|
Jaykaran, Yadav P. Quality of reporting statistics in two Indian pharmacology journals. J Pharmacol Pharmacother 2011;2:85-9
| Introduction|| |
Statistics is a tool in the hand of a researcher by which he can analyze his study findings. If statistics methods used in the study are inappropriate, the conclusions drawn from the study become questionable. Studies with poor methodological quality and poor statistics cannot prove or disprove study hypothesis with certainty. So conduction of these kind of studies raises many ethical issues like exposure of participants to risk of new intervention, deprivation of participants to established treatment, unnecessary use of animals in experimental studies, misuse of resources, and wrong clinical judgments on the basis of these studies once they get published. ,,,, Despite publication of various guidelines related to the reporting of various methodological and statistical parameters of a study, it has been observed that quality of statistical reporting is poor in various biomedical journals. , Various surveys done for the articles published in western medical journals indicate that statistical error in the published article is a common phenomenon and error rate may vary from 30% to 90%. ,,, Although many surveys are done for statistical reporting in western journals, data are lacking for studies published in Indian medical journals. Some small studies done for articles published in Indian medical journals observed the same phenomenon of poor reporting of various statistical parameters. , It is observed that data related to the statistical reporting of articles published in pharmacology journals of India are lacking. So this study was designed with the aim of evaluating articles published in Indian pharmacology journals (Indian Journal of Pharmacology (IJP) and Indian Journal of Physiology and Pharmacology (IJPP)) for statistical reporting. These two pharmacology journals are widely circulated and Pubmed-indexed Indian pharmacology journals; hence they were selected for evaluation.
| Materials and Methods|| |
All articles published in IJP and IJPP between 2002 to the latest issue of 2010 were downloaded from journals website ( www.ijp-online.com and www.ijpp.com ). In case of IJPP, articles published since 2002 were available on website. So to maintain uniformity for both journals, all articles which were published in or after 2002 were downloaded. Only original studies were considered for analysis. Short communications, research letters, and letter to editors were not taken into account. In case of IJPP, only articles related to pharmacology were downloaded. All articles were evaluated independently by first (J.K.) and second author (P.Y.). These articles were appraised for quality of reporting of descriptive statistics and quality of reporting of inferential statistics. Descriptive statistics is evaluated on the basis of appropriate reporting of data as mean, median, or frequency with the central tendencies. Inferential statistics was evaluated on the basis of reporting of assumptions of statistical tests and inappropriateness of statistical tests. Common methods of statistical analysis were also noted. Common reasons for inappropriate descriptive statistics and common reasons for inappropriate statistical tests were also noted. Any disagreements between two authors were resolved by consensus (k = 0.87 for inappropriate statistical tests). Appropriate method of descriptive statistics of ratio and interval data following the normal distribution is mean (SD) or mean ± SD. For ordinal data and for ratio and interval data not following the normal distribution, appropriate descriptive statistics is median and interquarantile range and for nominal data, frequency and percentage are appropriate. Appropriate statistical tests are selected on the basis of aim of the study and types of data. Once the statistical test is selected, all the assumptions for that particular statistical test should be checked before applying that statistical test.
Values are described as frequencies, percentages, and 95% confidence interval around percentages.
| Results|| |
Total 196 articles from various areas of research were downloaded from the journal sites. Major areas of research were diabetes (39 (19.8%) studies), central nervous system (17 (8.6%)), hepatoprotection (18 (9.1%)), and cardiovascular (17 (8.6%)). Other areas were inflammation (11 (5.6%)), antioxidants (9 (4.5%)), pain (7 (3.5%)), gastrointestinal (6 (3%)), and immunomodulation (4 (2%)). Most of the articles were dealing with animal studies (83% vs. 17%).
Out of these 196 articles, information related to descriptive statistics was missing in four articles. Out of remaining 192 articles, inappropriate descriptive statistics was reported in 150 (78.1%, 95% CI 71.7-83.3%) articles. Out of these 150 studies 106/129 (82.1%) were from IJP and 44/63 (69.8%) from IJPP.
Most common reason for inappropriate reporting of descriptive statistics was the use of mean ± SEM at the place of "mean (SD)" or "mean ± SD" [Table 1].
|Table 1: Inappropriate descriptive statistics in articles published in two Indian pharmacology journals (n = no. of articles)|
Click here to view
Most common type of statistical method used in the articles of both pharmacology journals was "one-way analysis of variance (ANOVA)" [Table 2]. Out of 214 statistical methods only 10.7% were nonparametric methods.
|Table 2: Statistical methods used in articles published in two Indian pharmacology journals|
Click here to view
Assumptions of statistical tests
Information related to fulfillment of assumptions of statistical tests was mentioned in only two articles. In one article, normal distribution was checked by Komolgorov-Smirnov test.
Inappropriate statistical tests
Out of 196 articles from both journals, information related to statistical test was missing in four articles. Out of remaining 192 articles inappropriate statistical tests were found in 61 (31.7%, 95% CI 25.6-38.6%) articles. Most common reason for inappropriate statistical test was use of two group test for analysis of three or more than three groups (22.9%) [Table 3].
|Table 3: Inappropriate statistical tests in articles published in two Indian pharmacology journals (n = no. of articles)|
Click here to view
| Discussion|| |
Main findings of this study are as follows: majority of articles published in two Indian pharmacology journals have inappropriate reporting of descriptive statistics, assumption of statistical tests were checked in only two article, and inappropriate statistical tests was used to analyzed data in 31.7% of articles.
One major finding was inappropriate use of "mean ± SEM" for description of data. The ideal method of reporting of these kinds of data is "mean (SD)" or "mean ± SD." Although SD and SEM look similar, they give different information.  Standard deviation (SD) shows variability around the mean within the sample and standard error of mean (SEM) shows probability of proximity of sample mean around the population mean.  Readers and researchers are interested in knowing variability within the sample not the proximity of mean to the population mean. The value of SEM is always less than SD so when it is used as descriptive statistics readers may falsely conclude that variability of sample is small. To prevent confusion with CI in the place of "mean ± SEM" reporting as mean (SD) is a better method.  Similar findings were also observed in other studies done for western and Indian journals. In a study done by Negele (2001) for the articles published in four anesthesia journals, it was observed that inappropriate use of SEM was present in 23% articles.  In a similar study done for four Indian medical Journals by Saurabh et al. (2010), it was observed that inappropriate reporting of SEM was common in articles published in basic science journals but this inappropriateness was negligible in journals related to clinical practices.  In spite of highlighting this issue in various surveys, this practice of reporting the variability as SEM is common and is a matter of concern. ,, Ordinal data like scores or scales are sometimes described as "mean ± SEM" which is wrong as they should be reported as median (range).  This error was not much observed in this study as majority of data were in ratio scale but in some other studies this error found to be much more. 
In this study, majority of statistical tests were parametric tests. Nonparametric tests were used less frequently (10.7%). It has been observed that the use of nonparametric statistics is increasing regularly in articles published in medical journals.  Low proportion of nonparametric statistics may be because of ignoring of assumptions underlying parametric statistics by authors.  Most of the articles in this study were animal experiments where usually many groups are used for comparison; hence one-way ANOVA was most frequently used statistical method whereas in studies done for articles published in clinical journals student t test seems to be the most common method.  Most of the statistical methods were simple methods and sophisticated methods like survival analysis, multiple regressions were not observed. In this study, it is found that three statistical tests - one-way ANOVA, unpaired t test, and paired t test - cover about 82% of all statistical methods, so these are the most frequently used tests and interpretation of these tests should be taught in detail to postgraduate students and young researchers.
In this study, it was observed that fulfilling of assumptions of statistical tests was not reported in almost all the studies. One reason may be underreporting and second reason may be ignorance of researcher. Each statistical test has some assumptions and these assumptions need to be fulfilled before application of that statistical test. Information regarding fulfilling of these assumptions should be included in the manuscript. Similar observation was made in other studies. 
About 32% articles have at least one inappropriate statistical test and most frequent mistake was the use of two group test for comparison of three or more groups like use of unpaired t test for comparison of three unpaired groups. This problem was observed in other studies done for statistical reporting in western journals. ,, Frequency of statistical errors varies from journal to journal like for Chinese journals it is 46%,  for surgical journal it is 64%,  and for urology journals it is 28%.  Most common problem was the use of multiple unpaired t tests at the place of one-way ANOVA. Despite repeated recommendations, unpaired t test still continues to be used at the place of ANOVA,  which is a matter of concern. Another mistake observed was the use of parametric statistical tests for ordinal data like scores or scales. It is very important to understand that ordinal data do not follow the normal distribution. Hence the use of parametric tests for these kinds of data is not justifiable.  In a study, it was found that ordinal data were used in about one-third of articles and these data are appropriately presented and analyzed in 50% articles.  In this study, most of the articles were dealing with continuous variables so this finding is not as prominent as observed in other journals.
There may be various reasons for finding these kinds of statistical errors in the published articles like insufficient knowledge of statistics and research methodology in researcher, , insufficient ethical review of protocol submitted for permission from institutional ethics committee, insufficient peer review of submitted manuscript, and less knowledge of statistics in journal editors. It is observed that in ethics committee statistical issues are not discussed in detail as members of ethics committee usually focus their attention on informed consent, etc. It is important to understand that poor-quality research is also unethical. So ethics committee should also have a qualified medical statistician who can give advice regarding the methodological and statistical aspects of the protocol.  Every article submitted to the journal should also be sent for statistical review and journals should have statistical advisors in their editorial board. It is observed that many journals do not have statistical advisors.  Postgraduate students and young researchers should be trained in research methodology and biostatistics. Research methodology should be incorporated in the curriculum of postgraduate course.
This study has some limitations. One of the major limitations is that focus of this study is very narrow. Only few but very important statistical parameters were observed. Parameters like post hoc power, adjustment of multiple endpoints, sample size calculation, confidence interval, use of exact P value etc. were not taken into consideration. Second limitation is only two pharmacology journals were considered for evaluation. As far as our perception goes, this is the first study done for articles published in Indian pharmacology journals and may be at international level.
This study shows that inappropriate statistics is very common in the articles published in Indian pharmacology journals. Measures should be taken by journal editors, ethics committee, and researchers to prevent these errors.
| References|| |
|1.||Altman DG. Statistics and ethics in medical research: Misuse of statistics is unethical. BMJ 1980;281:1267-9. |
|2.||Halpern S, Karlawish JH, Berlin JA. The continuing unethical conduct of underpowered clinical trials. JAMA 2002;288:358-62. |
|3.||Altman DG: Statistics and ethics in medical research: III. How large a sample? BMJ 1980;281:1336-8. |
|4.||Newell DJ. Type II errors and ethics. BMJ 1978;23:1789. |
|5.||Altman DG. The scandal of poor medical research. BMJ 1994;308:283-4, |
|6.||Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research. PLoS Biol 2010;8: e1000412. |
|7.||Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Ann Intern Med 2001;134:663-94. |
|8.||Altman DG. Statistics in medical journals: Developments in the 1980s. Stat Med 1991;10:1897-913. |
|9.||Gore SM, Jones IG, Rytter EC. Misuse of statistical methods: Critical assessment of articles in the BMJ from January to March 1976. BMJ 1977;1:85-7. |
|10.||Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials. A survey of three medical journals. N Engl J Med 1987;317:426-32. |
|11.||MacArthur RD, Jackson GG. An evaluation of the use of statistical methodology in the Journal of Infectious Diseases. J Infect Dis 1984;149:349-54. |
|12.||Karan J, Goyal JP, Bhardwaj P, Yadav P. Statistical Reporting in Indian Pediatrics. Indian Pediatr 2009;46:811-2. |
|13.||Jaykaran, Kantharia N, Yadav P, Bharddwaj P. Reporting statistics in clinical trials published in Indian Journals: A survey. Pak J Med Sci 2010;26:212-6. |
|14.||Carlin JB, Doyle LW. Basic concepts of statistical reasoning: Standard errors and confidence intervals. J Paediatr Child Health 2000;36:502-5. |
|15.||Curran-Everett D. Explorations in statistics: Standard deviations and standard errors. Adv Physiol Educ 2008;32:203-8. |
|16.||Jaykaran. "Mean ± SEM" or "Mean (SD)"?. Indian J Pharmacol 2010;42:329. |
|17.||Nagele P. Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. Br J Anaesth 2003;90:514-6. |
|18.||Saurabh MK, Jaykaran N, Chavda P, Yadav ND, Kantharia N, Kumar L. Misuse of standard error of mean (SEM) when reporting variability of a sample: A critical appraisal of four Indian journals. J Pharm Res 2010;3:277-9. |
|19.||Streiner DL. Maintaining standards: Differences between the standard deviation and standard error, and when to use each. Can J Psychiatry 1996;41:498-502. |
|20.||Altman DG, Gore SM, Gardner MJ, Pocock SJ. Statistical guidelines for contributors to medical journals. Br Med J 1983;286:1489-93. |
|21.||Avram MJ, Shanks CA, Dykes MH, Ronai AK, Stiers WM. Statistical methods in anesthesia articles: An evaluation of two American journals during two six-month periods. Anesth Analg 1985;64:607-11. |
|22.||Kurichi JE, Sonnad SS. Statistical methods in the surgical literature. J Am Coll Surg 2006;202:476-84. |
|23.||Qian Wang MS, Boheng Zhang MS. Research Design and Statistical Methods in Chinese Medical Journals. JAMA 1998;280:283-5. |
|24.||Al-Benna S, Al-Ajam Y, Way B, Steinstraesser L. Descriptive and inferential statistical methods used in burns research. Burns 2010;36:343-6. |
|25.||Rushton L. Reporting of occupational and environmental research: Use and misuse of statistical and epidemiological methods. Occup Environ Med 2000;57:1-9. |
|26.||Simundic AM, Nikolac N. Statistical errors in manuscripts submitted to Biochemia Medica Journal. Biochemia Medica 2009;19:294-300. |
|27.||Morris RW. A statistical study of papers in the journal of bone and joint surgery (BR) 1984. J Bone Joint Surg Br 1988;70:242-6. |
|28.||Scales CD Jr, Norris RD, Peterson BL, Preminger GM, Dahm P. Clinical research and statistical methods in the urology literature. J Urol 2005;174:1374-9. |
|29.||Spina D. Statistics in Pharmacology. Br J Pharmacol 2007;152:291-3. |
|30.||LaValley MP, Felson DT. Statistical Presentation and Analysis of Ordered Categorical Outcome Data in Rheumatology Journals. Arthritis Rheum 2002;47:255-9. |
|31.||Jakobsson U. Statistical presentation and analysis of ordinal data in nursing research. Scand J Caring Sci 2004;18:437-40. |
|32.||Altman DG, Goodman SN, Schroter S. How statistical expertise is used in medical research. JAMA 2002;287:2817-20. |
|33.||Chanter DO. Maintaining the integrity of the scientific record: New policy is unlikely to give investigators more control over studies. BMJ 2002;324:169. |
|34.||Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA 2000;283:2701-11. |
|35.||Goodman SN, Altman DG, George SL. Statistical reviewing policies of medical journals: Caveat lector? J Gen Intern Med 1998;13:753-6. |
[Table 1], [Table 2], [Table 3]
|This article has been cited by|
||Reporting statistics in biomedical research literature: The numbers say it all
| ||Amir Maroof Khan,Siddarth Ramji |
| ||Indian Pediatrics. 2016; 53(9): 811 |
|[Pubmed] | [DOI]|
||Research Design and Statistical Methods in Indian Medical Journals: A Retrospective Survey
| ||Shabbeer Hassan,Rajashree Yellur,Pooventhan Subramani,Poornima Adiga,Manoj Gokhale,Manasa S. Iyer,Shreemathi S. Mayya,Daniele Marinazzo |
| ||PLOS ONE. 2015; 10(4): e0121268 |
|[Pubmed] | [DOI]|
||Inappropriate use of standard error of the mean when reporting variability of study samples: A critical evaluation of four selected journals of obstetrics and gynecology
| ||Wen-Ru Ko,Wei-Te Hung,Hui-Chin Chang,Long-Yau Lin |
| ||Taiwanese Journal of Obstetrics and Gynecology. 2014; 53(1): 26 |
|[Pubmed] | [DOI]|
||A Review of the Study Designs and Statistical Methods Used in the Determination of Predictors of All-Cause Mortality in HIV-Infected Cohorts: 2002–2011
| ||Kennedy N. Otwombe,Max Petzold,Neil Martinson,Tobias Chirwa,Barbara Ensoli |
| ||PLoS ONE. 2014; 9(2): e87356 |
|[Pubmed] | [DOI]|
||Policy of reviewing statistics in Indian medical and surgical journals
| ||Kannan, S. and Deshpande, S.P. and Gogtay, N.J. and Thatte, U.M. |
| ||Journal of Pharmacology and Pharmacotherapeutics. 2013; 4(2): 139-140 |
||Reporting quality of multivariable logistic regression in selected Indian medical journals
| ||Kumar, R. and Indrayan, A. and Chhabra, P. |
| ||Journal of Postgraduate Medicine. 2012; 58(2): 123-126 |
||Beware – statistics may deceive you
| ||Jaykaran C |
| ||Int J Med Sci Public Health. 2012; 1(1): 10-13 |
|[Pubmed] | [DOI]|
||Quality of Antidepressant Drugs Research Articles Published in Indian Medical Journals
| ||Charan J, Saxena D, Yadav P, Kantharia ND. |
| ||Indian J Psychol Med. 2011; 33(2): 141-144 |
|[Pubmed] | [DOI]|