This article may be reprinted free of charge provided 1) that there is clear attribution to the Orthomolecular Medicine News Service, and 2) that both the OMNS free subscription link http://orthomolecular.org/subscribe.html and also the OMNS archive link http://orthomolecular.org/resources/omns/index.shtml are included.
FOR IMMEDIATE RELEASE
Health Statistics and Study Design for the Rest of Us
by Michael Passwater
OMNS (Apr. 13, 2022) Given the flood of health information in the news, the increase in the number of health and medical journals, and journal publications, blogs, social media posts, websites, and opinions of family and friends, this brief overview is an attempt to help the reader evaluate headlines and discoveries related to human health. Reading newsfeeds today, one can feel surrounded by data in an information desert.
Determining whether two things among many are merely associated, coincidentally seen together like two strangers passing in a crowded city coffee house, or truly causal, with one reliably following the other in a consistent, predictable pattern, is difficult. The human body contains 60 thousand miles of blood vessels and over 37 trillion cells. It is estimated that each of these cells has approximately one billion chemical reactions per second. Essential nutrients are biochemicals and minerals required by the body for constructing its extensive structure and performing its sophisticated functions. Further, human behavior interacting with the environment is complex and affects the body in a myriad of ways. Therefore a relevant question is: how can a study of one or two variables in a complex biochemical network reliably determine a causal relationship with a specific outcome -- in other words, how can it differentiate a mirage from an oasis? A closer look at study design and analysis can improve evaluation of the meaningfulness of past assumptions and the latest health news.
A basic goal of scientific inquiry in the health sciences field is to isolate a variable and study the impact of changing this variable. In this context, a variable is a characteristic that differs from one person or group to another, or that may change over time within a given person, and can be measured or categorized. By studying changes to the one variable while keeping everything else constant, each specific outcome can be attributed to the change that caused it. In simple systems this works well. For instance, plants from the same batch of seeds can be divided into different groups, and each group of plants can be exposed to equal amounts of a different wavelength of light. Outcomes such as growth of the plants can be measured, and the connection between changes in the variable (light wavelength) and the outcome (growth) can be evaluated.
However, as the system of interest to study becomes more complex, isolating a single variable becomes more challenging. In a complex system, it is often difficult to discover the optimal variable change for discovering what causes a condition or disease. The human body is an extremely complex network of sophisticated physical, chemical, mental, and emotional systems. Finding large groups of humans that are truly identical is impossible. Many human characteristics interact with one another causing a single variable change to have many unintended consequences (which may be unmeasured within a given study). A change to a single variable may fail to trigger important synergistic benefits that would occur if the full set of relevant variables were optimized rather than just one element of the set.
Nutrient synergy is important in human wellness because nutrients work together to support a healthy body. Leaving one or more nutrients in a deficient state while testing the effect of a single other nutrient is a poor approach. For example, vitamin D, selenocysteine, and magnesium have strong co-dependencies in biochemical pathways, with each being a rate limiting factor for the other. Studying the effect of varying one without ensuring adequate levels of the others may produce misleading results. Vitamin K2 is also an important partner to vitamin D. However, measuring and matching the full set of essential nutrients for all participants in a study is resource intensive and difficult.
For those conducting and reviewing nutrient research, below are "rules" published by Robert P. Heaney in his landmark article "Guidelines for optimizing design and analysis of clinical studies of nutrient effects" 
Other excellent articles specific to nutrition research design include:
In addition to isolating variables of interest, and controlling for other variables, there are many other aspects of study design. Blinding refers to whether or not the study participants and the observers are aware of which treatment has been given to which person or group. A single blinded study typically means the subjects are unaware of which treatment is given, but the observers are aware. A double blinded study indicates that neither the subjects nor the observers are aware of which treatment is given. Blinding is an attempt to eliminate bias. Observers excited about a new intervention are more likely to see positive effects in people receiving it -- and less likely to see benefits when an intervention they are not excited about is used. And a person's thoughts, behaviors, and perceptions are influenced when they know they are receiving a test intervention or a control placebo. Keeping the study subjects and the study observers "blinded" to who is getting what intervention helps to minimize perception bias.
Another important aspect of study design is randomization. A randomized trial means that people are assigned to the study's groups in a random, impartial manner. Inclusion and exclusion criteria are another important aspect of study design. Does the study only enroll patients on Tuesday when Dr. X is in the clinic? Are there so many exclusions restricting entry to the study that the results are unlikely to be generalizable to a real-world population? Are there not enough exclusions causing the overall study results to miss a subpopulation that benefitted from the treatment?
A large sample size is desirable to increase the ability of the study to detect a difference between the test and control group, and to minimize the risk of the study results being due to chance. A large sample size is also thought to minimize the impact of unmeasured factors (confounding variables), although the only way to truly control for a variable is to measure it in the test and control study participants. Sample size is also important. Several online aides for determining appropriate sample sizes are available, two examples are included in the references. [2,3]
Retrospective and prospective
Whether a study is retrospective (looking back upon) or prospective (planning ahead and observing outcomes as they happen) is another important aspect of studies. Generally, a planned prospective study offers the opportunity to match variables in test and control groups, and to standardize interventions more thoroughly than a retrospective study.
Traditional evidence-based medicine and public health ranks the quality of study designs as follows: 
Interventional vs. observational studies
An observational study is one that does not intervene with a treatment -- it merely observes outcomes and associates them with different conditions or treatments. An interventional trial gives an active treatment to one group and may also give a null treatment (placebo) to another. While there are merits to randomized double-blind, placebo-controlled trials, the notion that it is unsafe to put an intervention into practice without such a trial is unsound. Much wisdom can be gleaned from retrospective observational studies. For example, there are no prospective double-blind placebo-controlled trials to support the use of parachutes when jumping from airplanes , or for performing cardiopulmonary resuscitation (CPR). Call it recklessness, but I support the performance of these procedures when necessary.
A placebo is an inert intervention given to the "control group" of a study. Its purpose is to make sure the test intervention effects are real, and not just the perception of the patients or observers. However, in an attempt to mimic the test intervention as closely as possible, the placebo may not be truly inert as intended. For instance, even a classic "sugar pill" placebo is not inert when studying diabetes. Olive oil and IV multivitamins have been used as placebos in large studies published within the past year. [7,8] Use of anti-inflammatory olive oil as a control in a study evaluating inflammation may obscure benefits of the test intervention since both the test and control arm may have reduced inflammation compared to a group receiving a true placebo. A caustic placebo may make a test drug appear more effective. Similarly, a non-inert placebo may blunt the recognition of side effects in the test intervention if, for instance, it contains nut products or other common allergens that may inflate the rate of reactions in the control group. The administration vehicle for the test substance may impact outcomes as well. A vitamin D study in Brazil used peanut oil to administer the single dose vitamin D intervention, and, sure enough, some people had strong reactions (the projectile vomiting also likely prevented vitamin D from reaching the circulation of those unfortunate patients). 
Two fundamental questions for evaluating health research are:
Additional Items to consider when reviewing a study
Statistics and Study Jargon
"No statistic is perfect, but some are less imperfect than others. Good or bad, every statistic reflects its creators' choices. ... Being Critical requires more thought, but failing to adopt a Critical mind-set makes us powerless to evaluate what others tell us. When we fail to think critically, the statistics we hear might as well be magical." ~ Joel Best 
Hypothesis - an educated guess at a relationship between a treatment and an outcome. For example, a researcher might speculate based on the results of a previous study that people taking a gram of vitamin C with each meal and a good multivitamin once a day will have fewer unplanned absences from work than those who don't. Or that women with a vitamin D level >40 ng/mL are less likely to have a preterm child than those with a vitamin D level <30 ng/mL.
Null Hypothesis - the assumption that there is no relationship between the test intervention and desired outcome. The null hypothesis basically states that the hypothesis is wrong. Technically, statistics evaluate whether or not the null hypothesis is correct rather than the hypothesis. If the null hypothesis is correct, then there is no relationship between the test variable and the outcome, and the hypothesis is incorrect. If the null hypothesis is proven to be incorrect, then the study results support the hypothesis. Technically, the hypothesis can be proven wrong, but not proven right. If not proven wrong, the hypothesis remains viable and subject to further evaluation. There is no definitive number of studies that will guarantee acceptance of an hypothesis.
P value - an expression of the probability that the results of an experiment testing a hypothesis are due to chance. Generally speaking, the lower the p value, the higher the reliability of the data. A p value below 0.05 is generally required to declare results "statistically significant" (the study outcomes are unlikely to be due to chance). A p value below 0.01 is more convincing.
Odds Ratio [21,22] - measures the relative effect of the study intervention. The odds ratio is the outcome of the test group divided by the outcome of the control group. If the outcome is a rate such as the risk of having a stroke, then it may be called a Risk Ratio or Hazard Ratio.
If the odds ratio = 1 this means the outcomes in the test and control group are the same
If the odds ratio is > 1 this means the outcome occurred more often in the test group than in the control group
If the odds ratio is < 1 this means the outcome occurred more often in the control group than in the test group
Confidence Interval - reflects the certainty of the odds ratio. Because samples of a population are studied rather than the entire population, the study results are an estimate of what the results may be for the full population of interest. A 95% confidence interval (95%CI) shows the range of values within which we can be 95% certain that the odds ratio is contained for the population. If the 95%CI crosses one (e.g. 95%CI = 0.95 - 1.05), the results are not statistically significant because one cannot be certain that the test intervention produced outcomes that differed from the control group.
Incidence - the number of new cases of a disease, event, or health-state; typically reported as the number of new cases per period of time, which may be called an incidence rate.
Prevalence - the total proportion of a population with a particular condition. Prevalence differs from incidence in that it is not restricted to new cases. For example, the annual incidence of Rheumatoid Arthritis in the USA is estimated to be 132,000 cases, while the prevalence of Rheumatoid Arthritis in the USA is estimated to be 3 million cases. [23,24,25]
Age adjustment - the rate that would have resulted if the population of interest had the same age distribution as a reference standard. Age adjustment is a critical step in population studies (epidemiology). The number of people over 84 years old in Florida is 331,287 (2.1% of the Florida population). The number of people over 84 years old in Utah is 28,951 (1.1% of the Utah population). If more people in Florida are dying from or being diagnosed with a certain condition than in Utah, what does it mean? The populations of each state must be adjusted to a common standard population such as the 2020 USA census to allow an "apples-to-apples" comparison. Without adjusting for the age of the different populations being compared, the data has little meaning and may be harmfully misleading.
Confounding variables - variables other than the intervention being studied may influence theso outcome(s) measured and confuse the interpretation of the study results. Do ice cream sales cause crime? Lots of data can be assembled to make the case that it does. However, other variables associated with warmer weather are more likely involved than the sale of ice cream. While it is true that ice cream sales increase in warmer weather, and it true that crime increases in warmer weather, the connection between them is coincidental. Such "true-true-unrelated" associations are abundant in our complex world. As another example, studying disease outcomes and vaccination rates without realizing that a much larger percentage of vaccinated people in the study had vitamin D levels >40 ng/mL, selenoprotein P levels between 3 - 4.5 mg/L, and took one or more grams of vitamin C per day may lead to a false conclusion as to what caused the observed outcomes. Measuring as many variables as possible in a study is important, but resource limitations force investigators to choose the measurements they believe will be the most important.
Controlling for confounding variables can also be misapplied. Interestingly, a major journal published a study last year that used conditions known to be associated with vitamin D deficiencies as variables that then canceled out the test variable of vitamin D as impacting the outcome - essentially saying that low vitamin D is not associated with the disease because conditions with low vitamin D also had the same disease association. Be wary of studies that mix biochemical health markers with non-biochemical health markers. Actual measurements of nutrients within appropriate timeframes in study subjects is critical for evaluating the effects of nutrients. If a vitamin has a half life of 20 minutes or even 12 weeks in the human body, using a measure of the vitamin in a study subject from 10 years ago to evaluate a current disease is curious, yet publishable.
"Confounding by indication" is a serious challenge in healthcare studies, especially for retrospective observational studies. PeopIe receiving blood transfusions are more likely to be bleeding than people not receiving blood transfusions. However, it is not wise to suggest that blood transfusions cause people to bleed to death. In this example, the indication (bleeding) for the intervention (blood transfusion) confuses or confounds the association between the intervention (transfusion) and the measured outcome (death). Careful attention to control populations and the "baseline state" of study subjects is important when conducting studies.
Testing new ideas
Critical thinking, developing and testing ideas, and keeping an open mind are challenging yet essential to gain a deeper and more accurate understanding of ourselves and our relationships with our surroundings.  I once thought all creatures eating carotenoid-rich algae and brine shrimp were pink Flamingos. Then I observed a pink bird with a white head and neck, and a beak that resembled a wooden spoon eating shrimp. Instead of rejecting the observation, I modified my original hypothesis. The association between eating shrimp and being a pink bird was now stronger, but I recognized two possible outcomes: either being a pink Flamingos or a Roseate Spoonbill. I ate shrimp, and to my disappointment, I did not turn into either of these beautiful pink birds. It turns out more variables were involved in achieving the desired outcome. By prospectively planning a study with an expanded selection criteria allowing a representative samples of all shrimp eating creatures, and evaluating many more characteristics of each creature in the study, it became apparent that only white feathered birds with very large carotenoid intake and the proper liver enzymes were able to display pink feathers.
"If we all worked on the assumption that what is accepted as true is really true, there would be little hope of advance." - Orville Wright (1871 - 1948) 
Humans and their interactions with the environment are highly complex. Nutrition studies are difficult because they require measuring the baseline level of several synergistic nutrients which cannot be easily done with retrospective studies. However, observational studies often contribute important evidence about the outcome of dietary deficiences, that can be further tested with prospective interventional studies. Persistent inquiry, carefully designed studies, detailed observations - including timely measurement of nutrients, along with rigorous analysis and critical review helps us better understand how to more reliably prevent, manage, and cure diseases and lead our best lives.
References and Additional Resources
1 Heaney, RP (2014) Guidelines for optimizing design and analysis of clinical studies of nutrient effects Nutrition Reviews 72:48-54. https://pubmed.ncbi.nlm.nih.gov/24330136
2 ClinCalc Sample Size Calculator https://clincalc.com/stats/samplesize.aspx
3 Sample Size Calculators for designing clinical research. UCSF Clinical and Translational Science Institute https://sample-size.net
4 Designing Clinical Research, 4th edition, online companion https://www.dcr-4.net
5 Dahabreh IJ, Sheldrick RC, Paulus JK, et al (2012) Do observational studies using propensity score methods agree with randomized trials? A systematic comparison of studies on acute coronary syndromes. Eur Heart J. 33:1893-1901. https://pubmed.ncbi.nlm.nih.gov/22711757
6 Smith GCS, Pell JP (2003) Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ, 327:1459-1461. https://pubmed.ncbi.nlm.nih.gov/14684649
7 Korley FK, Durkalski-Mouldin V, Yeatts SD, et al. (2021) Early Convalescent Plasma for High-Risk Outpatients with Covid-19. NEJM, 385:1951-1960. https://pubmed.ncbi.nlm.nih.gov/34407339
8 Costenbader KH, Hahn J, Cook NR. (2022) Vitamin D and marine omega 3 fatty acid supplementation and incident autoimmune disease: VITAL randomized controlled trial. BMJ 2022;376:e066452. https://pubmed.ncbi.nlm.nih.gov/35082139
9 Murai IH, Fernandes AL, Sales LP, et al. (2020) Effect of Vitamin D3 Supplementation vs Placebo on Hospital Length of Stay in Patients with Severe COVID-19: A Multicenter, Double-blind, Randomized Controlled Trial. https://pubmed.ncbi.nlm.nih.gov/33595634
10 Passwater M (2021) The Victas Trial: Designed to Fail. Orthomolecular Medicine News Service. http://www.orthomolecular.org/resources/omns/v17n08.shtml
11 Klenner FR. (1971) Observations On the Dose and Administration of Ascorbic Acid When Employed Beyond the Range of A Vitamin In Human Pathology. J Applied Nutrit. 23:61-87. http://orthomolecular.org/library/jom/1998/pdf/1998-v13n04-p198.pdf
12 Case HS. (2022) Vitamin C and Infants: Determing dose. Orthomolecular Medicine News Service. http://www.orthomolecular.org/resources/omns/v18n05.shtml
13 Arastu AH, Elstrott BK, Martens KL, et al (2022) Analysis of Adverse Events and Intravenous Iron Infusion Formulations in Adults With and Without Prior Infusion Reactions JAMA Network Open. 5:e224488. https://pubmed.ncbi.nlm.nih.gov/35353168
14 Dean C (2017) The Magnesium Miracle, 2nd Ed. Ballantine Books. ISBN-13 : 978-0399594441
15 Lindberg JS, Zobitz MM, Poindexter JR, Pak CY (1990) Magnesium bioavailability from magnesium citrate and magnesium oxide. J Am Coll Nutr. 1990 9:48-55. https://pubmed.ncbi.nlm.nih.gov/2407766
16 Durrant LR, Bucca G, Hesketh A, et al. (2022) Vitamins D2 and D3 Have Overlapping but Different Effects on the Human Immune System Revealed Through Analysis of the Blood Transcriptome. Front. Immunol. 13:790444. https://pubmed.ncbi.nlm.nih.gov/35281034
17 Rayman MP (2008) Food-chain selenium and human health: emphasis on intake. British Journal of Nutrition, 100:254-268. https://pubmed.ncbi.nlm.nih.gov/18346308
18 Penberthy WT, Saul AW, Smith RG (2021) Niacin and Cancer: How vitamin B-3 protects and even helps repair your DNA. Orthomolecular Medicine News Service. http://www.orthomolecular.org/resources/omns/v17n05.shtml
19 Aggarwal BB, Sundaram C, Prasad S, Kannappan R (2010) Tocotrienols, the Vitamin E of the 21st Century: Its potential against cancer and other chronic diseases. Biochem Pharmacol. 80: 1613-1631. https://pubmed.ncbi.nlm.nih.gov/20696139
20 Best J (2012) Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists. Berkeley: University of California Press, Updated version, ISBN-13: 9780520274709
21 Hicks T. (2013) A beginner's guide to interpreting odds ratios, confidence intervals, and p-values. August 13, 2013. https://s4be.cochrane.org/blog/2013/08/13/a-beginners-guide-to-interpreting-odds-ratios-confidence-intervals-and-p-values-the-nuts-and-bolts-20-minute-tutorial
22 GraphPad QuickCalcs https://www.graphpad.com/quickcalcs
23 Myasoedova E, Crowson CS, Kremers HM, et al. (2010) Is the incidence of rheumatoid arthritis rising?: results from Olmsted County, Minnesota, 1955-2007. Arthritis Rheum, 62:1576-1582. https://pubmed.ncbi.nlm.nih.gov/20191579
24 Hunter TM, Boytsov NN, Zhang X, et al. (2017) Prevalence of rheumatoid arthritis in the United States adult population in healthcare claims databases, 2004-2014. Rheumatol Int,37:1551-1557. https://pubmed.ncbi.nlm.nih.gov/28455559
25 Eriksson JK, Neovius M, Ernestam S, et al. (2013) Incidence of rheumatoid arthritis in Sweden: a nationwide population-based assessment of incidence, its determinants, and treatment penetration. Arthritis Care Res (Hoboken), 65:870-878. https://pubmed.ncbi.nlm.nih.gov/23281173
26 Orville Wright Quotes. Quotes.net.STANDS4 LLC, 2022. Web. 1 Apr. 2022. https://www.quotes.net/quote/19271
27 Best J. (2021) Is That True? Critical Thinking for Sociologists. University of California Press. ISBN-13: 9780520381407
Nutritional Medicine is Orthomolecular Medicine
Orthomolecular medicine uses safe, effective nutritional therapy to fight illness. For more information: http://www.orthomolecular.org
Find a Doctor
To locate an orthomolecular physician near you: http://orthomolecular.org/resources/omns/v06n09.shtml
The peer-reviewed Orthomolecular Medicine News Service is a non-profit and non-commercial informational resource.
Editorial Review Board:
Albert G. B. Amoa, MB.Ch.B, Ph.D. (Ghana)
Comments and media contact: email@example.com OMNS welcomes but is unable to respond to individual reader emails. Reader comments become the property of OMNS and may or may not be used for publication.
To Subscribe at no charge: http://www.orthomolecular.org/subscribe.html
To Unsubscribe from this list: http://www.orthomolecular.org/unsubscribe.html
This website is managed by Riordan Clinic
Information on Orthomolecular.org is provided for educational purposes only. It is not intended as medical advice.
A Non-profit 501(c)(3) Medical, Research and Educational Organization
3100 North Hillside Avenue, Wichita, KS 67219 USA
Phone: 316-682-3100; Fax: 316-682-5054
© (Riordan Clinic) 2004 - 2017
Consult your orthomolecular health care professional for individual guidance on specific health problems.
Information on Orthomolecular.org is provided for educational purposes only. It is not intended as medical advice.