1 Bodybuilderinfo: the china study
Showing posts with label the china study. Show all posts
Showing posts with label the china study. Show all posts

Wednesday, 5 May 2010

The Study Everyone Talks About: Part 2: The Ravaging Reviews

The China Study

Yes, you have heard of it before.

Here is our first post about it: The Study Everyone Talks About Part 1: Correlation is NOT Causation

Quick review: The China Study spearheaded by Dr. T. Colin Campbell incorporates meta-analysis from studies on 65 counties in China and 8000 statistically significant correlations to conclude that animal-based diets lead to more chronic disease than plant-based diets.  Thus, the authors conclude that people should be vegans.

My Intent


My intent is NOT to review The China Study.  That has been done sufficiently by many before me.  My intent is NOT to provide a balanced sampling of reviews.  That would be exhausting to both of us.  My intent is to present an online literature review of the arguments against The China Study.  I want to be able to confidently talk about why I DON'T believe the book is a convincing argument to give up meat and become a vegan.  Thus, I am looking into the arguments critically analyzing the book for my own personal benefit and highlighting them for your perusal.  My intent is to be a resource for your conversations about The China Study and provide a stepping stone for further research into these sources.

Last time in Part 1 we talked about how correlation does NOT mean causation, a topic that The China Study addresses and warns against, but ultimately commits itself when leaping from correlation to generalization (an animal protein-based diet leads to disease) and dietary prescription (recommendation of a vegan diet).  Don't get me wrong: While it is very useful to look at commonalities amongst those inflicted with a medical malady, not everything that correlates is meaningful or stands up to further testing.  And further testing is the NECESSARY next step to validate hypotheses gathered from the correlations.  It is that leap from correlation to dietary prescription without testing and validation that provides the fatal hiccup for The China Study.    


So what are the arguments against The China Study?


Wow, where to begin?  I am going to make this a two-parter since there is a lot to digest and I think it is worth digesting every morsel so that you have a foundation to draw upon next time someone pops the question to you, "So what about that China Study I've heard so much about?" Today, we'll discuss main flaws (lumped by topic) and leave the deficiencies of the vegan dietary prescription for another day.  Okay, let's dive in!

The Study Itself

One of the prominent reviewers of the The China Study is Chris Masterjohn, a writer and PhD student whose review was published in the quarterly journal of The Weston A. Price Foundation and sparked an animated discourse between The China Study author T. Colin Campbell and the reviewer.  Masterjohn exposes some limitations of the original China Study, the study that provides the basis for the book (which is named after the study).  The data collection occurred over 3 days in the autumn of 1983 and canvassed 65 rural counties, considered 367 variables, and included 6,500 individuals.  Why these people?  Their genetic background was very similar, yet their rate of disease showed significant variation.  Something was up.  Despite the large data collection, Masterjohn exposes some key limitations of this study including:
  • Timing: The research was conducted over three days in the autumn of 1983, so data on the health and diet of the population as well as foods consumed at different times of year, were not taken into account.
  • Blood samples: The blood samples were pooled by village instead of individually tested.  
  • Disconnected data: Mortality rates were taken years before, from 1973-75.  Upon further investigation, Masterjohn found that these rates conflicted with household questionnaire data in regard to animal-protein consumption and cancer rate.  Higher animal protein consumption did NOT lead to higher cancer mortality, in contrast to the questionnaire results.  Yearly meat intake did NOT yield statistically significant correlations by government statistics or questionnaire.  
  • Standardization: Nutrients from foods were determined from standard tables rather than by testing the food itself.  This limits variability in environment, such as soil quality affecting minerals in the food.  
  • Data Collection: Questionnaires were used that limited the breadth of animal foods in the diet.  For example, "fish and sea food" lumped fish and shellfish--two foods with very different nutrient profiles, and "meat" included poultry, beef, pork, etc.  
Also read Campbell's rebuttal to Masterjohn's review and Masterjohn's subsequent response.  

A review aptly named The China Study by Harriet Hall, MD, a retired family physician and one of the founders of Science-Based Medicine blog, further highlights limitations of the study to include:
  • Age Range: Only adults 35-64 were considered because death certificates for those older than 64 were considered "unreliable."  
Another researcher, a PhD assistant professor at The University of Economics in Romania, named Claudiu Herteliu published a book review of The China Study.  His critique of the methods include:
  • The samples of rural Chinese villages and villagers: were they random?  If not, conclusions about the greater rural Chinese population are at risk.  
  • The pooling of samples, which can lead to problems since assumptions about similarity were NOT proven.  If data is pooled, then conclusions about individual diet and behavior can NOT be substantiated.  Even amongst a rural village, genetic and behavioral inheritance is NOT identical. 
Finally, there are the correlations themselves that lead Campbell to make conclusions and dietary prescriptions.  An article entitled Comparative Anatomy and Physiology Brought Up to Date: Are Humans Natural Frugivores/Vegetarians or Ominvores/Faunivores? by Tom Billings posted at Beyond Vegetarianism also critiques The China Study.  The limitations it proposes include:
  • only 65 observations taken, excluding hundreds of variables, which is not exhaustive and leads to statistical limitations
  • geographical limitations to correlations and to applying these results to other populations and cultures around the world
  • lack of correlations supporting their conclusions:
The China Study report lists only 6 statistically significant correlations between meat-eating and disease mortality. Further, 4 of the correlations are negative, which indicates that the mortality rate for that disease decreased as meat consumption increased. The two diseases that had positive correlations with meat consumption are schistosomiasis, a parasite, and pneumoconiosis and dust disease.
  • that hypotheses generated require clinical testing to validate them--they cannot lead to conclusions or dietary recommendations, especially jumping from group observations to individual dietary prescriptions 
  • the lack of income data, which makes adjustment for the effect of income less reliable--and income would have a close association to degree of Westernization 
  • that NO villages surveyed were actually vegetarian or vegan since the average percentage of energy intake from animal foods ranged from 1-59%
Denise Minger also systematically addresses Campbell's claims with an exhaustive review of his work.  Her critique: The China Study: Fact or Fallacy? lit a fire in the online community and it is well worth the read.  Here is a snippet:
On page 106 of his book, Campbell makes a statement I wholeheartedly agree with:
"Everything in food works together to create health or disease. The more we think that a single chemical characterizes a whole food, the more we stray into idiocy."
It seems ironic that Campbell censures reductionism in nutritional science, yet uses that very reductionism to condemn an entire class of foods (animal products) based on the behavior of one substance in isolation (casein).
In sum, “The China Study” is a compelling collection of carefully chosen data. Unfortunately for both health seekers and the scientific community, Campbell appears to exclude relevant information when it indicts plant foods as causative of disease, or when it shows potential benefits for animal products. This presents readers with a strongly misleading interpretation of the original China Study data, as well as a slanted perspective of nutritional research from other arenas (including some that Campbell himself conducted).
Also read: Cambell's rebuttal to her critique and Minger's response.

Author T. Colin Campbell makes it clear that the China Study itself isn't the only source of his conclusions.  He also uses his own laboratory research and that of others and various clinical studies on a vegetarian-style diet.  Masterjohn takes those other studies to task in his response to Campbell's rebuttal of his original review.  He believes the picking and choosing of which research to include and which to exclude is meaningful and limits the conclusions that can be drawn.  We'll discuss this more below.


Okay, want a snapshot of that "other research" from which Campbell based his conclusions?  Take Dr. Michael Eades's recent commentary: The China Study vs the China study.  It's a proverbial nail in coffin.  The main idea here is "obfuscation."
In fact, in my studied opinion, The China Study is a masterpiece of obfuscation.
It is obfuscatory in so many ways it could truly qualify as a work of obfuscatory genius. It would be difficult for a mere mortal to pen so much confusion, ambiguity, distortion and misunderstanding in what is basically a book-length argument for a personal opinion masquerading as hard science.
In his dissection of key research on animal protein versus plant protein, Dr. Eades found some serious flaws.  For one, the key research uses rats not humans and that fact is not made plainly clear to the reader.  For another, those rats are bred to be very susceptible to cancer, so slight changes in diet can definitely precipitate the desired effect.  For another and another, rats are not humans and they have evolved to withstand plant protein better than animal protein since it is their natural diet.  And the worst part?  The flaws, obfuscation, and misrepresentation are not isolated in The China Study, in fact they are the trend.

Lacking Evidence for the Prescribed Diet 

Setting aside the fact that The China Study authors jumped from correlation to dietary prescription, what is unsettling is that the prescribed diet was NOT followed by any of the groups studied.  While rural Chinese that had a low animal protein diet were concluded to be healthier (keep reading for more on this below), NONE had zero animal protein or followed the vegetarian or vegan approach prescribed by the authors.  Chris Masterjohn in his response to Campbell's rebuttal to his original review addresses the problem of quality in a diet devoid of animal products.  He makes the claim that even just 2% of the diet consisting of animal protein can have health benefits from the nutrient-, vitamin-, and mineral-dense animal foods.  Populations utilizing a small percentage of animal protein in their diets seek out the most beneficial sources, such as shellfish--which in the China Study were obscured by their categorization with fish.

If you look beyond the China Study itself, the clinical trials the authors cite as evidence of the efficacy of the vegan diet do not wholeheartedly endorse this dietary prescription.  Masterjohn shows that only two of the clinical trials actually tested a vegan diet; however, he only describes one in which no control groups were used and results are confounded by weight loss and scope of research (only looking at arthritis, which did improve).  The vegetarian diet clinical trial was confounded by dropout rate, lack of participation at the end, and simultaneous use of cholesterol-lowering drugs.

Another study cited in The China Study as evidence against a high protein diet was Dr. Lester Morrison's heart disease trial.  However, as Anthony Colpo author of The Great Cholesterol Conwho reviewed the study too found: according to Morrison, himself, the study was "high-protein, low-fat" and included protein greater than the RDA that included meat and dairy.  What was the result of the study?  A reduction in heart attack reoccurrence as compared with the control group.

Missing or Flawed Mechanisms

Dr. T. Colin Campbell writes that a mechanism is necessary to link correlations to causation.  For example, he says that even though telephone poles correlate with heart disease (in countries where there are more telephone poles, there is more incidence of heart disease) there is no mechanism to explain the correlation (they are two entirely different things), whereas the correlation between smoking and lung cancer is meaningful since there is a biological mechanism visible: smoke damages lung tissue, and damaged cells are more likely to become cancerous.  (see page 41 of The China Study)

Despite the need for a mechanism, Campbell links animal-based diet to diabetes without explanation, according to Dianne Cowan at The Migraineur in her review of The China Study.  She expresses frustration at this slip:
So, there might be a correlation between animal product consumption and diabetes, but without that mechanism, it’s gonna be hard to show causation.
Meanwhile, there is also a correlation between carbohydrate consumption and diabetes, and there is a mechanism that explains it:  carbohydrate consumption leads to an insulin response; excessive carbs mean excessive insulin; after a while the insulin receptors in the cell wear out, leading to higher blood sugar; pancreatic beta cells try to compensate by producing more insulin, while at the same time, chronic high blood sugar causes more beta cells to die.  Where does excessive consumption of protein and fat fit into all this?
Chris Masterjohn (discussed above) has difficulty finding the biological connection between diet and cancer in his response to Campbell's rebuttal of his original review.  In The China Study, the authors state,
We measured six blood biomarkers that are associated with animal protein intake. Do they confirm the finding that animal protein intake is associated with cancer in families? Absolutely. Every single animal protein-related blood biomarker is significantly associated with the amount of cancer in a family.
However, Masterjohn looked into this claim and found some problems.  Of the six biomarkers associated with animal protein intake only plasma copper is actually associated with cancer and with animal protein intake, but not at the exclusion of plant consumption, since vegetarians have MORE copper in their diets than meat-eaters.  Thus, a critical connection of animal protein to cancer is lost.

According to wikipedia, The China Study authors do cite mechanisms to support their conclusions against animal protein.  For example, they base their argument of animal protein leading to Western diseases upon the association of dietary animal protein with high cholesterol and high cholesterol and disease, or the Lipid Hypothesis.  Unfortunately, the Lipid Hypotheses has been discredited because there is not enough evidence (see Weston Price Foundation: The Skinny on Fats) to link a diet rich in saturated fat and/or animal protein with heart disease or high cholesterol with heart disease, since it is the particle size of LDL and the triglycerides that have the strongest association with disease (see this research review).  A ketogenic diet with low carb and high fat and high animal protein actually lowers small, dense LDL that do lead to heart disease (see these studies: Journal of Nutrition, International Journal of Cardiology, etc.).

Other mechanisms The China Study draws upon to support their conclusions include:
  • the antioxidants in plants as counteracting harmful free radicals to protect from disease.  While true, the fact that many plants also introduce harmful elements to the body like lectins, gluten, and antinutrients do NOT make plants a health boon.  See The Paleo Diet's discussion of grains and legumes for more information on how plants can harm rather than nourish.  
  • animal protein pushes growth, which can promote the growth of disease.  This is good and bad.  Growth is obviously beneficial to the growth and maintenance of living beings, but can get out of control with growth and spread of disease.  Valid point, but plants are also at the root of inflammation (see Cooling Inflammation and Mark's Daily Apple for a current series on gut health) and their carbohydrates pave a path to heart disease, obesity, and diabetes (see The Heart Scan Blog).  Heck, sugar feeds cancer!  So excuse me if I am NOT convinced of the animal protein-cancer link (which I also discussed in my Starter Series: Eat meat.)  To me, it looks like plants are doing much more damage than animal-based foods.   
  • the acid-base balance is given as a mechanism to reduce animal protein since it is acidic and tips our scale toward the acid side, which results in leaching calcium from our bones (osteoporosis), which then sets up more problems due to the roaming calcium in our blood.  My counter: grains are acidic too and don't come with the benefit of omega-3 fatty acids, vitamins, minerals, or nutrients in such density as in grass-fed, wild-caught,  or pastured meat.  For more, see Dr. Cordain's Cereal Grains: Humanity's Double Edged Sword and Acid-Base Balance from the Paleo Diet website.      

Correlation with Disease: Is it the animal-based protein or other accoutrements of a Western diet? 


While there is NO argument with a Western diet leading to greater incidence of disease, the cause of that increased risk is debatable.

Journalist Gary Taubes, author of Good Calories, Bad Calories, has tackled The China Study, courtesy of a question-answer session at Dr. Eades's blog.  His take?  It's the sugar.  High-carb or low-carb, high-fat or low-fat, the variable we should be looking at is sugar: refined carbohydrate.  He says that sugar was the lowest quantity food item ("sugar consumption wasn't even measured in the study since it was so low") in the The China Study; thus, lack of sugar is the key to their health and that of many rural/traditional peoples. 

Anthony Colpo author of The Great Cholesterol Con reviewed the study, too.  He found fault with Campbell's attack of all animal-based protein.  Campbell saw that children in the Philippines on a Western diet had higher rate of liver cancer.  He also saw that rats fed casein, a protein in dairy products, got cancer.  Putting two and two together, he concluded animal-based protein leads to cancer.  Unfortunately, Colpo explains, dairy products have two proteins: casein and whey.  While casein did cause cancer, whey had the opposite effect, improving the health of rats fed whey protein.  Colpo continues his review by detailing potential deficiencies for a vegan diet and goes back to the original China Study to review the data and find correlations.  Notably, he found that wheat and wheat flour were associated with a significantly higher risk of coronary heart disease, although these were never mentioned by Campbell.  Here are some other correlations based on the original research:
Animal protein, fish protein, meat intake, saturated fat, and fat calories were all negatively associated with all-cause mortality in infants, children, teenagers and adults, although none of the associations reached statistical significance (for those unfamiliar with research-speak, a negative correlation means that as intake of these foods increased, mortality risk decreased; failure to reach statistical significance means that researchers can't be sure these findings were not due to chance).
and
No statistically significant relationships, protective or otherwise, were found for milk intake, fiber, cereal grains, legumes, and vegetables among those aged 0-64.
The only other dietary factor that was significantly associated with overall mortality among those aged 0-64 was soy sauce (not soy products), which showed a 43% decrease in mortality risk (p=0.001).
Cancer?
With regards to specific types of cancer, no statistically significant associations were observed for total protein, animal protein, fish protein, meat intake, milk intake, saturated fat, total fat, fiber, cereal grains, legumes, vegetables and mortality from colorectal or breast cancers.
Heart disease?
No statistically significant associations were observed for total protein, animal protein, fish protein, meat intake, milk intake, saturated fat, total fat, fiber, legumes, and mortality from coronary heart disease.
Here is what another researcher, Chris Masterjohn whom we introduced above, looking into that study DID find:
It is interesting to see, however, the general picture that emerges. Sugar, soluble carbohydrates, and fiber all have correlations with cancer mortality about seven times the magnitude of that with animal protein, and total fat and fat as a percentage of calories were both negatively correlated with cancer mortality.
The only statistically significant association between intake of a macronutrient and cancer mortality was a large protective effect of total oil and fat intake as measured on the questionnaire. As an interesting aside, there was a highly significant negative correlation between cancer mortality and home-made cigarettes! 
Dr. Hall, cited above from Science-Based Medicine, found contradictions with Campbell's research as well.  In her review, she cites a few PubMed articles attributing the health of vegetarians to their consumption of fruit and vegetables, not the exclusion of animal protein.  Dr. Lawrence Wilson in his review comes to the same conclusions.  He also highlights factual errors in the book, most already discussed.  For example, the authors assert that vitamin D deficiency arises from too much protein and dairy in the diet (Wilson says he was unable to find any study supporting this statement) and that all vitamin D needs can be met by the sun (the deficiency of those even living in sunlight-drenched areas like Florida runs contrary).  Similarly, Tom Billings, author of an article cited above that detailed The China Study limitations also presents evidence of no significant difference in most cancer rates amongst vegetarians and non-vegetarians, referencing a study by Key et al. 1998.  One thing is for certain: there are no clear cut associations here.

Robb Wolf, nutrition guru and strong proponent of the Paleo Diet, tackled The China Study with this post: Heard of T. Colin Campbell?  He discusses a debate he spearheaded between Campbell and Loren Cordain, The Paleo Diet author.  We'll discuss that in Further Resources below.  An important point that Robb Wolf brings up is the existence of paradoxes.  The premise of vegetarian or vegan diet as the healthiest diet has to hold up when compared to different societies and it deflates when given the Inuit Paradox.  The Inuit Paradox sees a disconnect with current nutritional advice to go low-fat and high-carbohydrate in order to lose weight and be healthy.  In fact, the Inuit did best on a high animal fat diet and did poorly when forced to subsist on lean protein as their energy source in an extreme environment that makes carbohydrate very scarce.  The Inuit did not have the trappings of modern dietary diseases like cancer, diabetes, or heart disease.  As with most (if not all?) aboriginal peoples, switching over to a Western Diet has brought disease and death, NOT prosperity, at least in health.  If the animal protein and fat if anything fell, how can they cause these diseases?  And the Inuit are not alone; there are many more paradoxes out there to discredit a vegetarian diet (ex. French, Masai, Spanish, etc.).

Bottom line: high fat, including animal fat, is actually beneficial!  Surprise, surprise!  We addressed this in the past with the posts Saturated with Fat, Fatphobia, and Starter Series Part 3: Eat Fat.  The healthy fats are monounsaturated, saturated, and omega-3 fatty acids (you also need omega-6, but we want to minimize it and maximize omega-3, read the cited posts for more information).  Now, you can get monounsaturated and even saturated fat from plants like avocados, olive oil, and coconut oil.  The plant world can cover those, but what about the essential omega-3 fatty acids?  We need  to consume animals, especially fish and their oils.  Sorry, but nuts and seeds just don't cut it since they come with the price of high omega-6 fatty acids or in flaxseed's case: disquieting phytoestrogens, rancidity, and possible links to prostate cancer (see Mark's Daily Apple).  You just aren't going to find the omega-3's outside of animal products in the forms our body needs most: EPA and DHA.  Even flaxseed requires conversion from ALA to DHA and EPA, which is inefficient.  So to get this metabolically healthy and heart-healthy high-fat diet, you need animal foods.  And if aboriginal people are getting sick when they Westernize their diet away from their high-fat and high-protein diets--that points the finger to something else nefarious, NOT the animal foods.

Saving the Best for Next Time
There are virtually no nutrients in animal-based foods that are not better provided by plants.
Yes, that was stated in The China Study, and yes, that is completely wrong.  Virtually does nothing to soften the blow of that unsubstantiated stupidity.  The fact is that vegetarians and vegans KNOW that their diet is deficient and seek out supplements to try to fix that.  It's not a secret.  I can understand that there are many reasons for one to follow a vegetarian/vegan diet, but "healthier than eating meat" is NOT a well-supported argument.  For now, let's just counter that statement with B-complex vitamins, zinc, and omega-3 essential fatty acids and we'll discuss this whopper another day.  It deserves our full attention and can easily stand alone, just like the cheese, Hi-Ho, the derry-o, the cheese stands alone.

To Summarize

1.  There are serious limitations to the original China Study on which the book is based.

2.  The prescribed vegan diet was NOT observed in the China Study itself, tested through the author's research, or convincingly elucidated through referenced clinical trials.

3.  There are missing or flawed mechanisms providing the integral connections to link animal protein to disease.  As such, it is NOT clear if animal protein actually does lead to disease or if vegetarian or vegan diets are, in fact, healthier.

4.  Westernized diets do lead to increased prevalence of disease, but blaming that on the consumption of animal foods is NOT supported by the research.

5.  A vegetarian diet, and especially vegan diet, is NOT the best source for nutrition.

Whew!  Done!  At least for now.  I leave you with some further resources to further feed your mind.

Further Resources

Here is a list of other resources that branch off from those already cited.  Each is worth a read for their contributions to the argument.

The Protein Debate
This is a spectacular debate between Loren Cordain, author of The Paleo Diet, and T. Colin Campbell, author of The China Study (just in case you totally missed that connection a hundred times earlier :)).  Although not specifically about The China Study, the two duke it out over its basic premise about dietary protein.  Cordain starts with his article: "The Evolutionary Basis for the Therapeutic Effects of High Protein Diets" in which he concludes:
The evolutionary evidence indicates that so called “high protein diets” (20 – 30 % total energy) and “very high protein diets” (30- 40 % total energy) actually represent the norm which conditioned the present day human genome over more than 2 million years of evolutionary experience. The evolutionary template would predict that human health and well being will suffer when dietary intakes fall outside this range. Hence the current U.S. consumption of protein (15 % total energy) may not optimally promote health and well being. There is now a large body of experimental evidence increasingly demonstrating that a higher intake of lean animal protein reduces the risk for cardiovascular disease, hypertension, dyslipidemia, obesity, insulin resistance, and osteoporosis while not impairing kidney function.
T. Colin Campbell turns the question of "how much protein" around to be how much excess is safe in his article: "How Much Protein Is Needed." Although he states that processed foods may confound or exacerbate the problems associated with animal protein, his conclusion:
My conclusion from these many observations is that animal protein, when added to diets already containing 10% protein, has the potential to promote the development of a wide variety of serious and oftentimes fatal diseases—not only because of the direct effects of protein but also because of the many parallel effects of companion nutrient imbalances created by animal-based foods simply to get access to the extra protein. I find that 10% total dietary protein, which has been long recommended by multiple policy committees and acknowledged in scientific reviews, is not only adequate but also is relatively devoid of risk of serious diseases and other ailments. It just so happens that 10% dietary protein is the same level that is typically found in a diet of varied whole plant-based foods. Increasing dietary protein above this 10% level generally means adding or substituting animal-based foods for plant-based foods but doing so invites a plethora of adverse health effects. To be very clear, I believe that total dietary protein should be 10% of calories, with virtually all of it being in the form of plant-based protein.
In his rebuttal to Campbell's article, Cordain makes a key point:
As I have laboriously and meticulously lain out in my initial essay, there is no credible fossil, archeological, anthropological, anatomical, ethnographic or biochemical evidence to show that members of our genus (Homo) routinely consumed low protein diets. In fact, without the inclusion of energetically dense animal food into the hominin diet, starting at least 2.5 million years ago, our large energetically active brains would not have evolved (16, 34, 35). Accordingly, the fundamental logic underlying Colin’s hypothesis (that low protein diets improve human health) is untenable and inconsistent with the evolution of our own species.
For more gory details on debate, check out this review by a reader of Dr. Eades's website.  A key point:  In their respective articles, Cordain cites 134 references, while Campbell none.  Yes, you heard me: none.  If the wealth of research supports his viewpoint, why not make that abundantly clear by referencing at least some of it?

Other fun resources are:
  • Dr. Eades author of Protein Power critically digests another China Study where researchers decided to blame vegetable oil, not increased fruit and veggies, in making the Chinese obese.  The kicker: there is no statistical difference in the amount of vegetable oil used between those who ate the most carbs and those who ate the least.  What did obesity relate to?  The amount of carbs they were eating.  
Dr. Eades suggests that their higher carbohydrate intake (+10%) and higher calorie intake (+120 kcal/day) are responsible for the weight gain, but I wasn't satisfied with that explanation so I took a closer look.
One of the most striking elements of the 'vegetable-rich' food pattern is its replacement of rice with wheat flour. The 25% of the study population that adhered the least to the vegetable-rich food pattern ate 7.3 times more rice than wheat, whereas the 25% sticking most closely to the vegetable-rich pattern ate 1.2 times more wheat than rice! In other words, wheat flour had replaced rice as their single largest source of calories. This association was much stronger than the increase in vegetable consumption itself! 
  • A cool website called Opposing Views gives you plenty of brain food by providing both sides of the debate for/against vegetarianism with supporting articles.
Did I forget any important reviews opposing The China Study?  Please let me know!  Hopefully this post provides a helpful resource for you next time you encounter The Question!

Friday, 16 April 2010

The Study Everyone Talks About Part 1: Correlation is NOT Causation

The China Study

Whenever I get to talking paleo with people, it comes up.  Inevitably.

"Have you heard of The China Study?"

"But what about The China Study?"

"The China Study is based on tons of RESEARCH, where is yours?" and

"My friend/cousin/neighbor/sibling/pet became a vegan/vegetarian after reading The China Study--it was THAT convincing!"

Sigh.

Okay, so let's compile those reviews and research on why The China Study is NOT an insta-kill to the paleo/primal diet or low-carb approach.  I am breaking this topic into bite-sized portions since it is GINORMOUS (yes, that's a word)!  So today we'll tackle what the study said and the limitations of a study that large.

*crackles knuckles*  *takes a deep breath*


The China Study: What Is It?

From The China Study website (my emphasis in bold):
"The research project culminated in a 20-year partnership of Cornell University, Oxford University, and the Chinese Academy of Preventive Medicine, a survey of diseases and lifestyle factors in rural China and Taiwan. More commonly known as the China Study, “this project eventually produced more than 8000 statistically significant associations between various dietary factors and disease.” 
The findings? “People who ate the most animal-based foods got the most chronic disease … People who ate the most plant-based foods were the healthiest and tended to avoid chronic disease. These results could not be ignored,” said Dr. Campbell.
Wikipedia further elaborates on study size:
"The China Study," referred to in the title is the China Project, a "survey of death rates for twelve different kinds of cancer for more than 2,400 counties and 880 million (96%) of their citizens" conducted jointly by Cornell University, Oxford University, and the Chinese Academy of Preventive Medicine over the course of twenty years.
and provides the authors' recommendations:
The authors recommend that people eat a whole food, plant-based diet and avoid consuming beef, poultry, eggs, fish and milk as a means to minimize and/or reverse the development of chronic disease. The authors also recommend that people take in adequate amounts of sunshine in order to maintain sufficient levels of Vitamin D and consider taking dietary supplements of vitamin B12 in case of complete avoidance of animal products. The authors criticize "low carb" diets (such as the Atkins diet), which include restrictions on the percentage of calories derived from complex carbohydrates.
Bottom line: Dr. T. Colin Campbell and his team found that animal protein in the diet correlated with increased risk of disease through observational or epidemiological studies and meta-analysis.  His recommendation?  Go vegan.


The Limitation of Epidemiological Studies: Correlation Is NOT Causation, Peoples!

One of the major limitations of this kind of research is its breadth.  You can pull so much data together that it becomes muddied with confounding factors and the linkages you make are tenable at best.  Dr. Eades, author of Protein Power, has gotten so tired of arguing against these studies that he posted a reference about them:
Observational studies – also called prospective or cohort studies and sometimes even epidemiological studies – are the kind most often reported in the media simply because there are so many of them.  These are the studies in which researchers look for disease disparities between large populations of people with different diets, lifestyles, medications, incomes, etc.  If disease disparities are found to exist between groups, then researchers try to make the case that the difference in diet, lifestyle, medication, etc. is the driving force behind the disparity.
And meta-analyses:
For those who don’t know, meta-analyses are compilation studies in which researchers comb the medical literature for papers on a particular subject and then combine all the data  from the individual studies together into one large study.  This combining is often done to bring together a collection of studies, none of which contain data that has reached statistical significance, to see if the aggregate of all the data in the studies reaches statistical significance.  I think these types of meta-analyses are highly suspect, because they can lead to conclusions not warranted by the actual data. 
and the problem:
Researchers using meta-analyses set up selection criteria to pick which studies will be included in their final product, which leaves the door open for all kinds of mischief.   
Dr. Eades has a great analogy to share, so read the original reference, but it boils down to:
Problem is they can never possibly think of all the differences between the groups.  As a consequence, they never have a perfect study with exactly the same number, sex, age, lifestyle, etc. on both sides with the only difference being the study parameter. And so they don’t really ever prove anything.  
Observational studies only show correlation, not causation, a fact that everyone doing research and reading about research should have tattooed on their foreheads. 
Correlation can create a hypothesis for further testing.  That's it.  No light-bulb-over-the-head, ah-HA! moment of realization.  Sorry.


So where does The China Study come in?  The problem is that although The China Study definitely covers the Correlation is not Causation topic, it doesn't heed its own warnings.  It still slips down the rabbit hole when it says:
This does not mean that correlations are useless.  When they are properly interpreted, correlations can be effectively used to study nutrition and health relationships.  The China Study, for example, has over 8,000 statistically significant correlations, and this is of immense value.  When so many correlations like this are available, researchers can begin to identify patterns of relationships between diet, lifestyle and disease.  These patterns, in turn, are representative of how diet and health processes, which are usually complex, truly operate.  However, if someone wants proof that a single factor causes a single outcome, a correlation is not good enough.
Okay, I am getting a little nervous.  Correlations are now "patterns." Are they trying to make correlations sound more concrete?  Patterns are usually obvious and if obvious, does that mean they are real?Continuing down that hole:
After obtaining the results from a variety of studies, we can then begin to use these tools and concepts to assess the weight of the evidence.  Through this effort, we can begin to understand what is most likely to be true, and we can behave accordingly.  Alternative hypotheses no longer seem plausible, and we can be very confident in the result.  Absolute proof, in the technical sense, is unattainable and unimportant.  But common sense proof (99% certainty) is attainable and critical.  
Now my fears are realized.  The China Study authors have leapt from correlation to 'truth,' and "behave accordingly" sounds a whole lot like a prescription to me.  In one fell swoop, they've eliminated other hypotheses and don't even have to test their own due to their 'confidence in the result.'  Heck, why even bother seeking proof since it's "unattainable" and "unimportant."  "Common sense" is enough.  Well, maybe for them, but NOT for me.  And throwing out the animal-based diet that we evolved upon and that may even have been the impetus for our evolutionary path is NOT common sense to me!  

Okay, take a deep breath, Kristy...  Watch those cortisol levels.  I am just flustered at how well they can parlay the Correlation is not Causation topic and then with a final jab leave the average reader feeling satisfied that their breadth of research is enough to provide 'truth' and prescription.  Who the hell needs causation when you have those?


Think I am just crazy?  Think The China Study makes perfect sense?  Read Gary Taubes's (author of Good Calories, Bad Calories) thorough article for the New York Times.  He describes the leap from epidemiological study to preventative medicine as skipping vital experimental testing of the hypotheses epidemiological studies produce.  Why is this leap so often made?  Well, it's complicated:
The randomized-controlled trials needed to ascertain reliable knowledge about long-term risks and benefits of a drug, lifestyle factor or aspect of our diet are inordinately expensive and time consuming. By randomly assigning research subjects into an intervention group (who take a particular pill or eat a particular diet) or a placebo group, these trials “control” for all other possible variables, both known and unknown, that might effect the outcome: the relative health or wealth of the subjects, for instance. This is why randomized trials, particularly those known as placebo-controlled, double-blind trials, are typically considered the gold standard for establishing reliable knowledge about whether a drug, surgical intervention or diet is really safe and effective.
But clinical trials also have limitations beyond their exorbitant costs and the years or decades it takes them to provide meaningful results. They can rarely be used, for instance, to study suspected harmful effects. Randomly subjecting thousands of individuals to secondhand tobacco smoke, pollutants or potentially noxious trans fats presents obvious ethical dilemmas. And even when these trials are done to study the benefits of a particular intervention, it’s rarely clear how the results apply to the public at large or to any specific patient. Clinical trials invariably enroll subjects who are relatively healthy, who are motivated to volunteer and will show up regularly for treatments and checkups. As a result, randomized trials “are very good for showing that a drug does what the pharmaceutical company says it does,” David Atkins, a preventive-medicine specialist at the Agency for Healthcare Research and Quality, says, “but not very good for telling you how big the benefit really is and what are the harms in typical people. Because they don’t enroll typical people.”
These limitations mean that the job of establishing the long-term and relatively rare risks of drug therapies has fallen to observational studies, as has the job of determining the risks and benefits of virtually all factors of diet and lifestyle that might be related to chronic diseases. The former has been a fruitful field of research; many side effects of drugs have been discovered by these observational studies. The latter is the primary point of contention.
That latter is the basis for The China Study.  Still not convinced?  Read Lierre Keith's scour of epidemiological studies in The Vegetarian Myth, and her warning:
...until all the variables are controlled and the results reproducible, no conclusions can be drawn.  
Can you even begin to imagine the variables in a study as large as the China Study happily professes?  Remember it is based upon a "survey of death rates for twelve different kinds of cancer for more than 2,400 counties and 880 million (96%) of their citizens."  Even just small studies have nearly unlimited variables like diet, sleep, exercise, family, work, stress, transportation, socioeconomic status, environmental differences, genetic endowment, life history, having an ingrown toenail, etc.  How is anything meaningful ever said?  Well, researchers find correlations that create testable hypotheses and test them.  Over and over again.  Once they get the same results over and over again, THEN they can say something meaningful like X leads to greater risk of Y.  One study of 20 people and one study compiling the results of 300 different studies are just as meaningless when it comes to predictive power.

Gary Taubes (in the same New York Times article) provides a suggestion for critically evaluating scientific research:
So how should we respond the next time we’re asked to believe that an association implies a cause and effect, that some medication or some facet of our diet or lifestyle is either killing us or making us healthier? We can fall back on several guiding principles, these skeptical epidemiologists say. One is to assume that the first report of an association is incorrect or meaningless, no matter how big that association might be. After all, it’s the first claim in any scientific endeavor that is most likely to be wrong. Only after that report is made public will the authors have the opportunity to be informed by their peers of all the many ways that they might have simply misinterpreted what they saw. The regrettable reality, of course, is that it’s this first report that is most newsworthy. So be skeptical.
If the association appears consistently in study after study, population after population, but is small — in the range of tens of percent — then doubt it. For the individual, such small associations, even if real, will have only minor effects or no effect on overall health or risk of disease. They can have enormous public-health implications, but they’re also small enough to be treated with suspicion until a clinical trial demonstrates their validity.
If the association involves some aspect of human behavior, which is, of course, the case with the great majority of the epidemiology that attracts our attention, then question its validity. If taking a pill, eating a diet or living in proximity to some potentially noxious aspect of the environment is associated with a particular risk of disease, then other factors of socioeconomic status, education, medical care and the whole gamut of healthy-user effects are as well. These will make the association, for all practical purposes, impossible to interpret reliably.
The exception to this rule is unexpected harm, what Avorn calls “bolt from the blue events,” that no one, not the epidemiologists, the subjects or their physicians, could possibly have seen coming — higher rates of vaginal cancer, for example, among the children of women taking the drug DES to prevent miscarriage, or mesothelioma among workers exposed to asbestos. If the subjects are exposing themselves to a particular pill or a vitamin or eating a diet with the goal of promoting health, and, lo and behold, it has no effect or a negative effect — it’s associated with an increased risk of some disorder, rather than a decreased risk — then that’s a bad sign and worthy of our consideration, if not some anxiety. Since healthy-user effects in these cases work toward reducing the association with disease, their failure to do so implies something unexpected is at work.
All of this suggests that the best advice is to keep in mind the law of unintended consequences. The reason clinicians test drugs with randomized trials is to establish whether the hoped-for benefits are real and, if so, whether there are unforeseen side effects that may outweigh the benefits. If the implication of an epidemiologist’s study is that some drug or diet will bring us improved prosperity and health, then wonder about the unforeseen consequences. In these cases, it’s never a bad idea to remain skeptical until somebody spends the time and the money to do a randomized trial and, contrary to much of the history of the endeavor to date, fails to refute it.

I think I have given your brain enough food for thought for today.  At least now your first line of defense against The China Study question is that it can provide correlation but NOT causation.  There is NO predictive value through The China Study that an animal-based diet causes chronic disease or that a plant-based one does not.  There are no X leads to Y conclusions possible.  Period.

Feast upon it and we'll come back to discuss the reviews and research.  Enjoy your weekend!

Here is the next part: The Study Everyone Talks About Part 2: The Ravaging Reviews