{"items": [{"author": "James", "source_link": "https://plus.google.com/106345404829653994850", "anchor": "gp-1384993751121", "service": "gp", "text": "I take a different lesson from these examples: don't trust a study you haven't actually read (or had a Metamed researcher read); and reasoning about mechanisms works pretty well.\n
\n
\nThe 2007 study you link to as showing \"people randomly assigned to drink regular or decaf coffee were equally likely to have miscarriages\" has only 12 miscarriages in it; the data analysis is exclusively about birth weights.\n
\n
\nThe 2008 RCT of soy in infant formula that you link to is surprisingly terrible.\u00a0In the Methods section, they randomize into three groups - but the results are only ever broken down into soy/non-soy, not into the three randomized groups.\u00a0All of the actual numbers in the RCT show a positive correlation between soy and allergies, but with p-values greater than 0.05, because they only had 40 actual allergy cases to work with (not n=620).\u00a0A third of their study population ignored the random assignment and used parent-selected infant formula products, but weren't excluded from the data set. And the real kicker is, when\u00a0I searched for the \"cow's milk-based formula\" by name, Nestle NAN, I found an ingredients list with soy oil in it. (I can't be sure that it's the same recipe, though, because Nestle has a lot of similarly-named products and the paper doesn't have enough info to be confident about which one it is.)", "timestamp": 1384993751}, {"author": "Daniel", "source_link": "https://www.facebook.com/jefftk/posts/638032435762?comment_id=638052530492", "anchor": "fb-638052530492", "service": "fb", "text": "I agree with the general point, but not the application to the specific case of blood donation health benefits. I have a strong prior belief that blood donation is beneficial for men, and the negative evidence in the medical literature doesn't change my prior much.", "timestamp": "1384998678"}, {"author": "Hollis", "source_link": "https://www.facebook.com/jefftk/posts/638032435762?comment_id=638053318912", "anchor": "fb-638053318912", "service": "fb", "text": "Is that a hypothetical situation there, the one with the tongue and the citric acid? ;-)", "timestamp": "1384999112"}, {"author": "Jeff Kaufman", "source_link": "https://plus.google.com/103013777355236494008", "anchor": "gp-1385003832668", "service": "gp", "text": "@James\n\u00a0\"The 2007 study you link to as showing ...\u00a0has only 12 miscarriages in it\"\n
\n
\nYou're right, this is bad. \u00a0I included it after seeing someone else refer to it as showing \"no effect\" and didn't examine it closely enough. \u00a0It was not powerful enough to detect the claimed effect and probably started people on caffeine too late anyway. \u00a0I've added an update to the post\n
\n
\n\"they randomize into three groups - but the results are only ever broken down into soy/non-soy, not into the three randomized groups\"\n
\n
\nI think that's because they're using the randomization from a larger experiment? \u00a0That is strange.\n
\n
\n\"A third of their study population ignored the random assignment and used parent-selected infant formula products, but weren't excluded from the data set.\"\n
\n
\nI'm reading the study as saying \"we told people to use X and some also used some Y\" not \"we told people to use X and some ignored us and used Y instead\".\n
\n
\nAs for excluding them from the data, the closest they got was: \"Exposure to the randomly allocated soy formula was then considered separately from exposure to parent-selected soy products. A logistic regression model was fitted that simultaneously allowed for separate effects of peanut sensitization caused by consumption of the randomly allocated soy formula and by consumption of any parent-selected soy formula or soy milk. There was no evidence of an association between consumption of the randomized soy formula and peanut sensitization (OR, 1.35; 95% CI, 0.67-2.703; P = .399). By contrast, consumption of a parent-selected soy formula or soy milk was associated with sensitization to peanut at 2 years of age compared with those who had not ingested soy (OR, 2.02; 95% CI, 1.04-3.92; P = .039).\"\n
\n
\nI'm reading this as saying \"parent supplied soy is correlated with peanut sensitization, but randomly supplied soy was not\". \u00a0Is this your reading? \u00a0I'm not sure what they mean by fitting a logistic regression model in this case.\n
\n
\n\"Nestle NAN, I found an ingredients list with soy oil in it.\"\n
\n
\nPrevious studies had found things like \"Of the total cohort of children, 8.3 percent had consumed soy milk or soy formula in the first two years, as compared with 24.5 percent of those with peanut allergy and 34.8 percent of those with a positive peanut challenge (P<0.001 for both comparisons).\" (From the 2003 study linked in the post.)\n
\n
\nUnless most standard formula doesn't have soy oil but Nestle NAN does?", "timestamp": 1385003832}, {"author": "James", "source_link": "https://plus.google.com/106345404829653994850", "anchor": "gp-1385059076728", "service": "gp", "text": "@Jeff Kaufman\n\u00a0 \"I'm reading this as saying \"parent supplied soy is correlated with peanut sensitization, but randomly supplied soy was not\". \u00a0Is this your reading?\"\n
\n
\nNot quite. My reading is: parent supplied soy is correlated with peanut sensitization (odds ratio=2.02, 95% CI 1.04-3.92, p=.039), but randomly supplied soy was only a little bit correlated with peanut sensitization and because we only have 40 actual allergy cases we declared it insignificant (odds ratio=1.35, \u00a095% CI 0.67-2.703, p=.399). If soy were a 2.7-to-1 risk factor, that would be inside their confidence interval, and that's on top of all the other wacky problems. So I count this one as \"no meaningful test was performed\", not as a falsification.", "timestamp": 1385059076}]}