This week, Nature has an online news article and editorial about a new meta-analysis published in the British Medical Journal (BMJ). The BMJ article addresses discrepancies in published reports of clinical trials of adult stem cells to treat patients with heart disease/heart failure. For some time now, this has appeared to many to be a particularly promising use of adult stem cells.
Well, not so fast, maybe.
For the BMJ article, a cardiology research team took it upon themselves carefully to read reports of 49 early trials, intended to get a first look at whether the use of bone marrow mesenchymal cells, a type of adult stem cell, to treat people with heart failure is potentially safe and effective (in this case, by improving the “ejection fraction,” or blood-pumping effectiveness, of the heart). Positive results would not prove the stem cell treatment works, but only that doctors might be convinced enough to mount large, definitive “Phase 3” trials. Such trials are in progress. Not only that, but this kind of stem cell treatment is already being sold to people in some parts of the world.
The BMJ writers looked for “discrepancies”—instances where two or more facts were simultaneously asserted but could not, logically or mathematically, be true at the same time and in the same way. That is, they were contradictions. They also looked at irregularities—not frank contradictions within the data, but things that appear obviously amiss. Most of the 49 studies had discrepancies: 5 had none, 24 had 1-10, 12 had 11-20, 3 had 21-30, and 5 had more than 30 discrepancies. Moreover, the more discrepancies the trials had, the better were the results they claimed. The 5 with no discrepancies showed no benefit at all to the stem cell treatment.
The kinds of discrepancies and irregularities reported make a clinical trialist’s skin crawl. To name a few items: Some studies were said to be randomized but weren’t. Some data, that should have shown scattered results, were clustered together; that is, they were “too good to be true,” often a sign that the data could have been manipulated. Data were graphed for more people than were said to be on the entire study. Statistics were miscalculated. Results were listed for people after they were said to have died. (No mention is made of whether they voted in concurrent elections.) Consent procedures were improper. Some control subjects got an infusion of an unapproved culture medium—a safety risk and apparent regulatory violation.
It all makes one wonder whether the investigators, or the journals that published the reports, were paying attention, or whether ostensibly promising results were accurately measured and analyzed. It sounds like they may well not have been.
Does this prove that the studies were in fact negative, or that the stem cell treatments don’t, or won’t, or can’t work? No. But it suggests that results were hyped and spun, something that is usually, but not only, associated with industry.
Briefly, four implications for ethical discussions of stem cell treatments:
- Don’t take everything you read in the literature, much less the press, at face value. There are lots of details behind headline claims. Maybe those will become more transparent, but sifting through them will often be more than the non-specialist can or is willing to do.
- Beware basing ethical arguments on the notion that “this [ethically preferred] approach will be better/just as good as” an approach that is more ethically questionable. The ethics of the experimental goals and methods provide the main underpinnings of ethics arguments.
- That said, given that risk-benefit assessments are central to certain ethics reviews—specifically, to the work of an IRB—the conscientious IRB member is in a pickle. Does an IRB have to audit, in detail, the prior work on which a clinical trial is based? Not feasible.
- “Scoreboards” of “successes” with adult vs embryonic stem cells are uninterpretable and should generally be avoided.