Theranos: Stay away from Hype-I

I stumbled on this interesting paper and wanted to share it with you all. I have included the pertinent summary. I’ll be writing more soon on this.

Theranos: five investor lessons from the downfall of Elizabeth Holmes | Financial Times

Hype is intrinsic to the culture of Silicon Valley. The computer industry’s marketing inspired the term “vapourware” back in the 1980s. But overpromising and underdelivering is unacceptable in healthcare, where real human misery may result.
In healthcare, that sort of transparency is essential — though there are growing concerns Silicon Valley’s paranoid culture is eroding it. John Ioannidis, a Stanford professor and early sceptic on Theranos, co-authored a paper showing that more than half of healthcare unicorns have few citations of their work.

Summary:

• In 2014, one of us (JPAI) wrote a viewpoint article coining the term “stealth research” for touted biomedical innovation happening outside the peer‐reviewed literature in a confusing mix of “possibly brilliant ideas, aggressive corporate announcements, and mass media hype”.1 These reflections were prompted by Theranos, a medical diagnosis start‐up company; Theranos had not published any peer‐reviewed papers,[1] but made claims that its technology would “disrupt medicine.” in contrast to the tech sector, in healthcare published peer‐reviewed research is essential to ensure a minimum threshold of transparency, accountability and credibility for the underlying work in the scientific community.
• The viewpoint article on Theranos and its stealth research was reviewed at JAMA, eventually accepted on December 2014 and published in February 2015.1 Theranos at that time had the highest valuation ($9Bn) of all healthcare‐related “unicorns”.[2].
• The authors further examined how the peer‐reviewed publication records of unicorns compared against start‐ups with lower valuations.
• The authors identified 413 papers, of which 47 (11%, including nine reviews) were highly cited.
• More than half of the current unicorns (10/18) and almost 40% of the exited unicorns (12/29) had no highly cited papers.
• Five (Enobia Pharma, Neotract, Qualicorp, Cameron Health and China Nuokang Biopharmaceutical) had no published papers and two (Flexus Biosciences and Cardioxyl Pharmaceuticals) had just two.
• 23andMe (107 articles) and Adaptive Biotechnologies (89 articles) published almost half of all unicorn papers.
• Of the 34 papers, the 16 highly cited were genetic association studies, and none, except a retrospective analysis, described trial data.
• Two top‐ranking companies published 11 (Fibrogen) and 9 (Ironwood Pharmaceuticals) highly cited papers; 22 of the 43 highly cited papers represented basic/preclinical research, 13 were phase I or II trials, and 1 was a phase III.
• WITH FEW OR NO HIGHLY CITED PAPERS: CASE STUDIES
• There was a negative, non‐statistically significant association between company valuation and number of published (r = −0.16, P = 0.28) or highly cited papers (r = −0.17, P = 0.25).
• The authors scrutinized companies with a high valuation and very few or no highly cited papers.
• Stemcentrx, bought in 2016 by AbbVie for $10.2 Bn, was the second largest private VC‐backed acquisition in history.[10] The company has 16 published papers, and one highly cited one, describing preclinical research on a monoclonal antibody in tumour models.
• The company has nine published and one highly cited paper, an uncontrolled phase I/II trial for relapsed chronic lymphocytic leukaemia.
• The highest‐valued unicorn ($5.5 Bn), Intarcia, had six published papers, none surpassing 50 citations.
• Outcome Health and United Imaging Healthcare ranked next to Intarcia in valuation ($5 Bn each), but neither had any highly cited papers.
• Wall Street Journal claimed it misled advertisers, providing inflated and inaccurate data about the performance of its products.[16] These had already been in use in several hospitals or medical practices despite no data about their use.[17] The company was forced to let off employees, offer buyouts to more than a third of its workforce and settle two lawsuits—one of which was instigated by prominent investors; it ousted its founders first from their positions as CEO and president and completely from the board.[18] Investors, customers and professional associations that provided content, like the American Medical Association, distanced themselves from the company.[18] Little media information is available for United Imaging Healthcare, except for accusations of bribery in exchange for insider information on federally funded research.[19]
• Does the dearth of peer‐reviewed research apply only to unicorns or does it extend to start‐ups with lesser valuations?
• The authors extracted information about published papers and citations for matched competitors.
• The index unicorn had published more PubMed‐indexed papers than the competitor in seven cases, and in three pairs, both companies published no papers (Wilcoxon signed‐ rank P = 0.01; P = 0.02 for highly cited papers).
• Some start‐ups publish few papers, but their founders or leaders may be influential scientists.
• The authors reviewed the company website (July 2018) to identify all individuals listed as founders, leadership, members of the board of directors or the scientific advisory board.
• The authors cross‐checked each individual against a published list of the 84 116 most highly cited scientists, compiled from Scopus.[20] The list includes all scientists who are among the top 30 000 in at Unicorn Outcome Health Moderna Therapeutics Oscar Health Insurance ZocDoc Oxford Nanopore Technologies GuaHao/Weiyi 23andMe Human Longevity iCarbonX Adaptive Biotechnology
• Founders were identified for 13/18 unicorns, and in three cases (Oxford Nanopore Technologies, Human Longevity, iCarbonX), these were highly cited scientists.
• Information about scientific advisory boards was available for only 4/18 unicorns: Adaptive Biotechnology, Moderna Therapeutics (9 highly cited scientists), Human Longevity (8 highly cited scientists) and CureVac (5 highly cited scientists).
• Scientific advisory boards may be more populated with highly cited scientists, but information about their composition was frequently missing from most company websites.
• The highest‐valued healthcare start‐ups, past or present, contribute minimally to relevant, high‐impact published research.