April 26, 2018

Stanton A. Glantz, PhD

When assessing models that estimate the effects of e-cigarettes, the devil is in the details

Ken Warner and David Mendez recently published another population model to estimate the public health effects of the advent of e-cigarettes, “E-cigarettes: Comparing the Possible Risks of Increasing Smoking Initiation with the Potential Benefits of Increasing Smoking Cessation,” in Nicotine and Tobacco Research.  In contrast to several (but not all) earlier models, they found  population benefits, “[w]ith base-case assumptions, the population gains almost 3.3 million life-years by 2070.”

The question several people have asked me, is why such different conclusions?

Most of these models are generally structured to project the effects of e-cigarettes over time as the population ages and people enter and leave the tobacco market by starting to use tobacco, stopping using tobacco, or dying.  (The one Sara Kalkhoran and I developed is a steady state model at equilibrium.)  While there are some differences in how people to this, these differences do not explain the different outcomes.

The thing that makes the big difference is the assumptions that the different modelers make about the behavioral and health effects of e-cigarettes.  Despite saying that they are using “conservative” assumptions about e-cigarettes Warner and Mendez model makes very optimistic assumptions, most notably:

  • They assume that vaping increases smoking cessation by 10% even though the overall literature shows that e-cigarettes depress smoking cessation by about 20%
  • They assume that e-cigarettes are harmless

As justification for assuming 10% increase in smoking cessation, Warner and Mendez rely on three studies, two from England (study 1, study 2) and one from the US by Zhu et al, both of which show increased quitting.  The Zhu paper is well-done, but focuses on the short-term (one year) effects of e-cigarettes on quitting and does find increases in quitting.  A close reading of the data in their paper, however, shows increased relapse to smoking in the out years

Warner and Mendez ignore the much larger number of papers that show depressed quitting.

Assuming that e-cigarettes are harmless is even more extreme than the evidence-free “95% safer” number still being quoted in England and by e-cigarette enthusiasts here and around the world.

Consistent with good modeling practice, they do a sensitivity analysis where they examine other possible values for these key parameters (including the gateway effect on kids, which they estimate as a 2% increase in initiation), but they do not examine a wide enough range of effects, particularly allowing for the possibility that e-cigarettes depress quitting.

This is important because this is essentially the same model that is used in the report on e-cigarettes by the National Academy of Sciences, Engineering, and Medicine, which suffers from similar problems.  (Mendez was a member of the committee that wrote the report.)

So far, the best available model is the one by Soneji et al, which shows, even making optimistic assumptions about the effects of e-cigarettes on quitting and using the “95% safer” number, shows substantial negative population effects because for every smoker who quits, 8 kids start smoking.  They also validate their model against what actually happened over the first few years and test a wider range of parameter values.  Not surprisingly, using more realistic values for the effects of e-cigarettes on smoking cessation and the risks of using them, the population effects are even more negative.

Here is the abstract from Warner and Mendez:

Introduction:  The public health community is divided regarding electronic cigarettes. Skeptics emphasize potential vaping-induced increases in smoking among children and possible health hazards for adults. Enthusiasts consider e-cigarettes much less dangerous than smoking and believe they increase adult smoking cessation. We compare potential health benefits and costs to put these two perspectives in context.

Methods:  Using a dynamic model that tracks the US adult population's smoking status and smoking-related deaths over time, we simulate the effects of vaping-induced smoking initiation and cessation on life-years saved or lost to the year 2070. The base case assumes that vaping annually increases smoking initiation by 2% and smoking cessation by 10%. Sensitivity analyses raise the initiation rate increase to 6% while decreasing the cessation rate increase to 5%. Sensitivity analyses also test vaping's reducing the health benefits of quitting smoking by 10%.

Results: With base-case assumptions, the population gains almost 3.3 million life-years by 2070. If all people who quit smoking by vaping lose 10% of the benefit of quitting smoking, the net life-year gain falls to 2.4 million. Under worst-case assumptions, in which vaping increases smoking initiation by 6% and cessation by 5%, and vaping-induced quitters lose 10% of the health benefits, the population gains over 580,000 life-years.

Conclusion:  Potential life-years gained as a result of vaping-induced smoking cessation are projected to exceed potential life-years lost due to vaping-induced smoking initiation. These results hold over a wide range of plausible parameters.

Implications:  Our analysis strongly suggests that the upside health benefit associated with e-cigarettes, in terms of their potential to increase adult smoking cessation, exceeds their downside risk to health as a result of their possibly increasing the number of youthful smoking initiators. Public messaging and policy should continue to strive to reduce young people's exposure to all nicotine and tobacco products. But they should not do so at the expense of limiting such products' potential to help adult smokers to quit.

The full citation is Warner K. Mendez D.  E-cigarettes: Comparing the Possible Risks of Increasing Smoking Initiation with the Potential Benefits of Increasing Smoking Cessation. Nicotine Tob Res. 2018 Mar 29. doi: 10.1093/ntr/nty062. [Epub ahead of print] It is available here.

 

Comments

Comment: 

On behalf of my co-author and myself, I want to respond to Dr. Glantz's post. Dr. Glantz's analysis of our paper reflects his well-known biases about vaping. He makes several points that we believe are misleading. They include the following:

1. He begins with the statement "In contrast to several (but not all) earlier models, they found population benefits." This is misleading because it suggests that there may be a balance between studies that have found population benefits and those that have found net costs. As we review in our article, the majority of the analyses find net benefits. Only one to date has found an unequivocal net cost (Soneji et al.) and Dr. Glantz cites it as the "best available model." It's not surprising he likes it. It relies on his assessment of the literature that finds that vaping decreases smoking cessation. That analysis has been challenged by Villanti et al. and others. Of the simulation studies that do not find (unequivocal) net benefits, those other than Soneji do so because they consider an enormous range of outcomes, many of which are, frankly, ridiculous (highly excessive risks of reduced-risk products, excessive new adoptions of them by non-smokers, etc.). Those studies readily acknowledge the possibility of net benefits. Ironically, Dr. Glantz's own model (Kalkhoran and Glantz) found positive net benefits unless the risks of e-cigarettes were in the range of 30% those of smoking or greater. Given his interpretation of the risks of vaping, his paper was equivocal about whether vaping would produce net benefits or costs.

2. Dr. Glantz says that our estimate of the impact of vaping on smoking cessation - 10% in our base case - is optimistic. (Note that that 10% increase is on top of an underlying cessation rate of about 4.35%, so it represents an increase in cessation of less than one-half a percentage point.) Dr. Glantz then says that "the overall literature shows that e-cigarettes depress smoking cessation by about 20%." To validate this, he cites his own new Annual Review of Public Health article. That edition of the Annual Review also includes a review by David Abrams et al. that comes to the opposite conclusion, as have several reviews of the literature. Dr. Glantz does not cite any of them in this blog post. Clearly, if vaping decreases smoking cessation on balance, ALL of the models (including his own) would necessarily have to come to the conclusion that vaping represents a net cost to public health. One more important observation: The rate of decline in cigarette smoking in the US has accelerated greatly in the most recent years for both total cigarette consumption and youth and adult prevalence. These are precisely the years of vaping's growth. There is no good explanation for the decreased smoking deriving from the evidence-based interventions that have been documented to decrease smoking in the past - no major tax increases, no significant growth in smoke-free workplaces, no consequential changes in advertising and marketing policies. The only potentially contributing intervention is a series of mass media antismoking campaigns by FDA, CDC, and the Truth Initiative. While I suspect these campaigns are contributing to decreasing smoking, the magnitude of their potential impact, based on previous research, is certainly inconsistent with the 50% (or so) increase we've observed in adult smoking cessation rates only recently. I am not concluding that this is evidence that vaping is increasing smoking cessation (and we do not cite it as such in our paper). Rather, it is not consistent with the proposition that vaping is increasing smoking among either young people (by increasing initiation) or adults (by decreasing cessation).

3. Dr. Glantz says that we assume that e-cigarettes are harmless. This is false. We do not assign a risk to them in the base case because we assume (as evidence is increasingly showing) that most people who vape stop doing so, including people who quit smoking by vaping. (See, in particular, the recent data from ASH UK.) For all such people, the only relevant risk would be the risk from short-term use, which we understand to be minimal. While Dr. Glantz acknowledges that we do sensitivity analyses that "examine other possible values for these key parameters," he does not acknowledge (does not even mention) that one of those values attributes a vaping-induced 10% reduction in mortality savings from quitting smoking. This is a strong and conservative assumption, since it's the equivalent to saying that (for example) 40% of vapers will continue vaping for the rest of their lives after quitting smoking and, in the process, will incur a mortality risk 25% that of continued smoking. To my knowledge, the only published estimates of risk being that high or higher come from Dr. Glantz and his colleagues - and again, this is assuming a lifetime of vaping after quitting smoking, which is itself not consistent with the data. Even with this assumption, our most conservative sensitivity analysis still finds a positive net health benefit from vaping. Note that that sensitivity analysis included a halving of our base case cessation benefit (therefore an increase in cessation of only 5% attributable to vaping) and a tripling of what we believe to be the increase of vaping on smoking initiation implied by the 3.50 adjusted odds ratio presented in the Soneji et al. meta-analysis of the "gateway" studies (a paper written by the authors of those analyses)

Dr. Glantz's assessment of our study is based exclusively on his interpretations of the effects of vaping (or rather his and those of his co-authors), judging from the fact that all of his cites to the evidence contradicting our assumptions are his papers. He does not present any of the evidence contrary to his own, despite the size and credibility of that body of evidence. His choice of the "best available model" (Soneji et al.) uniquely concludes that vaping produces net public health costs, in contrast with the more than a dozen other published studies (including Dr. Glantz's own), and it does so solely because it accepts Dr. Glantz's interpretation of the effects of vaping including, most significantly, the conclusion that vaping harms cessation.

The recent committee of the highly regarded National Academy of Sciences, Engineering, and Medicine that studied vaping concluded that under the most plausible assumptions, e-cigarettes will result in net benefits. That cannot happen unless vaping increases smoking cessation.

Comment: 

Clearly there is a problem with the underlying data for all the models. The NASEM committee reviewed these studies and concluded "there is LIMITED evidence that e-cigarettes may be effective aids to promote smoking cessation." This was an appropriate conclusion given the quality of the evidence.

Critical to the Warner and Mendez model is their decision that 10% is a reasonable estimate for the degree to which e-cigarette use will increase cessation. NASEM did NOT reach this same conclusion. To get this estimate, Warner and Mendez rely on a high estimate from a series of cross-sectional studies reported by Zhu. Cross-sectional studies are at the bottom end of quality for assessing causal inference. Then, for their sensitivity analysis, they assumed that the minimum increase in cessation would be 5%! These estimates should be treated as advocacy statements rather than scientifically valid estimates of effect size.

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.