Shiffman et al paper in Nicotine & Tobacco Research is not a reliable estimate of effects of ecig flavors

On January 7, 2015, Saul Shiffman and colleagues published "The impact of flavor descriptors on nonsmoking teens’ and adult smokers’ interest in electronic cigarettes" in the journal Nicotine and Tobacco Research.  This paper, reported work funded by the NJOY e-cigarette company, concluded that "The e-cigarette flavors tested appealed more to adult smokers than to nonsmoking teens, but interest in flavors was low for both groups."
 
This is an important conclusion for the e-cigarette industry, which has been challenged by evidence that flavors attract kids to e-cigarettes. 
 
Robert Jackler from Stanford has asked me to post this on his behalf about the Shiffman et al paper.
 
A Comment on the Context of My Critique:
 
Earlier this year, Stan Glantz submitted a critique of the Shiffman et al paper on flavor preferences in eCigs to the journal which published the original work (Nicotine & Tobacco Research).  As it is customary practice for journals to encourage responsible criticism to appear in their pages, I was more than a little surprised to learn that they refused to publish his comments.  After Stan’s critique later appeared on his blog, I decided to also submit my own comment to NTR.  As I was at the time a consultant to the California Attorney General’s office on negotiations related to the paper’s sponsoring company (NJOY) flavored eCig practices I needed to delay my submission until this matter had resolved.   
My commentary (see below) emphasizes both my concerns over the scientific validity of the paper and what I view as compelling evidence of commercial bias.  I find it troubling that NTR also rejected my commentary as well.  Refusing to publish both Stan and my critiques raises questions about the journal’s commitment to thoughtful dialog on issues of scientific validity and bias.   In rejecting my critique, NTR explained that conflict of interest among the authors had been disclosed.  However, merely disclosing COI does not excuse any and all sorts of commercial bias contained in the design of the research and/or interpretation of the data.  As disclosure is meant to help readers to consider the possibility of bias, it seems most appropriate for the journal to publish letters from responsible academics who perceive commercial interest and seek to share their rationale with the scientific community.
In rejecting my comment, NTR went on to say: “Your letter I think implies that the article has significant flaws that mean it should not have been published in the first place, but it has been published and the journal's view is that the final accepted version dealt adequately with the original reviewer's comments.”  By definition, all published papers have passed the journal’s expert peer review process and all subsequent letters to the Editor follow this approval.  Having long served as an Editor-in-Chief of a major scientific journal, I am keenly aware of the limitations of the peer review process and the importance of responsible post publication commentary to enhance the value of scientific literature.
 
My Commentary on Shiffman et. al. study:
 
The paper ‘Impact of flavor descriptors on non-smoking teens and adult smokers’ interest in electronic cigarettes” by Shiffman and coworkers suffers from flaws in experimental design and data interpretation which call into question their conclusions that flavored e-cigarettes are not of differential appeal to teens.1 As discussed in detail below, their survey methodology has a high likelihood of selection bias, the study groups are too dissimilar to enable the comparisons made, informed consent for minors was not obtained despite evident risks, and commercial interest appears to have influenced the study’s outcome.
 
In comparing the relative appeal of flavored e-cigarettes to non-smoking teens versus smoking adults, the obvious hypothesis would be that flavors would be of greater attraction to the young.  The factual basis for this conjecture is the extensive body of research which found that flavored tobacco products have differential appeal to youth. This led the US Congress to ban flavored cigarettes in 2009.  In a broader context, a finding that adolescents have no preference for sweet and fruity flavors would mean that e-cigarettes somehow are a special exception to well established consumption trends throughout the food and beverage industry. 
 
Concerns over survey validity:
 
The logical way to determine the differences in e-cigarette flavor preferences between adults and teens would be to compare the actual frequency of flavor use by teen and adult e-cigarette users.  Rather than use a direct method, Shiffman et. al. compared a small cohort of teen non-smokers with adult smokers via a marketing survey conducted for the sponsoring e-cigarette brand (NJOY).  There is reason to question the validity of comparing non-smoking teens with adult smokers as they are notably unequal groups. Nationally only approximately 15% of teens smoke whereas the entire adult survey group smoked.   In this survey teen smokers were systematically excluded. 
 
The Shiffman et. al. study consisted of an online survey inquiring about interest in products of the NJOY e-cigarette brand.  These included then current flavors (tobacco and menthol) and a list of contemplated flavors.  A survey of flavor preferences for bottled water and ice cream was used as a comparison.  Invitations were sent to 20,235 adults and 14,151 teens with a mere 432 adults and 216 teens enrolled. This is an under 2% response rate. With such a low response rate, responders may have had a special motivation to participate.  This raises the likelihood of selection bias, differences between responders and non-responders, a factor which undermines the value of surveys with a small percentage of responders.2
 
A sizable fraction of teens are opposed to smoking, some with notable vehemence instilled by parents and schools.  It would be expected that a fraction of teen responders felt a special impetus to do so as a means of communicating their opposition to tobacco products.  Two sorts of teens are likely to reply to such a survey: those contemplating e-cigarette use who may be attracted to the products being surveyed and those who are vehement in their intention to neither smoke nor vape and are motivated to send a rejectionist message. The motivation of teen responders to oppose smoking is the most probable reason for the low interest in e-cigarettes which the authors erroneously interpreted as a lack of interest in flavored e-cigarettes.   That the paper reaches the conclusion: “Nonsmoking teens’ interest in e-cigarettes was very low” is yet another reason to question the survey’s validity.   This observation is at variance with 2014 CDC data showing a rapid rise in e-cigarette use among high school students coupled with a reduction in combustible cigarette use.
 
Not surprisingly, their survey showed non-smoking teens queried on common consumer products (ice cream and flavored water) indicated widespread interest in flavors.   In the adult cohort, flavor preferences for e-cigarettes, other than tobacco and menthol, roughly paralleled that of ice cream and water. (Shiffman et al. Figure 3b)  That teens expressed a liking for flavored ice cream, but not e-cigarettes, may well be accounted for by responses of teens who had no interest in e-cigarette products.  (Shiffman et. al. Figure 3a)  It was not that these non-smoking teens did not like the e-cigarette flavors, but rather they categorically were uninterested in NJOY’s products in all flavor variations.
 
In their flavor analysis, the paper’s data reveals a marked preference among adult smokers for tobacco flavor and menthol flavors with all other flavors (except vanilla) appearing at a fraction of their appeal.  (Shiffman et al. Figure 1)  Among teen non-smokers, by contrast, interest was essentially equal across flavors with no preference for tobacco or menthol flavors.  One interpretation of these data is that, relatively speaking, non-smoking teens prefer a variety of sweet and fruity flavors while adult smokers prefer tobacco flavor.  This interpretation of their data is the opposite the authors’ conclusions.
 
Inappropriate Claim of Exemption from IRB for Survey of Minors
 
The Shiffman et. al. survey offers no benefit to the minors surveyed and entails significant risks. Exposing teens to a harmful product they may not have considered using, such as use of e-cigarettes and flavored e-cigarettes, could increase the likelihood that they will try the product and develop nicotine addiction.  This represents a clear risk to the future health of the minor. The existence of risk, coupled with the lack of benefit to the individual child, necessitates both institutional review and parental informed consent.3
 
Evidence of Commercial Bias
 
The authors’ declaration in the Shiffman et. al. paper discloses that the study was sponsored by the NJOY e-cigarette brand.   It also reveals that the company was involved in determining the study design.  The survey was conducted by a ResearchNow a company which conducts surveys for commercial marketing purposes on subjects such as customer satisfaction and loyalty. All of the study authors were all employees of the Pinney Associates a firm who represents both the tobacco industry (Reynolds American) and e-cigarettes (NJOY - Soterra Inc).  Pinney Associates describe their service to companies: “Evaluate, support and develop new commercial opportunities including new products and claims; Manage issues and the scientific and policy environments to gain regulatory approval for new products; and “Enhance the scientific and policy environments to maximize product acceptance.”4
 
Given the long history of tobacco industry manipulation of scientific research, the fact that the Shiffman et. al. study was designed and sponsored by an e-cigarette brand gives reason for skepticism.   That the authors we all employed by a company contracted to NJOY whose goal is to “Enhance the scientific and policy environments to maximize product acceptance” further raises concern over the independence and objectivity of the paper.  The circumstances under which the paper arose, as illustrated by the timeline below, indicates that NJOY had a powerful motivation to commission a paper exculpating e-cigarette flavors from preferential youth appeal proximate to the time that Pinney Associates was engaged and the survey was conducted.
 
In December 2009, NJOY CEO declared that they would halt flavor sales to discourage underage use and to align with the Family Smoking Prevention and Tobacco Control Act recently passed by Congress.5 Between 2012 and 2013 NJOY lost its lead position in market share to Blu (2013 Blu 44.1%, NJOY 24.3%).6  By the second quarter of 2014, Logic overtook NJOY as the number two brand in all tracked channels.7   Lorillard’s purchase of Blu and the 2014 emergence of RJ Reynolds (Vuse) and Altria (Mark Ten) into the e-cigarette market further increased NJOY competition.   The New York Times quoted NJOY CEO Craig Weiss as having said in 2013 that other manufacturers use flavors to attract children.8 This New York Times article of July 2014 described NJOY as having chosen to resume flavor sales, after a 5 year hiatus, due to a 50% decline in convenience store sales over the preceding year.  In the month following re-introduction of flavors in July 2014, NJOY realized a 65% increase in convenience store sales.9
 
NJOY leadership was undoubtedly concerned about possible repercussions from their reintroduction of flavors.  After all, over a four year period two of their CEOs had publicly linked e-cigarette flavors with youth.  Complicating matters further was the 2010 they a consent judgment they agreed to in the nation’s largest state (California) stipulating that they would not market flavored e-cigarettes.10   In an effort to manage anticipated regulatory and public relations difficulties, NJOY engaged Pinney Associates in January 2014.   As part of this campaign, it appears that Pinney Associates was commissioned to create a scientific publication with the expectation that it would conclude that flavored e-cigarettes possess no special attractiveness to youth.  The work product of this engagement was the scientifically unconvincing Shiffman et. al. study in which the authors seem to have gone to great length to conjure a methodology which would appear to support their sponsors’ desired outcome.
 
Shortly after the July 2014 re-introduction of flavored NJOY products, the California Attorney General contacted the company to request compliance with the 2010 judgment.  In September 2015, the Attorney General’s office agreed to allow flavored NJOY products to remain on the market in California in exchange for certain limitations on youth marketing.11
 
In summary, it appears that the Shiffman et. al. paper was not hypothesis driven research, but rather a study commissioned by NJOY to create cover for their reversal on flavors driven by dropping market share.  Both the circumstances under which it came about, and the scientific content of the work, suggest that the result was not an open ended question but rather attained a result preordained by the sponsor.   Fundamentally it was “science” commissioned to support a product launch and to bolster defense against regulatory action. 
 
The paper concludes that adult interest in flavors as modest, with the exception of classic tobacco flavor, which they observed can “ease the transition for smokers from a familiar product to a less-familiar one.”  With teens uninterested and adults minimally interested in flavors (other than tobacco and menthol), one wonders why do they bother to market them.  If so unappealing to their customers, what accounts for the dramatic rise in NJOY sales after introduction of flavors such as butter crunch, peach tea, and wild berry?
 
Robert K. Jackler MD
Sewall Professor & Chair
Otolaryngology-Head & Neck Surgery
Principal Investigator
Stanford Research into the Impact of Tobacco Advertising (SRITA)
Stanford University School of Medicine
jackler@stanford.edu
 
 
Interest Disclosure:
Dr. Jackler served as a consultant to the State of California Attorney General’s office on the 2014 motion to enforce the 2010 Soterra (NJOY) consent judgment.
 
References:
 
1. Shiffman S, Sembower M, Pillitteri J, Gerlach K, Gitchell J. Impact of electronic cigarette flavor descriptors on non-smoking teens’ and adult smokers’ interest in electronic cigarettes.  Nicotine Tob Res. 2015 Jan 7. pii: ntu333.
 
2. Groves RM, Peytcheva E. The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis The Public Opinion Quarterly Vol. 72, No. 2 (Summer, 2008), pp. 167-189
 
3.  Heath and Human Services.  Institutional Review Board Exemptions: Special Classes of Subjects: Children and Minors: (http://www.hhs.gov/ohrp/archive/irb/irb_chapter6.htm)
 
4. Pinney Associates: About our Company. (http://www.pinneyassociates.com/overview.xml)
5. NJOY to Discontinue Flavors, Takes Additional Steps to Prevent Underage Electronic Cigarette Use. Reuters.  Dec 10, 2009
6. Giovenco DP, Hammond D. Corey CG, et. al. E-Cigarette Market Trends in Traditional U.S. Retail Channels, 2012–2013.  Nicotine & Tobacco Research, 2015, 1–5
7. Wells Fargo Securities. Equity Research: Vapor - NJOY’s New Product Line Positions it Ahead of the Pack. Wells Fargo Securities; 2014.
8. Richtel M.  E-Cigarette Makers Are in an Arms Race for Exotic Vapor Flavor. New York Times, July 15, 2014.
9. NJOY tops US retail Vaping sales.  Convenience Store news Nov 11, 2014. http://www.csnews.com/product-categories/tobacco/iri-njoy-tops-us-retail-vaping-sales
10. People of the State of California v. Sottera, Inc. d.b.a. NJOY. Stipulated consent judgment. Case No. RG10528622. July 29, 2010. (http://oag.ca.gov/system/files/attachments/press_releases/n1965_sottera_...)
11. People of the State of California v. Sottera, Inc. d.b.a. NJOY.  Stipulation to modify consent judgment of July 30, 2010. August 18, 2015. Case No. RG10528522
 

Comments

Nic & Tob Res declines to publish another critique of Shiffman

Commentary on Shiffman et. al. study which surprisingly found no preference for flavored electronic cigarette products among teens.
 
Earlier this year, Stan Glantz and Lucy Popova submitted the critique of the Shiffman et al paper on flavor preferences in eCigs to the Nicotine and Tobacco Research. the journal which published the original work.  (Their critique appears at the root of this series of comments.)  As it is customary practice for journals to encourage responsible criticism to appear in their pages, I was more than a little surprised to learn that they refused to publish his comments.  After Stan’s critique later appeared on his blog, I decided to also submit my own comment to NTR.  As I was at the time a consultant to the California Attorney General’s office on negotiations related to the paper’s sponsoring company (NJOY) flavored eCig practices (in which the Shiffman et al paper was prominently featured) I needed to delay my submission until this matter had resolved.   
 
My commentary (see below) emphasizes both my concerns over the scientific validity of the paper and what I view as compelling evidence of commercial bias.  I find it troubling that NTR also rejected my commentary as well.  Refusing to publish both Stan and my critiques raises questions about the journal’s commitment to thoughtful dialog on issues of scientific validity and bias.   In rejecting my critique, NTR explained that conflict of interest among the authors had been disclosed.  However, merely disclosing COI does not excuse any and all sorts of commercial bias contained in the design of the research and/or interpretation of the data.  As disclosure is meant to help readers to consider the possibility of bias, it seems most appropriate for the journal to publish letters from responsible academics who perceive commercial interest and seek to share their rationale with the scientific community.
 
In rejecting my comment, NTR went on to say: “Your letter I think implies that the article has significant flaws that mean it should not have been published in the first place, but it has been published and the journal's view is that the final accepted version dealt adequately with the original reviewer's comments.”  By definition, all published papers have passed the journal’s expert peer review process and all subsequent letters to the Editor follow this approval.  Having long served as an Editor-in-Chief of a major scientific journal, I am keenly aware of the limitations of the peer review process and the importance of responsible post publication commentary to enhance the value of scientific literature.
 
Robert K. Jackler MD
Sewall Professor & Chair
Otolaryngology-Head & Neck Surgery
Principal Investigator
Stanford Research into the Impact of Tobacco Advertising (SRITA)
Stanford University School of Medicine
jackler@stanford.edu
 
My Commentary on Shiffman et. al. study:
 
The paper ‘Impact of flavor descriptors on non-smoking teens and adult smokers’ interest in electronic cigarettes” by Shiffman and coworkers suffers from flaws in experimental design and data interpretation which call into question their conclusions that flavored e-cigarettes are not of differential appeal to teens.1 As discussed in detail below, their survey methodology has a high likelihood of selection bias, the study groups are too dissimilar to enable the comparisons made, informed consent for minors was not obtained despite evident risks, and commercial interest appears to have influenced the study’s outcome.
 
In comparing the relative appeal of flavored e-cigarettes to non-smoking teens versus smoking adults, the obvious hypothesis would be that flavors would be of greater attraction to the young.  The factual basis for this conjecture is the extensive body of research which found that flavored tobacco products have differential appeal to youth. This led the US Congress to ban flavored cigarettes in 2009.  In a broader context, a finding that adolescents have no preference for sweet and fruity flavors would mean that e-cigarettes somehow are a special exception to well established consumption trends throughout the food and beverage industry. 
 
Concerns over survey validity:
 
The logical way to determine the differences in e-cigarette flavor preferences between adults and teens would be to compare the actual frequency of flavor use by teen and adult e-cigarette users.  Rather than use a direct method, Shiffman et. al. compared a small cohort of teen non-smokers with adult smokers via a marketing survey conducted for the sponsoring e-cigarette brand (NJOY).  There is reason to question the validity of comparing non-smoking teens with adult smokers as they are notably unequal groups. Nationally only approximately 15% of teens smoke whereas the entire adult survey group smoked.   In this survey teen smokers were systematically excluded. 
 
The Shiffman et. al. study consisted of an online survey inquiring about interest in products of the NJOY e-cigarette brand.  These included then current flavors (tobacco and menthol) and a list of contemplated flavors.  A survey of flavor preferences for bottled water and ice cream was used as a comparison.  Invitations were sent to 20,235 adults and 14,151 teens with a mere 432 adults and 216 teens enrolled. This is an under 2% response rate. With such a low response rate, responders may have had a special motivation to participate.  This raises the likelihood of selection bias, differences between responders and non-responders, a factor which undermines the value of surveys with a small percentage of responders.2
 
A sizable fraction of teens are opposed to smoking, some with notable vehemence instilled by parents and schools.  It would be expected that a fraction of teen responders felt a special impetus to do so as a means of communicating their opposition to tobacco products.  Two sorts of teens are likely to reply to such a survey: those contemplating e-cigarette use who may be attracted to the products being surveyed and those who are vehement in their intention to neither smoke nor vape and are motivated to send a rejectionist message. The motivation of teen responders to oppose smoking is the most probable reason for the low interest in e-cigarettes which the authors erroneously interpreted as a lack of interest in flavored e-cigarettes.   That the paper reaches the conclusion: “Nonsmoking teens’ interest in e-cigarettes was very low” is yet another reason to question the survey’s validity.   This observation is at variance with 2014 CDC data showing a rapid rise in e-cigarette use among high school students coupled with a reduction in combustible cigarette use.
 
Not surprisingly, their survey showed non-smoking teens queried on common consumer products (ice cream and flavored water) indicated widespread interest in flavors.   In the adult cohort, flavor preferences for e-cigarettes, other than tobacco and menthol, roughly paralleled that of ice cream and water. (Shiffman et al. Figure 3b)  That teens expressed a liking for flavored ice cream, but not e-cigarettes, may well be accounted for by responses of teens who had no interest in e-cigarette products.  (Shiffman et. al. Figure 3a)  It was not that these non-smoking teens did not like the e-cigarette flavors, but rather they categorically were uninterested in NJOY’s products in all flavor variations.
 
In their flavor analysis, the paper’s data reveals a marked preference among adult smokers for tobacco flavor and menthol flavors with all other flavors (except vanilla) appearing at a fraction of their appeal.  (Shiffman et al. Figure 1)  Among teen non-smokers, by contrast, interest was essentially equal across flavors with no preference for tobacco or menthol flavors.  One interpretation of these data is that, relatively speaking, non-smoking teens prefer a variety of sweet and fruity flavors while adult smokers prefer tobacco flavor.  This interpretation of their data is the opposite the authors’ conclusions.
 
Inappropriate Claim of Exemption from IRB for Survey of Minors
 
The Shiffman et. al. survey offers no benefit to the minors surveyed and entails significant risks. Exposing teens to a harmful product they may not have considered using, such as use of e-cigarettes and flavored e-cigarettes, could increase the likelihood that they will try the product and develop nicotine addiction.  This represents a clear risk to the future health of the minor. The existence of risk, coupled with the lack of benefit to the individual child, necessitates both institutional review and parental informed consent.3
 
Evidence of Commercial Bias
The authors’ declaration in the Shiffman et. al. paper discloses that the study was sponsored by the NJOY e-cigarette brand.   It also reveals that the company was involved in determining the study design.  The survey was conducted by a ResearchNow a company which conducts surveys for commercial marketing purposes on subjects such as customer satisfaction and loyalty. All of the study authors were all employees of the Pinney Associates a firm who represents both the tobacco industry (Reynolds American) and e-cigarettes (NJOY - Soterra Inc).  Pinney Associates describe their service to companies: “Evaluate, support and develop new commercial opportunities including new products and claims; Manage issues and the scientific and policy environments to gain regulatory approval for new products; and “Enhance the scientific and policy environments to maximize product acceptance.”4
 
Given the long history of tobacco industry manipulation of scientific research, the fact that the Shiffman et. al. study was designed and sponsored by an e-cigarette brand gives reason for skepticism.   That the authors we all employed by a company contracted to NJOY whose goal is to “Enhance the scientific and policy environments to maximize product acceptance” further raises concern over the independence and objectivity of the paper.  The circumstances under which the paper arose, as illustrated by the timeline below, indicates that NJOY had a powerful motivation to commission a paper exculpating e-cigarette flavors from preferential youth appeal proximate to the time that Pinney Associates was engaged and the survey was conducted.
 
In December 2009, NJOY CEO declared that they would halt flavor sales to discourage underage use and to align with the Family Smoking Prevention and Tobacco Control Act recently passed by Congress.5 Between 2012 and 2013 NJOY lost its lead position in market share to Blu (2013 Blu 44.1%, NJOY 24.3%).6  By the second quarter of 2014, Logic overtook NJOY as the number two brand in all tracked channels.7   Lorillard’s purchase of Blu and the 2014 emergence of RJ Reynolds (Vuse) and Altria (Mark Ten) into the e-cigarette market further increased NJOY competition.   The New York Times quoted NJOY CEO Craig Weiss as having said in 2013 that other manufacturers use flavors to attract children.8 This New York Times article of July 2014 described NJOY as having chosen to resume flavor sales, after a 5 year hiatus, due to a 50% decline in convenience store sales over the preceding year.  In the month following re-introduction of flavors in July 2014, NJOY realized a 65% increase in convenience store sales.9
 
NJOY leadership was undoubtedly concerned about possible repercussions from their reintroduction of flavors.  After all, over a four year period two of their CEOs had publicly linked e-cigarette flavors with youth.  Complicating matters further was the 2010 they a consent judgment they agreed to in the nation’s largest state (California) stipulating that they would not market flavored e-cigarettes.10   In an effort to manage anticipated regulatory and public relations difficulties, NJOY engaged Pinney Associates in January 2014.   As part of this campaign, it appears that Pinney Associates was commissioned to create a scientific publication with the expectation that it would conclude that flavored e-cigarettes possess no special attractiveness to youth.  The work product of this engagement was the scientifically unconvincing Shiffman et. al. study in which the authors seem to have gone to great length to conjure a methodology which would appear to support their sponsors’ desired outcome.
 
Shortly after the July 2014 re-introduction of flavored NJOY products, the California Attorney General contacted the company to request compliance with the 2010 judgment. Absent voluntary compliance, in September 2014 the California Attorney General filed a motion to enforce NJOY’s 2010 e-cigarette flavor consent judgment.11  The Shiffman et. al. study became the centerpiece of the legal argument NJOY made in asserting that flavored e-cigarettes did not appeal to teens and thus should be allowed on the California market.  The fact that the paper was peer reviewed and accepted by the prestigious journal Nicotine and Tobacco Research lent it credibility.  In September 2015, the Attorney General’s office agreed to allow flavored NJOY products to remain on the market in California in exchange for certain limitations on youth marketing.12
 
In summary, it appears that the Shiffman et. al. paper was not hypothesis driven research, but rather a study commissioned by NJOY to create cover for their reversal on flavors driven by dropping market share.  Both the circumstances under which it came about, and the scientific content of the work, suggest that the result was not an open ended question but rather attained a result preordained by the sponsor.   Fundamentally it was “science” commissioned to support a product launch and to bolster defense against regulatory action. 
 
The paper concludes that adult interest in flavors as modest, with the exception of classic tobacco flavor, which they observed can “ease the transition for smokers from a familiar product to a less-familiar one.”  With teens uninterested and adults minimally interested in flavors (other than tobacco and menthol), one wonders why do they bother to market them.  If so unappealing to their customers, what accounts for the dramatic rise in NJOY sales after introduction of flavors such as butter crunch, peach tea, and wild berry?
 
Robert K. Jackler MD
Sewall Professor & Chair
Otolaryngology-Head & Neck Surgery
Principal Investigator
Stanford Research into the Impact of Tobacco Advertising (SRITA)
Stanford University School of Medicine
jackler@stanford.edu
 
Interest Disclosure:
Dr. Jackler served as a consultant to the State of California Attorney General’s office on the 2014 motion to enforce the 2010 Soterra (NJOY) consent judgment.
 
References:
 
1. Shiffman S, Sembower M, Pillitteri J, Gerlach K, Gitchell J. Impact of electronic cigarette flavor descriptors on non-smoking teens’ and adult smokers’ interest in electronic cigarettes.  Nicotine Tob Res. 2015 Jan 7. pii: ntu333.
 
2. Groves RM, Peytcheva E. The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis The Public Opinion Quarterly Vol. 72, No. 2 (Summer, 2008), pp. 167-189
 
3.  Heath and Human Services.  Institutional Review Board Exemptions: Special Classes of Subjects: Children and Minors: (http://www.hhs.gov/ohrp/archive/irb/irb_chapter6.htm)
 
4. Pinney Associates: About our Company. (http://www.pinneyassociates.com/overview.xml)
5. NJOY to Discontinue Flavors, Takes Additional Steps to Prevent Underage Electronic Cigarette Use. Reuters.  Dec 10, 2009
6. Giovenco DP, Hammond D. Corey CG, et. al. E-Cigarette Market Trends in Traditional U.S. Retail Channels, 2012–2013.  Nicotine & Tobacco Research, 2015, 1–5
7. Wells Fargo Securities. Equity Research: Vapor - NJOY’s New Product Line Positions it Ahead of the Pack. Wells Fargo Securities; 2014.
8. Richtel M.  E-Cigarette Makers Are in an Arms Race for Exotic Vapor Flavor. New York Times, July 15, 2014.
9. NJOY tops US retail Vaping sales.  Convenience Store news Nov 11, 2014. http://www.csnews.com/product-categories/tobacco/iri-njoy-tops-us-retail-vaping-sales
10. People of the State of California v. Sottera, Inc. d.b.a. NJOY. Stipulated consent judgment. Case No. RG10528622. July 29, 2010. (http://oag.ca.gov/system/files/attachments/press_releases/n1965_sottera_...)
11. People of the State of California v. Sottera, Inc. d.b.a. NJOY.  Motion to enforce consent judgment. September 2014.  Case No. RG10528522
12. People of the State of California v. Sottera, Inc. d.b.a. NJOY.  Stipulation to modify consent judgment of July 30, 2010. August 17, 2015. Case No. RG10528522
 

Shiffman et al respond

Impact of Flavor Descriptors on Youth Interest in Electronic Cigarettes
 
Saul Shiffman, PhD, Mark. A. Sembower, MS, Janine L. Pillitteri, PhD, Karen K. Gerlach, PhD, MPH, and Joe Gitchell, PinneyAssociates, Inc.
 
We were interested to read Glantz and Popova's comments on our study, and glad for the opportunity to respond.  The critique was familiar – it was, in fact, the critique offered by one of the four anonymous expert referees who reviewed our paper for the journal Nicotine and Tobacco Research. In the end, the journal, in deciding to publish the paper, decided that we had overcome these concerns and that the paper was a valuable addition to the literature.
 
Glantz and Popova raise a methodological issue regarding our study, focusing on the use of a single item to assess product interest.  While multi-item scales can often be useful, or superior, it is not always so (e.g., West and Ussher, 2010).  Indeed, as we note in the paper itself, single items have been used to assess interest in smoking and in e-cigarettes, and have shown themselves to be sensitive to variations due to many factors (e.g., Pepper et al, 2014).  We also note that it was not our objective to assess a range of factors like social influences on e-cigarette adoption, but specifically and simply to focus on the effect of flavors, per se, and that it was important to keep the assessment brief, given that we were asking participants to respond to 24 stimuli.
 
Glantz and Popova also assert (without offering any basis) that the scale was subject to floor or ceiling effects.  Since the teen nonsmokers in our sample already rated their interest in e-cigarettes at an average of 0.41 on a 0-10 scale, this seems to imply that the teens may have actually wanted to give e-cigarettes even lower – negative! – ratings.  This certainly does not match Glantz and Popova's expressed concern about uptake of e-cigarettes among teens.
 
Glantz and Popova also failed to note that the paper reports data on participants' interest in other consumer products, using the same single item, and those data clearly demonstrate sensitivity to variations by population, product, and flavor.
 
Finally, Glantz and Popova attack our ethics by claiming (a) that consent had not been obtained, (b) that we had failed to provide some kind of required corrective education to overcome some expected adverse effect of taking our survey, and (c) that the study must be biased because it was sponsored by NJOY, a maker of e-cigarettes.  We take each of these up in turn.
 
As the paper notes, there were multiple levels of consent.  For the teen participants, parents had to consent for the teen to be enrolled in the panel and be recruited to surveys.  The teen also had to consent to participate in the panel.  The teen then had to consent to do the particular survey, and the teens (and adults) essentially consented to complete each item, as they were free to skip any item or bail on the entire survey.
 
We do not believe there was any need for corrective education after study participation.  The study survey did not provide positive (or negative) information (true or deceptive) about smoking or e-cigarettes, or promote them in any way, as illustrated by the fact that interest in e-cigarettes was near zero among the teens.  Participants were simply asked to rate their interest, as survey participants are asked to do in thousands of surveys every day.  We note that surveys, including those fielded and sponsored by the government, routinely ask participants about smoking and their interest in smoking without any implication they have misled or damaged participants, and without any requirement for corrective education.  (Dr. Popova has been criticized for conducting research in which participants were deceived about the basis for participation and deceived about product risks without any debriefing on either count [Reader Comments on Popova and Ling, 2014].)
 
The study reported was indeed sponsored by NJOY, a maker of e-cigarettes.  We openly disclosed NJOY's (limited) role, as expected by research standards and the journal's policies.  Calling out the sponsorship as a basis for claiming bias is weak.  If Glantz and Popova believe the study was biased due to industry sponsorship, they should provide specific evidence of such bias.  Better yet, they or others should conduct a study of the same issue using what they believe are unbiased methods.  Data are the best corrective.  On the issue of sponsorship and bias, analysts increasingly point out that many other author 'interests' need to be considered (Smith et al., 2009), and declared, and, indeed, others have wondered whether Glantz and Popova's interests might keep them from evaluating evidence even-handedly (Ledwith, 2015).
 
In short, we believe the paper stands on its own, and informs our understanding of the appeal of e-cigarettes and flavors to nonsmoking teens and smoking adults.  To inform a debate that has been rich on loudly-expressed opinion and starved for data, we welcome further research on the topic.
 
Disclosure
Since conducting and publishing this research, PinneyAssociates has begun providing consulting services on tobacco harm minimization (including nicotine replacement therapy and digital vapor products) to Niconovum USA, RJ Reynolds Vapor Company, and RAI Services Company, all subsidiaries of Reynolds American Inc. In the past three years, PinneyAssociates has consulted to GlaxoSmithKline Consumer Healthcare on smoking cessation and NJOY on electronic cigarettes. SS and JGG also own an interest in intellectual property for a novel nicotine medication, an option for which has been sold to Niconovum USA.
 
 
NB: Readers who are interested in an open discussion on these issues can view and comment at http://nicotinepolicy.net/jg/3697-impact-2.

References
Ledwith S. INSIGHT-The smoke around e-cig science. Published online January 22, 2015. Accessed on February 24, 2015 and available at: http://reut.rs/1GBXo39
 
Pepper JK, Emery SL, Ribisl KM, Southwell BG, Brewer NT. Effects of advertisement on smokers’ interest in trying e-cigarettes: the roles of product comparison and visual cues. Tob Control 2014 Jul;23 Suppl 3:ii31-36.
 
Smith R, Feachem R, Feachem NS, Koehlmoos TP, and Kinlaw H. The fallacy of impartiality: competing interest bias in academic publications. J R Soc Med 2009;102:44-45.
 
West R, Ussher M. Is the ten-item Questionnaire of Smoking Urges (QSU-brief) more sensitive to abstinence than shorter craving measures? Psychopharm 2010;208:427-432.
 

NJOY's marketing has a different view

Flavors don't matter? NJOY marketing has a different view:

"Choose from five delicious flavors for a whole new world of e-cigarette pleasure with NJOY Flavor Chambers"
"Judge for yourself why our Flavor Chambers are so delicious

https://www.njoy.com/recharge/flavor-chambers

"Looking for a refreshing flavor experience when vaping? We've grouped together
five amazing, refreshing NJOY e-liquid flavors"

https://www.njoy.com/vaping/e-liquids

"focus groups showed that flavors were 'critical'"

http://www.nytimes.com/2014/07/16/business/e-cigarette-makers-are-in-an-...

NJOY's funded research that claims flavors have nothing to do with it
may have something to do with "candy- and fruit-like flavorings
...'drive regulators crazy'" (same NYTimes article)

At any rate, NJOY's marketing department doesn't seem to have gotten the
memo that flavors don't matter.

Jon Krueger

Remember this? NJOY CEO distinguishes between flavours . . .

http://time.com/2896962/electronic-cigarette-executives-get-schooled-in-senate-hearing/
"Responding to questions from Senator Boxer (D-California) about the flavors NJOY plans to introduce, Weiss made a slip, saying “for adults, we have single malt scotch.”
<?xml:namespace prefix = o /> “Adult flavors?” said Boxer. “As opposed to those for children.” Weiss continued nervously, attempting to remember the new flavors off the top of his head: “In addition, there’s vanilla bean, there’s also peach tea, there’s also, um…”

At the end of her time to question, Boxer said: “Mr. Healy and Mr. Weiss, you can con yourself. But we don’t know if this product gets people off cigarettes yet, so don’t think you are doing some great mission. Don’t say you care about kids,” said Senator Boxer. “Don’t be a part of this, because you’ll regret it.”
But the harshest words came from Senator Jay Rockefeller (D- West Virginia), who said to the executives: “I’m ashamed of you. I don’t know how you go to sleep at night. I don’t know what gets you to work in the morning except the color green of dollars. You are what is wrong with this country.”
Other exerts included:
 “The only difference between your testimony today and testimony of the tobacco executives is that you are not under oath,” said Senator Blumenthal. “I find in your testimony a sense of denial that I cannot credibly accept because it is defied by the numbers. 18 million teens were exposed to blu’s print TV ads in 6 months and NJOY’s ads reached 3 million teens [these statistics come from the American Legacy Foundation report]. There is a legal principle that people are responsible for the natural and logical effects of what you do, and you know that you are reaching children.”
“I think we have seen this movie before,” he continued. “It is called big nicotine comes to children near you and you are using the same kinds of tactics and promotions and ads that were used by big tobacco and proved so effective.”
Other data contrdicts the Shiffman et al paper findings, e.g. :
 “E-cigarettes that were rechargeable and had sweet flavors were most popular.” http://www.ncbi.nlm.nih.gov/pubmed/25385873
Dave Bareham