- FAMRI Center
UCSF TCORS public comment on FDA proposed NNN standard
The Regulations.gov tracking number is 1k1-8vee-7yms. A PDF version is here.
The FDA’s Proposed Tobacco Product Standard Limiting NNN Levels in Finished Smokeless Tobacco Products is Well-Justified, but the Regulatory Impact Analysis Understates Benefits and Overstates Costs
Docket Number: FDA-2016-N-2527
Wendy Max, Lauren Lempert, Benjamin Chaffee, Bonnie Halpern-Felsher, Eunice Neeley, Lucy Popova,[*] Peyton Jacob, Stanton Glantz
March 22 2017
In its proposed rule setting a tobacco product standard requiring that the mean level of N-nitrosonornicotine (NNN) in finished smokeless products sold in the U.S. must not exceed 1.0 μg/g of tobacco, the FDA took a major positive step towards protecting public health and reducing cancer risks associated with smokeless tobacco use. Congress gave FDA the express authority to regulate the design and contents of tobacco products by creating tobacco product standards, and UCSF TCORS applauds FDA for taking this action.
FDA found that “NNN is a potent carcinogenic found in smokeless tobacco products and is a major contributor to the elevated cancer risks associated with smokeless tobacco use.” Because products with higher NNN levels pose higher risks of cancer, FDA believes establishing a NNN limit in smokeless products would be appropriate for the protection of the public health. We agree.
- The scientific evidence is clear. Smokeless tobacco users face increased risks of oral and other cancers, an association that is most pronounced in countries like the United States where smokeless tobacco products contain higher levels of NNN.1
FDA’s proposed product standard addresses a public health emergency. Every day, on average, 1,315 adolescents aged 12-17 use smokeless tobacco for the first time. Reducing NNN levels in smokeless tobacco would not only save thousands of lives, but would also prevent tens of thousands of youth and adults from suffering the effects of disfiguring and life-altering cancers.
All tobacco products contain carcinogenic tobacco-specific nitrosamines (TSNAs), and among TSNAs, NNN and nicotine-derived nitrosamine ketone (NNK) are the most carcinogenic. 2-5 FDA’s methodology and rationale for developing its NNN standard is supported by strong scientific evidence. In particular, FDA provided extensive scientific evidence on pages 28-54 of its proposed rule supporting its findings that NNN is a potent carcinogenic agent found in smokeless tobacco products, that NNN in smokeless tobacco products is a major factor underlying oral and pharyngeal cancers, and that reducing levels of NNN in tobacco products in the U.S. will reduce the incidence of oral and pharyngeal cancers among smokeless tobacco users.
Smokeless tobacco consumption in the United States increased by 23% from 2000-2015, reaching 6.7% among adult males.6 Among male high school students nationally, the prevalence of smokeless tobacco use (11.9%) equals that of cigarette smoking (11.8%),7 and among male high school athletes, smokeless tobacco use is even higher (17.4%).8
As detailed below, however, FDA did not consider all the available epidemiologic evidence demonstrating that use of smokeless tobacco products currently marketed in the United States causes cancers, so it is substantially underestimating the benefits of the proposed rule.
- The FDA's estimates of the public health benefit of the proposed product standard underestimates the actual health benefits and leave room for improvement.
There are several instances in which the FDA has made assumptions that likely underestimate the potential public health benefit of this proposed rule.
First, the FDA states that, "relative risk estimates used to model the population attributable risk come from a published systematic review and meta-analysis of studies of oral cancer among U.S. smokeless tobacco users."1
However, rather than use the relative risk estimate (RR) for oral cancer as Boffetta et al.1 reported in this peer-reviewed review (RR: 2.60), the FDA elected to recalculate the RR after excluding one study.9 The redlined version of the proposed rule (page 82) shows that this exclusion was not part of the FDA's initial estimates, but was made by the White House Office of Management and Budget. The exclusion was based on a critique that was published as a commentary10 by authors Lee and Hamling with funding from the European Smokeless Tobacco Council, a lobbying group for smokeless tobacco manufacturers and sellers. The commentary was published over peer-review objections that noted a "biased selection" of citations and "no scientific justification to publish."11
The FDA should not capitulate to an industry-funded attack on the scientific literature. Using a posteriori rationale to exclude the one study that showed the largest association between smokeless tobacco and oral cancer while overlooking the methodological limitations of studies that reported results more favorable to the industry is not scientifically justified. The FDA should base their estimates on the work of investigators free from conflicts of interest.
Second, the FDA states its expectation that "the proposed standard would reduce the number of new and fatal cases of esophageal cancer among continuing smokeless tobacco users and may reduce the risk of pancreatic cancer as well" and later notes that the standard may reduce the risk of "other cancers such as pancreatic, laryngeal, prostate, and lung cancer." The International Agency for Research on Cancer (IARC) confirmed a causal role for smokeless tobacco in oral, esophageal, and pancreatic cancer.2, 3
Rightly, the FDA notes that there is less epidemiologic evidence to quantify precisely the health benefits of the proposed rule that will result from preventing cancers outside the oral cavity. However, there is strong evidence that those benefits are greater than zero.
Although based on fewer studies, Boffetta et al.8 do provide relative risk estimates for the association in the United States between smokeless tobacco use esophageal cancer (RR: 1.2), pancreatic cancer (RR: 1.4), and lung cancer (RR: 1.8). As quoted by the FDA, the National Cancer Institute and the Centers for Disease Control and Prevention estimate that smokeless tobacco use nationally contributes to 200 cases of esophageal cancer and 500 cases of pancreatic cancer each year.12 Even if one conservatively assumes that the reduction in the excess lifetime cancer risk (ELCR) for these cancers under the NNN standard will be half that for oral cancer (taking into account potential uncertainly in how NNK levels will change under the NNN standard), the proposed standard will still be expected to prevent thousands of cases of esophageal and pancreatic cancer over 20 years.
Therefore, the FDA's quantitative estimates of cancer risks should be taken as a lower limit of the potential benefit of this proposed standard. It is almost certain that the actual health benefits are higher. The FDA should also present the maximum likelihood estimate and the upper 95% confidence interval for the benefits and integrate them into the Regulatory Impact Analysis. Benefits need to be more comprehensively estimated to protect public health, the legal standard established in the Family Smoking Prevention and Tobacco Control Act section 907.
FDA specifically points out that FDA can refer the proposed product standard to TPSAC in response to a request for good cause. Given the high quality of the data presented in the proposed rule, there is no reason for the further delay that would be created by referring this issue to TPSAC.
- The proposed standard is technically achievable.
FDA correctly stated that the proposed NNN level has been achieved in some smokeless tobacco products sold in the U.S., and is thus achievable using current technology.
Several American smokeless tobacco products were reported to have lower than 1 µg/g NNN.13 On pages 37-41 of its proposed rule, FDA provided an excellent description of various ways companies could lower NNN levels in their smokeless tobacco products, including by altering the tobacco type, growing conditions, curing techniques, product process, and storage conditions. Especially promising are “low converter” tobacco plants with lower levels of the precursor nornicotine, lowering the amounts of the other precursor, nitrites and nitrates which often come from fertilizers, and changing curing processes to slow the reaction that produces NNN from nornicotine and nitrite.
In fact, it would be technically achievable to reduce the NNN level even more to less than 0.5 µg/g or even less than 0.2 µg/g. This is demonstrated by Stepanov et al.’s research13 in which most products tested were below 0.5 µg/g and some less than 0.1 µg/g.
While the proposed level is technically achievable, FDA is not required to demonstrate technical achievability when setting a new tobacco product standard. In particular, section 907 of the Family Smoking Prevention and Tobacco Control Act (FSPTCA) requires FDA to apply a public health standard, and only the risks and benefits to the population as a whole, considering users and non-users, should be considered in determining whether a proposed products standard is appropriate. Indeed, the first sentence of the section called “Information on Technical Achievability” originally stated, “the Tobacco Control Act does not require FDA to consider technical achievability to determine whether a product standard is appropriate for the protection of the public health.” (See page 63 of the version FDA originally submitted to OMB with tracked changes.14) However, the White House Office of Management and Budget removed this legally correct statement from the final version of the proposed rule as cleared by OMB (see pages 54-55.15)
- In its Regulatory Impact Analysis (RIA), the FDA conducts a detailed analysis of the benefits and costs of the proposed rule, and clearly demonstrated that the benefits of the proposed standard far exceed its costs, even though many of the benefits were ignored.
Comparing the primary estimates of benefits and costs using the primary estimates monetized over 20 years with a discount rate of 3%, the benefits exceed the costs by a factor of 35. Thus, it is clear that even with fairly substantial changes to the calculations presented, the proposed rule has a positive net benefit.
However, the benefits are underestimated and the costs are overestimated, and there are several instances in which the analyses could be improved. As discussed below, the RIA should be refined; doing so will reveal that the net benefits are even larger than suggested. Another pro-industry bias in the RIA is that it provides a modest estimate of the benefits of the product standard, omitting important benefits, while, at the same time, including a detailed accounting of the cost to the industry of implementing the proposed product standard.
Benefits are systematically underestimated.
Benefit estimates include the value of lives saved only (measured as the value of a statistical life using quality-adjusted life-years gained from reduced oral cancer mortality). The analysis should also account for the healthcare costs saved among people who develop tobacco-induced diseases before they die.
Healthcare cost savings are likely to be substantial but are not estimated (RIA, p. 38-39). Our preliminary estimates of costs attributable to smokeless tobacco (SLT) for hospitalizations, doctor visits, and emergency department use alone are over $3.5 billion for 2015.16 We found that SLT use resulted in excess annual healthcare utilization of 679,000 hospital nights, 4.3 million physician visits, and 807,000 emergency department visits in just that one year. This is a lower bound on the healthcare cost savings that could be experienced if SLT use were eliminated. Our SLT-attributable annual healthcare cost estimation of $3.5 billion is substantially larger than the FDA total annualized benefit estimation of $1.1 billion (primary value, 3% discount rate), highlighting the need for a more comprehensive measure of the benefit of this product standard.
FDA analysis considers only the impact of the proposed rule on oral cancer. Using the narrow FDA focus on reduced health risks among current SLT users, the FDA benefit estimates include only a reduction in oral cancer cases. The FDA acknowledges that changing NNN content in SLT is also likely to have an impact on other diseases, including esophageal and pancreatic cancer, but do not include these benefits in the RIA, which systematically understates the benefits of the proposed rule. The National Cancer Institute estimates that annual costs of initial (year after diagnosis), continuing (years until year of death), and year of death for those who die of cancer for these cancers for adults over age 65 are:
Annualized Mean Net Costs of Care for Adults Age 65+ (2010 dollars)
Last year of death
These estimates should be used as the basis for estimates of additional healthcare savings due to reduced incidence of these two cancers.
The FDA analysis does not consider that highlighting the health risks of NNN exposure might reduce SLT use. Benefit estimates are only based on the assumption that the new regulation would result in decreased health risks among current users. It is assumed that prevalence of SLT use would remain constant (RIA, p. 30). Thus, the analysis is built on the assumption that the only benefit will be reduced oral cancer cases among continuing SLT users.
However, it is possible that this product standard, which acknowledges the risk of disease and death from using SLT, might in fact lead to lower SLT prevalence. If this were to occur, the reduction in deaths and healthcare costs would be much larger. Thus, the total healthcare cost of SLT use is relevant for this analysis. The FDA does conduct a sensitivity analysis which assumes that SLT prevalence might increase (RIA, page 111-116), but they do not consider the case where SLT use might decrease if the rule causes more people to become aware of the risks associated with the products. The fact that FDA only considered increases in use is another pro-industry bias in the RIA that should be eliminated.
In fact, if smokeless tobacco companies choose to achieve the proposed NNN standard by eliminating the use of fermentation in the manufacture of smokeless tobacco products, this may reduce the addictiveness of smokeless tobacco by lowering levels of free nicotine. FDA has understood since at least 1995 that nicotine manipulation is accomplished by altering pH levels in smokeless tobacco.17, 18 Further, pH levels rise with increased fermentation time,19 and with a greater pH, more nicotine is delivered to the smokeless tobacco user. A balanced and impartial analysis requires that the FDA consider a scenario where SLT prevalence decreases when estimating the population health impact of the proposed product standard.
Future benefits and costs should be discounted at a rate of 3%. Estimated benefits and costs are presented using discount rates of 3% and 7%. 3% is the standard rate that should be used in these analyses.20
Costs are systematically overestimated.
The FDA requests comments on how to estimate a “welfare loss” to SLT users resulting from a reformulated product (RIA, page 40-41). We find this to be inappropriate. This issue has been discussed elsewhere as it relates to lost pleasure from those who are induced to quit smoking.21-23
Chaloupka and colleagues21conclude that:
Nearly all of the ‘lost pleasure’ from tobacco use, as represented by conventionally measured consumer surplus, should not be included as a cost in FDA analyses of the economic impact of its tobacco regulations. The principle of insufficient reason suggests that the vast majority of any consumer surplus loss should be ignored given that most tobacco users become addicted regular users before reaching the legal purchase age. For those who do begin as adults, their imperfect information and self-control problems (and the associated psychological costs), increased consumer surplus from alternative consumption, and the importance of peer effects reflected in strong anti-tobacco norms suggest that regulations that reduce their tobacco use are more likely to be welfare enhancing than not. Indeed, the data strongly suggest that many smokers do not find smoking pleasurable and that they derive little consumer surplus from smoking. Instead, most are struggling with or avoiding the withdrawal they would experience if they were able to stop smoking and break an addiction they regret having ever started, facing psychological costs from being addicted and lacking the self-control to quit.
Song and colleagues22 point out that while the concept of consumer surplus is grounded in rational choice theory, which is contradicted by a wide range of neuroscience and behavioral research:
Rational choice does not account for the roles of emotions, misperceptions, optimistic bias, regret, and cognitive inefficiency that are germane to smoking, particularly because most smokers begin smoking in their youth. Continued application of a consumer surplus discount will undermine sensible policies to reduce tobacco use and other policies to promote public health.
These comments refer to cigarette smoking. As reviewed in these two papers, it has been shown that a large proportion of smokers wish they did not smoke and lose no pleasure from quitting. While this has not been documented for SLT users, it is likely to be the case as well. Absent specific evidence to the contrary, to protect public health, the use of consumer surplus and lost pleasure should not be applied to the use of other tobacco products, including SLT; a “welfare loss” should not be included as a cost of the proposed rule.
Unlike the benefit estimation, which omits large categories of benefits, the cost estimation includes every possible minute of time that might be spent complying with the proposed rule. For example, costs of changing labeling on the packages are estimated as being as high as $14,860 per UPC (RIA, page 68). However, tobacco product producers regularly change their labels as an ongoing cost of marketing. Given the implementation periods written into the rule, the marginal cost of changing labels would be zero, and not attributed to the proposed rule.
Another cost included is the estimated time to prepare a substantial equivalence report under the proposed rule (RIA, page 72). Doing business in a regulated environment involves costs for compliance, and should not be counted as a cost in changing the regulatory requirements. Furthermore, the time required is costed out at an hourly wage of $73 (RIA, p. 75), which seems quite generous. This wage is then doubled to account for benefits and overhead (RIA, p. 78) is a generous rate. The hourly wage used for nonconforming products activities -- $142 – is even higher (RIA, p. 80) and even more excessive.
- Existing and potential smokeless tobacco users may inappropriately perceive smokeless tobacco products as safe, especially as compared with other tobacco products.
If consumers perceive that smokeless tobacco has lower health risks than other kinds of tobacco, they may (1) use more; and/or (2) initiate more; and/or (3) quit less. Roditis et al.24, 25 show that adolescents perceive the health risks associated with smokeless tobacco to be lower than perceived risks attributed to cigarettes. Further, adolescents who have used tobacco perceive even lower risks associated with smokeless tobacco. Additionally, adolescents perceive smokeless tobacco to be less addictive than cigarettes.
In its proposed rule, FDA states that it expects the impact on smokeless initiation among adolescents and young adults would be limited since their initiation is associated with social influences, rather than on perceptions of long-term health effects. Further, FDA says overall, the extent to which the proposed standard may influence behaviors of non-users and former users is likely to be minimal since “health-related reasons are not among the main drivers of smokeless tobacco use initiation or relapse.” However, Roditis et al. show that adolescents also believe that smokeless tobacco is less likely to confer short-term risks, and adolescents believe that smokeless tobacco is less addictive.24, 25
FDA must consider adolescents’ perceptions of reduced risk when communicating the new NNN product standard.
Moreover, the proposed standard does not prevent addition of flavors to offset any changes in the taste of the product, although the addition of flavors would likely lead to more teen use and initiation. Adolescent SLT users express preferences for menthol, mint, or sweet-tasting SLT,26 potentially related to masking tobacco taste or stronger sweet preferences at younger ages.27 Adolescent tobacco never-users who report perceiving flavored SLT to be "easier to use" than unflavored SLT are more likely to be susceptible to SLT initiation.28 To protect public health FDA should expand the proposed rule to prohibit the addition of flavors to smokeless tobacco products.
- FDA must be cautious when communicating the new NNN product standard
If FDA establishes a new product standard, but does not communicate this standard to the public, this would leave communication to the industry, which is motivated not to protect the health of the public, but to increase sales of tobacco products.
While the proposed product standard will reduce the health risks posed by smokeless tobacco, reduced NNN products will not be harm free. Smokeless tobacco use will continue to contribute to dental diseases, such as gum disease, tooth decay, and tooth loss. Recent evidence suggests that use of snuff is associated with reduced endothelial function, particularly among inactive men.29Notably, although Swedish snus has lower levels of NNN as compared with other SLT, it has been associated with type 2 diabetes30, stillbirths31, preterm births32, prostate cancer33, and increased cancer mortality.34 Messages about the new NNN product standard should communicate that all tobacco use is harmful because of other toxicants in addition to NNN, and that the use of the smokeless tobacco products with reduced NNN levels still carries substantial risks compared to quitting all tobacco use.
If FDA communicates about the new product standard, the public (both users and non-users) trust FDA more than they trust the industry (e.g., the public trusts FDA more than the industry to correctly communicate the harms of e-cigarettes).35 This may have the effect of changing the public’s perceptions of the harm of smokeless tobacco products, e.g., giving the public the impression that the FDA “approves” the smokeless products, or that they are “safe, or safer alternatives to cigarettes.” A study examining the impact of warning labels on Swedish snus showed that warnings conveying the “potential-reduced harms” of Swedish snus compared with cigarettes generate perceptions that snus is less harmful than cigarettes and produce fewer thoughts about not using snus among nonsmokers.36
- New smokeless tobacco products modified solely to comply with the new NNN product standard should be subject to rigorous FDA premarket review
FDA should not allow new smokeless tobacco products modified to comply with the proposed NNN standard to be reviewed with a streamlined and less rigorous Substantial Equivalence (SE) report. FSPTCA mandates rigorous premarket review of all new tobacco products, including smokeless tobacco products, to ensure that they do not present different questions of public health. Because consumers’ perceptions of the harms of smokeless tobacco products may be altered by the manufacturers’ communication of reduced NNN levels on packaging and labeling, these changes present different questions of public health requiring premarket tobacco application (PMTA) review under section 910(b).
Any review of smokeless tobacco products should include the product’s packaging and labeling to ensure that manufacturers do not make modified risk tobacco claims (unless they first obtained authorization through the MRTP process). Additionally, if FDA permits a more lenient SE review for smokeless products modified to comply with the proposed standard, FDA must ensure that companies do not try to use the SE pathway to bypass more rigorous review and quickly bring to market new smokeless tobacco products that have other changes in addition to the reduced NNN level that raise questions of public health.
1. Boffetta P, Hecht S, Gray N, Gupta P, Straif K. Smokeless tobacco and cancer. Lancet Oncol. 2008;9(7):667-75.
2. International Agency for Research on Cancer. Smokeless Tobacco and Some Tobacco-specific N-Nitrosamines, IARC Monographs on the Evaluation of Carcinogenic Risks to Humans, Volume 89 (2007). Available from: http://monographs.iarc.fr/ENG/Monographs/vol89/ (accessed 3 Mar 2017)
3. International Agency for Research on Cancer. Personal Habits and Indoor Combustions: A Review of Human Carcinogens, IARC Monographs on the Evaluation of Carcinogenic Risks to Humans, Volume 100E (2012). Available from: http://monographs.iarc.fr/ENG/Monographs/vol100E/ (accessed 3 Mar 2017)
4. Balbo S, James-Yi S, Johnson CS, O'Sullivan MG, Stepanov I, Wang M, et al. (S)-N'-Nitrosonornicotine, a constituent of smokeless tobacco, is a powerful oral cavity carcinogen in rats. Carcinogenesis. 2013;34(9):2178-83.
5. Hecht SS. Biochemistry, biology, and carcinogenicity of tobacco-specific N-nitrosamines. Chem Res Toxicol. 1998;11(6):559-603.
6. Wang TW, Kenemer B, Tynan MA, Singh T, King B. Consumption of Combustible and Smokeless Tobacco - United States, 2000-2015. MMWR Morb Mortal Wkly Rep. 2016;65(48):1357-63.
7. Kann L, McManus T, Harris WA, Shanklin SL, Flint KH, Hawkins J, et al. Youth Risk Behavior Surveillance - United States, 2015. MMWR Surveill Summ. 2016;65(6):1-174.
8. Agaku IT, Singh T, Jones SE, et al. Combustible and Smokeless Tobacco Use Among High School Athletes - United States, 2001-2013. MMWR Morb Mortal Wkly Rep 2015 Sep 4;64(34):935-9. 2015.
9. Stockwell HG, Lyman GH. Impact of smoking and smokeless tobacco on the risk of cancer of the head and neck. Head Neck Surg. 1986;9(2):104-10.
10. Lee PN, Hamling J. The relation between smokeless tobacco and cancer in Northern Europe and North America. A commentary on differences between the conclusions reached by two recent reviews. BMC Cancer. 2009;9:256.
11. Warnakulasuriya S. Reviewer's Report, "The relation between smokeless tobacco and cancer in Northern Europe and North America. A commentary on differences between the conclusions reached by two recent reviews." 2009. Available from: https://static-content.springer.com/openpeerreview/art%3A10.1186%2F1471-2407-9-256/12885_2009_1590_ReviewerReport_V2_R2.pdf (accessed 2 Mar 2017)
12. National Cancer Institute, Centers for Disease Control and Prevention, U.S. Department of Health and Human Services. Smokeless Tobacco and Public Health: A Global Perspective, NIH Publication No. 14-7983. 2014. Available from: https://cancercontrol.cancer.gov/brp/tcrb/global-perspective/SmokelessTobaccoAndPublicHealth.pdf (accessed 3 Mar 2017)
13. Stepanov I, Biener L, Knezevich A, Nyman AL, Bliss R, Jensen J, et al. Monitoring tobacco-specific N-nitrosamines and nicotine in novel Marlboro and Camel smokeless tobacco products: findings from Round 1 of the New Product Watch. Nicotine Tob Res. 2012;14(3):274-81.
14. Food and Drug Administration. TAB B 2015-1102 Smokeless NNN Proposed Rule Track Changes re Tobacco Product Standard for N-Nitrosonornicotine Level in Finished Smokeless Tobacco Products 2017. Available from: https://www.regulations.gov/document?D=FDA-2016-N-2527-0226 (accessed March 17, 2017)
15. Food and Drug Administration. TAB C 2015-1102 Smokeless NNN Proposed Rule as Cleared by OMB re Tobacco Product Standard for N-Nitrosonornicotine Level in Finished Smokeless Tobacco Products 2017. Available from: https://www.regulations.gov/document?D=FDA-2016-N-2527-0228 (accessed March 17, 2017)
16. Wang, Sung, Yao, Lightwood, and Max. Costs Attributable to Smokeless Tobacco (in review). 2017.
17. Food and Drug Administration. Nicotine in Cigarettes and Smokeless Tobacco Products is a Drug and These Products are Nicotine Delivery Devices Under the Federal Food, Drug, and Cosmetic Act; Analysis Regarding the Food and Drug Administration's Jurisdiction over Nicotine-Containing Cigarettes and Smokeless Tobacco Products, pp. 289-290, Docket No. 95N-0253J, August 1995. Available from: https://www.industrydocumentslibrary.ucsf.edu/docs/#id=zscm0091 (accessed 19 Mar 2017)
18. Kessler DA, Barnett PS, Witt A, Zeller MR, Mande JR, Schultz WB. The legal and scientific basis for FDA's assertion of jurisdiction over cigarettes and smokeless tobacco. JAMA. 1997;277(5):405-9.
19. Tso T. Kirk-Othmer Encyclopedia of Chemical Technology. John Wiley and Sons. 1970;20:510.
20. Gold M, Siegel J, Russell L, Weinstein M. (eds) Cost-Effectiveness in Health and Medicine, Oxford University Press, New York, 1996. Available from: https://global.oup.com/academic/product/cost-effectiveness-in-health-and-medicine-9780195108248?cc=us&lang=en& (accessed 21 Mar 2017)
21. Chaloupka FJ, Warner KE, Acemoglu D, Gruber J, Laux F, Max W, et al. An evaluation of the FDA's analysis of the costs and benefits of the graphic warning label regulation. Tob Control. 2015;24(2):112-9.
22. Song AV, Brown P, Glantz SA. When health policy and empirical evidence collide: the case of cigarette package warning labels and economic consumer surplus. Am J Public Health. 2014;104(2):e42-51.
23. Song AV, Glantz SA. Assessing tobacco regulation: moving beyond economists. Tob Control. 2015;24(2):123-4.
24. Roditis M, Delucchi K, Cash D, Halpern-Felsher B. Adolescents' Perceptions of Health Risks, Social Risks, and Benefits Differ Across Tobacco Products. J Adolesc Health. 2016;58(5):558-66.
25. Halpern-Felsher B, Roditis M. Adolescents’ Perceptions of Addictive and Pharmacological Effects Varies Across Tobacco Products. Poster, 2017.
26. Lavo K. Smokeless Tobacco Use Among Adjudicated Adolescents. Marywood University Graduate School of Arts & Sciences (Dissertation). UMI Number 3139001. 2004.
27. De Graaf C, Zandstra EH. Sweetness intensity and pleasantness in children, adolescents, and adults. Physiol Behav. 1999;67(4):513-20.
28. Chaffee B, Urata J, Couch E, Gansky S. Perceived Flavored Smokeless Tobacco Ease-of-use and Youth Susceptibility. Tob Regul Sci (In Press). 2017.
29. Skaug EA, Nes B, Aspenes ST, Ellingsen O. Non-Smoking Tobacco Affects Endothelial Function in Healthy Men in One of the Largest Health Studies Ever Performed; The Nord-Trondelag Health Study in Norway; HUNT3. PLoS One. 2016;11(8):e0160205.
30. Carlsson S, Andersson T, Araghi M, Galanti R, Lager A, Lundberg M, et al. Smokeless tobacco (snus) is associated with an increased risk of type 2 diabetes: results from five pooled cohorts. J Intern Med. 2017;281(4):398-406.
31. Wikstrom AK, Cnattingius S, Stephansson O. Maternal use of Swedish snuff (snus) and risk of stillbirth. Epidemiology. 2010;21(6):772-8.
32. Wikstrom AK, Cnattingius S, Galanti MR, Kieler H, Stephansson O. Effect of Swedish snuff (snus) on preterm birth. BJOG. 2010;117(8):1005-10.
33. Wilson KM, Markt SC, Fang F, Nordenvall C, Rider JR, Ye W, et al. Snus use, smoking and survival among prostate cancer patients. Int J Cancer. 2016;139(12):2753-9.
34. Nordenvall C, Nilsson PJ, Ye W, Andersson TM, Nyren O. Tobacco use and cancer survival: a cohort study of 40,230 Swedish male construction workers with incident cancer. Int J Cancer. 2013;132(1):155-61.
35. Weaver S, Jazwa A, Popova L, Rothenberg R, Slovic P, Eriksen M. Whom do Adults Trust for Health Information on Electronic Nicotine Delivery Systems? Poster, presented at the Annual Meeting of the Society for Research on Nicotine and Tobacco, Florence, Italy, March 2017.
36. Mays D, Moran MB, Levy DT, Niaura RS. The Impact of Health Warning Labels for Swedish Snus Advertisements on Young Adults' Snus Perceptions and Behavioral Intentions. Nicotine Tob Res. 2016;18(5):1371-5.
[*] Georgia State TCORS