Given that the health of American youth is in question and that so many states base their policies on reports issued by the State of Colorado, it is important to understand what the 2015 Healthy Kids Colorado Survey (HKCS) actually tells us.
The survey’s results are gleaned from voluntarily self-reported information collected every other year from Colorado middle-school and high-school students. It is produced by a partnership of the Colorado Department of Education, Colorado Department of Human Services, the Colorado Department of Public Health and the Environment and the University of Colorado.
News organizations tracking the impact of marijuana on Colorado since voters sanctioned the drug for medical and recreational use are understandably quick to report the survey’s findings — but they’re unfortunately just as quick to deliver inaccurate and misleading information. Coverage of the 2015 survey results was especially poor. Dozens of news organizations — including The Denver Post, Fox News, the Washington Post, Time, Scientific American and Reuters — should correct and clarify their work.
Why? Because for many reasons, the 2015 survey’s data do not support claims that marijuana use among Colorado teenagers has remained flat or has declined. Examination of the survey’s aggregate data, segmented by grade and geographic region, tells a different story than the Marijuana Infographic and some passages of the executive summary distributed by state officials.
New reporting should inform the public about youth marijuana use rates in several Colorado regions — particularly where marijuana is most heavily commercialized.
Here are some important things to know about the 2015 survey:
Because of its methodology and sample size, this survey is a snapshot in time that represents no one other than the Colorado youth who took it. It is inaccurate to present or describe the 2015 survey as a “state survey” or to present its findings as average use rates among Colorado youth. The 2015 survey does not include data from El Paso County (home to the state’s second largest city, Colorado Springs), Jefferson and Douglas counties (home to two of the state’s largest school districts) and Weld County. It is also important to note that Colorado’s private and parochial schools do not participate in this survey and that only students attending school are surveyed. Students with drug problems are less likely to be in school — and, therefore, less likely to be surveyed.
Differences in methodology make it difficult to compare the 2015 survey to previous HKC surveys. The randomly selected sample size dropped from 40,206 in 2013 to 15,970 in 2015. Similarly, the high school response rate dropped from 58 percent in 2013 to 46.5 percent in 2015. Counties participating in the survey also changed from 2013 to 2015. Clearly, something in the survey methods changed from 2013 to 2015, making direct comparisons risky. But if state officials and journalists insist on making these direct comparisons, there are significant increases in youth marijuana use to report from 2013 to 2015 — as detailed below. They should report this information to the public.
Because of differences in methodology, Colorado survey results should not be directly compared to other national studies of adolescent marijuana-use rates, such as the Centers for Disease Control’s Youth Risk Behavior Survey (YRBS). These surveys are different. For example, the YRBS requires a response rate of at least 60 percent. If student responses fall below that mark, the YRBS states the results “represent only the students participating in the survey.” Of note, the HKCS did not reach this threshold for high school students in either 2013 or 2015. Therefore, direct comparisons of the two studies is risky. Such differences in methodology also make it risky to compare the Colorado data and other national studies, such as the Substance Abuse and Mental Health Service Administration’s National Survey on Drug Use and Health and the Monitoring the Future (MTF), a survey from the National Institute on Drug Abuse run by contract through the University of Michigan. Further, the 2015 state report’s comparisons to a “national average” of youth marijuana use are also problematic. Please review explanations here and here from David Murray, a former chief scientist of the White House Office of National Drug Control Policy, who now serves as a senior fellow analyzing drug policy at the Hudson Institute. Among his observations:
“What is the possible source for deriving that ‘national average’? There is one genuinely national sample of youth drug use, that from the National Survey on Drug Use and Health (NSDUH) that covers all states. But this cannot be the basis for the (State of Colorado’s) claim. In their latest 2014 estimates, NSDUH reported that 7.2 percent of adolescents aged 12 to 17 across the nation used marijuana in the past month – that figure, not 21.7 percent, would be the youth ‘national average.’ Moreover, the NSDUH specifically declared that Colorado had the nation’s highest rates. Adolescent marijuana use ranged from 4.98 percent in Alabama to 12.56 percent in Colorado. Worse, the NSDUH showed for youth that from 2009, when medical marijuana took off in Colorado, there has been a stunning rise of 27 percent through 2014 (from 9.91 percent to 12.56 percent). So Colorado youth use rates in the NSDUH are not only higher than the national average, but, after freer access to marijuana, have been steeply climbing.”
To examine drug-use trends from year to year and make comparisons between states, the NSDUH is more reliable (not perfect, but more reliable). The NSDUH interviews youth who are in and out of school. It is conducted in every state — and, unlike the current version of the Colorado Healthy Kids survey, it has data from before 2013. Unfortunately, as Murray notes above, this survey shows the prevalence of past-month marijuana use among Colorado youth has increased, with Colorado ranked first among 12-17 year olds in 2014.
One strength of the HKCS is that it offers some county-level data. It is helpful to have a fine-grain look at what is happening at a local level. So, if we must compare 2013 and 2015 survey results, it is best to limit comparisons to the responses of specific regions as defined by the survey. You can find a map of those regions here. Because there are many differences between high school freshmen and seniors, combining their class data — especially given that 18-year-olds in Colorado can purchase medical marijuana legally — can give false impressions about “teen use” rates. So, it is important to segment students by grade for a more accurate look at marijuana use rates.
Remember: Because of significant differences in methodology and sample size, the 2015 HKCS shouldn’t be compared to its 2013 predecessor or any national survey — but if state officials and journalists insist on doing so, let’s all consider this closer look at student respondents by grade and region. It suggests adolescent marijuana use rates have reached levels worth considering a serious health problem in some parts of the state.
For a full breakdown of the regional data, please see this chart (produced with the significant help of Christine Miller, a Ph.D. pharmacologist and Colorado native). Among the findings:
Region 16 (Boulder, Broomfield): High school seniors in this region reported the highest rate of past-month use among 12th graders in the state. In 2015, 42.2 percent of high school seniors reported past-month use, versus 28.5 in 2013. That’s a 48.1 percent increase. The use rate among high school juniors in this region jumped from 22.3 percent to 33.4 percent, a 49.8 percent increase.
Region 20 (Denver): Use among high school seniors increased from 30 percent in 2013 to 33 percent in 2015, a jump of 10 percent. Among juniors, the use rate increased from 29 percent to 37.7 percent, an increase of 30 percent.
Region 12: Western Corridor (Summit, Eagle-Vail): Use among high school seniors increased 90 percent from 20.1 percent in 2013 to 38.2 percent in 2015. As a curious side note, this region also reported a 2.3 percent decrease in past-month marijuana use among high school juniors and a 54.7 percent increase among its high school sophomores.
Region 11: Northwest (Steamboat Springs, Craig): Marijuana use among this region’s high school students rose in grades 9-12. Among seniors the rate increased 57.3 percent from 22.5 percent in 2013 to 35.4 percent in 2015. Among juniors, use rose 18.8 percent from 18.1 percent to 21.5 percent. Among sophomores, use rose 72 percent from 8.2 percent in 2013 to 14.1 percent in 2015. Among freshmen, use rose 22.2 percent from 8.1 to 9.9 percent.
Region 19: (Mesa County/Grand Junction): Use among freshmen jumped to 13.7 percent, an increase of 57.5 percent from 2013. Use among sophomores increased 50.6 percent from 26.2 percent from 17.4 percent in 2013. The use rate among high school seniors rose to 24.4 percent, an increase of 20.8 percent.
Region 7: Pueblo: Although there was little change in use rates, the rates remain stubbornly high. They are higher than the state average for all grades; ranges from double the state average for high school freshmen to 31 percent greater than the state average for high school seniors.
A common theme among these regions is a high level of marijuana commercialization in the forms of retail and medical stores. Other commonalities should be investigated to determine the most appropriate interventions.
Analysis of the 2015 survey also found some good news — particularly in regions 8 (San Luis Valley), 10 (West Central, including Gunnison, Hinsdale and Montrose ) and 17 (Central, including Gilpin and Teller). The reasons for these reported declines in past-month use should be explored. For example, are the declines because of an effective intervention, or are they related to a change in the survey methodology from 2013 to 2015? Based on the findings, protocols for prevention and intervention should be implemented to encourage similarly favorable results in other school districts throughout the state.
This entry for DrThurstone.com was co-written by Dr. Christian Thurstone and Christine Tatum. He is an associate professor of addiction psychiatry and the director of medical training of the addiction psychiatry fellowship program at the University of Colorado. She is a longtime journalist, former national president of the Society of Professional Journalists and Dr. Thurstone’s wife. Together, they also wrote Clearing the Haze: Helping Families Face Teen Addiction (Rowman & Littlefield, 2015).
We welcome all thoughtful comments, but please abide by our commenting rules