Pathways to Prevention Workshop

Methods for Evaluating Natural Experiments in Obesity

December 5–6, 2017 in Natcher Conference Center, NIH Campus, Bethesda, Maryland

Speakers and Biographies

Speakers

 

Please note: This information was correct at the time of the workshop.

Speakers

S. Sonia Arteaga, Ph.D.
NIH Institute/Center P2P Coordinator
Program Director
Clinical Applications and Prevention Branch
Division of Cardiovascular Sciences
National Heart, Lung, and Blood Institute
National Institutes of Health
Bethesda, MD
Rachel Ballard, M.D., M.P.H.
Director of Prevention Research Coordination
Office of Disease Prevention
Division of Program Coordination, Planning, and Strategic Initiatives
Office of the Director
National Institutes of Health
Bethesda, MD
Shari L. Barkin, M.D., M.S.H.S.
William K. Warren Foundation Endowed Chair
Professor of Pediatrics
Division Chief of General Pediatrics
Director of Pediatric Obesity Research
Vanderbilt University School of Medicine
Nashville, TN
Sanjay Basu, M.D., Ph.D.
Assistant Professor of Medicine
Stanford University
Palo Alto, CA
David Berrigan, Ph.D., M.P.H., M.S.
NIH Institute/Center P2P Coordinator
Program Director
Behavioral Research Program
Division of Cancer Control and Population Sciences
National Cancer Institute
National Institutes of Health
Bethesda, MD
Jay Bhattacharya, M.D., Ph.D.
Professor of Medicine
Center for Primary Care and Outcomes Research
Stanford University School of Medicine
Stanford, CA
Jamie F. Chriqui, Ph.D., M.H.S.
Professor and Director, M.P.H. Program
Division of Health Policy and Administration
Co-Director, Health Policy Center
Fellow, Institute for Health Research and Policy
University of Illinois at Chicago School of Public Health
Chicago, IL
Tomas D. Cook, Ph.D.
Joan and Serepta Harrison Emeritus Professor of Ethics and Justice
Professor Emeritus of Sociology, Psychology, Education, and Social Policy
Northwestern University
Research Professor
George Washington Institute of Public Policy
Trachtenberg School of Public Policy
The George Washington University
Washington, DC
Tamara Dubowitz, Sc.D., M.Sc., S.M.
Faculty
Pardee RAND Graduate School
Senior Policy Researcher
RAND Corporation
Pittsburgh, PA
Christina D. Economos, Ph.D.
Professor
New Balance Chair in Childhood Nutrition
Friedman School of Nutrition Science and Policy
Tufts University Medical School
Boston, MA
Karen M. Emmons, Ph.D. (Workshop and Panel Chair)
Dean for Academic Affairs
Harvard T.H. Chan School of Public Health
Boston, MA
Stephen P. Fortmann, M.D.
Emeritus Faculty
Stanford University School of Medicine
Affiliate Professor
Oregon Health & Science University
Senior Investigator
Assistant Program Director
Medical Director
Kaiser Permanente Northwest
Portland, OR
Christopher L. Fulcher, Ph.D.
Director
Center for Applied Research and Engagement Systems
University of Missouri
Columbia, MO
Gary H. Gibbons, M.D.
Director
National Heart, Lung, and Blood Institute
National Institutes of Health
Bethesda, MD
Rachel Gold, Ph.D., M.P.H.
Lead Research Scientist
OCHIN, Inc.
Investigator
Center for Health Research
Kaiser Permanente Northwest Center for Health Research
Portland, OR
Steven L. Gortmaker, Ph.D.
Professor of the Practice of Health Sociology
Department of Social and Behavioral Sciences
Harvard T.H. Chan School of Public Health
Boston, MA
Ross A. Hammond, Ph.D.
Senior Fellow, Economic Studies
Director
Center on Social Dynamics and Policy
Economic Studies Department
The Brookings Institution
Washington, DC
Christine Hunter, Ph.D., M.A., ABPP
NIH Institute/Center P2P Coordinator
Deputy Director
Office of Behavioral and Social Sciences Research
National Institutes of Health
Bethesda, MD
Michael Jerrett, Ph.D., M.A.
Professor and Chair
Department of Environmental Health Sciences
Director
Center for Occupational and Environmental Health
Fielding School of Public Health
University of California, Los Angeles
Los Angeles, CA
Laura Kettel Khan, Ph.D.
Senior Scientist
Division of Nutrition, Physical Activity and Obesity
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Control and Prevention
Atlanta, GA
Robin McKinnon, Ph.D., M.P.A.
Senior Advisor for Nutrition Policy
Center for Food Safety and Applied Nutrition
Office of Foods and Veterinary Medicine
U.S. Food and Drug Administration
College Park, MD
David M. Murray, Ph.D.
Associate Director of Disease Prevention
Director
Office of Disease Prevention
Division of Program Coordination, Planning, and Strategic Initiatives
Office of the Director
National Institutes of Health
Bethesda, MD
Adetokunbo “Toks” Omishakin, M.U.R.P.
Deputy Commissioner/Chief
Environment and Planning Division
Tennessee Department of Transportation
Nashville, TN
Lisa M. Powell, Ph.D.
Distinguished Professor and Director
Division of Health Policy and Administration
University of Illinois at Chicago School of Public Health
Chicago, IL
Lorrene Ritchie, Ph.D., R.D.
Director
Nutrition Policy Institute
Cooperative Extension Specialist
Division of Agriculture and Natural Resources
University of California, Berkeley
Oakland, CA
Griffin P. Rodgers, M.D., M.B.A., MACP
Director
National Institute of Diabetes and Digestive and Kidney Diseases
National Institutes of Health
Bethesda, MD
James F. Sallis, Ph.D.
Professorial Fellow
Institute for Health and Ageing
Australian Catholic University, Melbourne
Distinguished Professor Emeritus
Department of Family Medicine and Public Health
University of California, San Diego
La Jolla, CA
Glenn E. Schneider, M.P.H.
Chief Program Officer
The Horizon Foundation of Howard County, Inc.
Columbia, MD
Andrew Turner, M.S.
Director and Chief Technology Officer
Research and Development Center, DC
Environmental Systems Research Institute
Arlington, VA

AHRQ Johns Hopkins Evidence-based Practice Center Speakers

Wendy L. Bennett, M.D., M.P.H.
Associate Professor
The Johns Hopkins University School of Medicine
Baltimore, MD
Lawrence J. Cheskin, M.D., FACP, FTOS
Director of Clinical Research
Associate Professor
Departments of Health, Behavior and Society, and International Health
Johns Hopkins Bloomberg School of Public Health
Baltimore, MD
Hadi Kharrazi, M.D., Ph.D., M.H.I.
Assistant Professor
Department of Health Policy Management
Johns Hopkins Bloomberg School of Public Health
Baltimore, MD
Emily A. Knapp, M.H.S.
Doctoral Candidate
Johns Hopkins Bloomberg School of Public Health
Baltimore, MD
Eva Tseng, M.D., M.P.H.
Assistant Professor
The Johns Hopkins University School of Medicine
Baltimore, MD

Biographies and Presentations

S. Sonia Arteaga, Ph.D.

S. Sonia Arteaga, Ph.D. S. Sonia Arteaga, Ph.D., is a Program Director at the National Heart, Lung, and Blood Institute (NHLBI). She leads the Healthy Communities Study, which is a large observational study of 130 diverse communities and 5,138 children and their families to assess the characteristics of programs and policies and their associations with body mass index, diet, and physical activity in children. She is also a member of the Senior Leadership Group of the National Institutes of Health (NIH) Obesity Research Task Force and provides leadership on the development and coordination of obesity research efforts across the NIH. Dr. Arteaga also oversees grants targeting appropriate gestational weight gain, has a diverse portfolio of obesity-related grants, and works on projects related to her research interests that include multilevel approaches to obesity prevention, behavior change, and reducing health disparities. She is also Deputy Director of the NHLBI’s Programs To Increase Diversity Among Individuals Engaged in Health-Related Research (PRIDE) training program. Dr. Arteaga received her Ph.D. in community-social/behavioral medicine psychology from the University of Maryland Baltimore County.


Rachel Ballard, M.D., M.P.H.

Rachel Ballard, M.D., M.P.H. Rachel Ballard, M.D., M.P.H., leads the Collaborative Prevention Initiatives team within the National Institutes of Health (NIH) Office of Disease Prevention. This team is responsible for working across the NIH and with other federal partners to advance collaborative prevention initiatives. She previously led the Applied Research Program at the National Cancer Institute—a program that focused on advancing evidence on the influence of cancer care and control activities. She has published widely on physical activity, diet and weight control, and screening and quality of care in cancer. She trained in environmental sciences, internal medicine, pediatrics, clinical nutrition, epidemiology, and preventive medicine. In 1990–1991, she served as the Nutrition Policy Advisor in the U.S. Department of Health and Human Services (HHS) providing input on dietary guidelines and efforts related to nutrition labeling. She has led national and international research initiatives to evaluate the performance of cancer screening in practice; directed a multi-ethnic cohort of breast cancer survivors designed to examine the effect of physical activity, diet, and weight on cancer biomarkers and outcomes; and led an NCI effort to advance research on the combined effects of diet, physical activity, and weight on cancer and overall health outcomes. She is the NIH Co-Chair of the National Collaboration on Childhood Obesity Research—recognized by the first HHS Innovates award for successful private/public partnerships. She served as one of the NIH senior editors developing the first federal report of nutrition research priorities—the National Nutrition Research Roadmap. She also is serving as one of the federal staff supporting the 2018 Physical Activity Guidelines Advisory Committee report, and on the steering committee for a weight-loss trial in cancer survivors.


Shari L. Barkin, M.D., M.S.H.S.

Shari L. Barkin, M.D., M.S.H.S. Shari L. Barkin, M.D., M.S.H.S., is the William K. Warren Foundation Chair and Professor of Pediatrics, Director of Pediatric Obesity Research in the Diabetes Center, and Chief of General Pediatrics at Vanderbilt University Medical Center. She also serves as the Executive Director of the Nashville Collaborative, an innovative academic-community partnership to measurably reduce pediatric obesity. Dr. Barkin earned her undergraduate degree at Duke University and her medical degree at the University of Cincinnati, and completed her pediatrics residency at Children’s Hospital of Los Angeles. She was selected as a University of California, Los Angeles Robert Wood Johnson Clinical Scholar and completed a fellowship in health services research. Funded by the National Heart, Lung, and Blood Institute (NHLBI), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), the National Cancer Institute (NCI), the Robert Wood Johnson Foundation, and the State of Tennessee, Dr. Barkin conducts family-based community-centered interventions to change health behaviors in parent-child dyads and measurably improve child health outcomes. Currently, her laboratory studies family-based, community-centered, behavioral interventions to measurably reduce pediatric obesity at sensitive periods of childhood development. The lab is focused on changing body mass index (BMI) trajectories in childhood, applying the ecologic model that considers the child in the context of their family, and the family in the context of their community. A theme of the lab is to identify and understand the interaction between behavior, environment, and genetics across childhood development, and ultimately, to prevent later common chronic conditions. This lab applies a wide variety of techniques to address these complex problems, including qualitative and quantitative methodologies. The lab considers objective biologic measurements (such as fat mass and BMI), genetic measurements (epigenetic patterns, genetic allelic risk scores), social measurements (social networks), and behavioral measurements (actigraphy and diet changes over time in both parents and children). She is the Principal Investigator of the Growing Right Onto Wellness Trial, a 7-year randomized, controlled trial to prevent childhood obesity funded by the NHLBI and the NICHD and part of the National Institutes of Health Childhood Obesity Prevention and Treatment Research Consortium. In addition, she is funded by the NCI to test dissemination and implementation science approaches for evidence-based pediatric obesity behavioral interventions. She served on the National Academy of Medicine’s Board of Children, Youth, and Families and is the Past President of the Society for Pediatric Research.

Opportunities To Fill the Data Gaps

Often we get lost in the meaning when we only report on mean prevalence and trends of obesity. We miss the nuance and what could matter for whom to move the field of obesity prevention forward. The U.S. population is becoming more diverse, and this must be adequately addressed in obesity research, including subpopulations within racial/ethnic groups and the full spectrum of childhood ages from birth to 18. Opportunities to leverage existing infrastructure and enhance collaborative efforts could fill data gaps, such as accurately and consistently measuring height and weight, capturing geographic differences, and assessing rural versus urban trends. For example, immunization registries could be expanded to include height and weight data, or data could be pooled from randomized controlled trials to examine these more granular differences for subpopulations. Moreover, we are at a time in history when emerging technologies potentially provide new vehicles for data capture. Examining how we could utilize wearable device data to merge with larger databases could provide us with a deeper understanding for more tailored pathways to prevention.


Sanjay Basu, M.D., Ph.D.

Sanjay Basu, M.D., Ph.D. Sanjay Basu, M.D., Ph.D., is an Assistant Professor of Medicine at Stanford University. He is a practicing primary care physician, and directs both the Analytics and Modeling Core of the SPHERE Center (Stanford Precision Health for Economic and Racial Equity) and the Health Disparities Research Group of Stanford’s Center for Population Health Sciences. Dr. Basu’s research focuses on public health operations research, which involves the application of data analysis and computer modeling methods from the fields of epidemiology, computer science, engineering, and econometrics to study how public health and primary health care programs can better prevent and treat common diseases. He received his undergraduate education at MIT, was a Rhodes Scholar at Oxford, and completed his M.D. and Ph.D. at Yale University before completing his residency in primary care internal medicine at the University of California, San Francisco.

Novel Strategies for Estimating Effects of Natural Experiments

Background

The problem I seek to address is how to better estimate the effect of population-level interventions evaluated through natural experiments. Three common methodological problems affect the accurate estimation of effects of such interventions: (1) the lack of availability of an ideal control population; (2) the lack of availability of individual-level data; and (3) the difficulty of extrapolating from the studied sample to the broader population to analyze the impact on overall disparities.1

Brief Content Summary

I will discuss the implications of three new methods to study natural experiments and mitigate the above problems: (1) near-far matching, a strategy that can correct for both measured and unmeasured confounders when an ideal control population is not available to compare against individuals affected by the natural experiment;2 (2) synthetic control methods, which help to produce a better control for population-level data and can bound the influence of unmeasured confounders;3 and (3) distributional decomposition methods, which help extrapolate from a study sample to a broader population, to identify the potential impact of an intervention on broader population-level disparities.4

  • Near-far matching involves the combination of classical matching techniques on observable characteristics (e.g., propensity score matching) with instrumental variable analysis to adjust for unmeasured confounders. By matching individuals to be �near� (similar) on observable characteristics and �far� (dissimilar) on values of an instrumental variable, the method can help isolate effects of an intervention even when available control groups have notable differences in characteristics or cultural/social features that are difficult to characterize. The matching method enables an observational analysis to mimic a matched-pair randomized controlled trial, where the instrumental variable acts like a randomizer and is strengthened in its randomization power by simultaneous matching on observables and the value of the instrument.5
  • Synthetic control methods involve the construction of populations that can better match a group exposed to a natural experiment. It is common that natural experiments will affect a group that has both observed and unobserved differences from possible control groups. For example, a natural experiment in the city of San Francisco may affect not only demographic groups that are different from the rest of the country, but also affect groups whose difficult-to-measure cultural and social features are very different from much of the rest of the country. Synthetic control methods use weighted samples from the rest of the country to assemble a synthetic San Francisco and bound the influence of unmeasured confounders, enabling less bias in effect size estimation when large populations are studied. Unlike classical difference-in-difference methods, the synthetic control method can be applied even if time trends in the outcome variable differ between affected and unaffected populations, and even if time-varying confounders exist.6
  • Distributional decomposition involves a rebalancing from the study populations to a reference population to which the study applies. Decomposition answers the question: if an intervention were extended to this real-world population, how much would it likely affect the disparities between groups in the real-world population, given how much it differently affected groups in the study population? The method can help to illustrate the population-level policy implications of a natural experiment studied in a cohort that is not necessarily representative of the overall population composition.4
Discussion

Current practice in studying natural experiments is to perform a standard difference-in-difference analysis, which relies on the availability of a good control group, can only correct for time-invariant confounders, and assumes a cohort study population is similar to the broader population to which the study applies.1 The above three methods can help overcome these limitations of standard natural experiment evaluation, to enhance the accurate estimation of natural experiment effects and generalize the implications of natural experiments for policy consideration.

Key Recommendations
  • Novel methods such as near-far matching, synthetic control analysis, and distributional decomposition should be considered when natural experiments are affected by a lack of ideal control populations, time-varying confounders, and differences between the studied population and the broader population to which the intervention might eventually be generalized.
References
  1. Basu S, Meghani A, Siddiqi A. Evaluating the health impact of large-scale public policy changes: classical and novel approaches. Annu Rev Public Health. 2017;38(1):351–370. http://www.annualreviews.org/doi/abs/10.1146/annurev-publhealth-031816-044208 . Accessed April 19, 2017.
  2. Rigdon J, Baiocchi M, Basu S. Near-far matching in R: the nearfar package. J Stat Soft. 2017. Epub ahead of print.
  3. Abadie A, Diamond A, Hainmueller J. Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J Am Stat Assoc. 2010;105(490):493–505.
  4. Basu S, Hong A, Siddiqi A. Using decomposition analysis to identify modifiable racial disparities in the distribution of blood pressure in the United States. Am J Epidemiol. 2015;182(4):345–353.
  5. Rigdon J, Berkowitz SA, Seligman HK, Basu S. Re-evaluating associations between the Supplemental Nutrition Assistance Program participation and body mass index in the context of unmeasured confounders. Soc Sci Med. 2017;192:112–124. http://www.sciencedirect.com/science/article/pii/S0277953617305543 . Accessed April 19, 2017.
  6. Basu S, Rehkopf DH, Siddiqi A, Glymour MM, Kawachi I. Health behaviors, mental health, and health care utilization among single mothers after welfare reforms in the 1990s. Am J Epidemiol. 2016;183(6):531–538.

Wendy L. Bennett, M.D., M.P.H.

Wendy L. Bennett, M.D., M.P.H. Wendy L. Bennett, M.D., M.P.H., is an Associate Professor of Medicine in the Johns Hopkins University School of Medicine, Division of General Internal Medicine, with joint appointments in the Johns Hopkins Bloomberg School of Public Health Department of Epidemiology and Department of Population, Family and Reproductive Health. Her research has focused on the prevention and management of obesity and type 2 diabetes. She recently co-led the systematic evidence review on “Methods for Evaluating Natural Experiments in Obesity” that has informed the National Institutes of Health Pathways to Prevention Workshop. In addition, she has developed and is testing a health-coaching behavioral intervention to limit weight gain in pregnancy and serves as a co-investigator for the weight cohort in the Patient-Centered Outcomes Research Institute-funded PaTH Clinical Data Research Network. She teaches in the Johns Hopkins Schools of Public Health and Medicine and is the Co-Director of the Johns Hopkins Center for Women’s Health, Sex and Gender Research. Clinically, she is a practicing general internist at Johns Hopkins Community Physicians.

Evidence-based Practice Center Presentation

Objective

Given the enormity of obesity as a public health problem, rigorous methodological approaches, including natural experiments, are needed to evaluate the effectiveness of policies and programs to prevent and control obesity. Our objective was to systematically review studies evaluating programs and policies addressing obesity prevention and control in terms of their population-based data sources, use of data linkages, measures reported, study designs, and analytic approaches.

Data Sources

We systematically searched PubMed, CINAHL, PsycINFO, and EconLit from 2000 to August 21, 2017, to identify all U.S. and non-U.S. studies of programs or policies targeting obesity prevention and control in people of all ages and in any setting.

Review Methods

Two independent reviewers screened abstracts and full-text articles. We required articles to address a program, policy, or built environment change and have a defined comparison or unexposed group. We used the Effective Public Health Practice Project (EPHPP) tool to rate studies for their risk of bias.

Results

In all, 295 studies were eligible for inclusion (189 U.S. and 106 non-U.S.); 157 (53%) were natural experiment studies. Studies reported 116 unique primary or secondary shareable data sources, of which 106 (71 U.S. and 35 non-U.S.) data sources met criteria for a data system and 26 of the 71 U.S. data systems were linked with a secondary data source other than the primary data source. Also, 112 studies reported childhood weight measures, 33 had adult weight measures, 152 had physical activity measures, and 149 had dietary measures. Most natural experiment studies were rated as having a “weak” global rating (i.e., high risk of bias), with 64% having a weak rating for handling of withdrawals and dropouts, 25% having a weak rating for study design, 40% having a weak rating for confounding, and 26% having a weak rating for data collection.

Conclusions

We identified a large number of studies and data sources that used a wide variety of outcome measures and analytic methods, often with substantial risk of bias. The findings reinforce the need for methodological and analytic advances that would strengthen efforts to improve obesity prevention and control.


David Berrigan, Ph.D., M.P.H., M.S.

David Berrigan, Ph.D., M.P.H., M.S. David Berrigan, Ph.D., M.P.H., M.S., has been a biologist in the Division of Cancer Control and Population Sciences since 2003. He joined the Health Behaviors Research Branch in 2015. Previously, he served as a Cancer Prevention Fellow with funding from the Division of Cancer Prevention from 1999 to 2003. Before coming to the National Cancer Institute, he was a postdoctoral fellow and lecturer at the University of Washington and at La Trobe University in Melbourne, Australia, with funding from the National Science Foundation and the U.S. Department of Agriculture. Dr. Berrigan received his Ph.D. in biology from the University of Utah in 1993, a B.A. from Reed College in 1983, M.S. from University of California, Davis in 1987, and M.P.H. from University of California, Berkeley in 2000. His recent research has examined energy balance, carcinogenesis, physical activity, and acculturation using a mix of animal models, population data, and methodological studies aimed at improving survey data and incorporating GIS tools and data layers into survey datasets. Two resources he has worked on are the Urban Sprawl Index Update (co-managed with Z. Tatalovich) for the GIS Resource Center for Cancer Research and the Measures Registry Project of the National Coalition for Childhood Obesity Research (co-lead with Dr. Jill Reedy), National Collaborative on Childhood Obesity Research. He has been on the editorial board of the journal Functional Ecology, a reviewer for the National Institutes of Health (NIH), National Science Foundation, Natural Sciences and Engineering Research Council of Canada, and other funding agencies, and a peer reviewer for many journals. He was on the program committee for the 2007 Active Living Research Conference and has received six NIH Merit Awards. Dr. Berrigan is strongly committed to research aimed at health for all via environments and institutions that foster healthy behaviors, preventive services, and health care regardless of demographic or economic circumstances.


Jay Bhattacharya, M.D., Ph.D.

Jay Bhattacharya, M.D., Ph.D. Jay Bhattacharya, M.D., Ph.D., is a Professor at the Stanford University School of Medicine, as well as a Professor (by courtesy) in the Department of Economics and the Department of Health Research and Policy at Stanford. He is a Research Associate at the National Bureau of Economics Research and a Senior Fellow at the Stanford Institute for Economic Policy Research and the Stanford Freeman Spogli Institute. Professor Bhattacharya received his Ph.D. in economics and his M.D. from Stanford University. Professor Bhattacharya’s research aims to understand the constraints that vulnerable populations face in making decisions that affect their health status, and, in particular, how government policies designed to benefit these populations actually affect the lives of people in such groups. He has over 100 peer-reviewed publications in economics, statistical, medical, and health policy journals, and has published National institutes of Health-supported papers on the economics of obesity.

When Is Correlation Causation?

A major statistical problem in research on the consequences of obesity is that body mass is never randomly assigned. Thus, the standard approach to questions of causal inference— controlled randomization with a well-defined control group—cannot work to address important questions about the health and economic consequences of obesity. Researchers have addressed this problem by applying statistical methods, such as instrumental variables, difference-in-difference methods, and regression discontinuity analyses, which are popular in the econometric literature and have the potential to identify causal parameters even when body weight is not randomly assigned. The difficulty with these methods is that they often entail strong assumptions that can be difficult to verify. In this talk, I will discuss a relatively recently developed method—bounding—that enables researchers to relax these strong assumptions. Bounding methods, even though they have not been widely adopted by obesity researchers, have great potential to credibly answer difficult questions in obesity research when randomization is not possible.


Lawrence J. Cheskin, M.D., FACP, FTOS

Lawrence J. Cheskin, M.D., FACP, FTOS Lawrence J. Cheskin, M.D., FACP, FTOS, is Associate Professor, Health, Behavior and Society at Johns Hopkins Bloomberg School of Public Health, with joint appointments in Medicine (Gastrointestinal); International Health (Human Nutrition), Nursing, and Public Health Studies. He directs the Johns Hopkins Weight Management Center, a multidisciplinary clinical research and treatment program he founded. He is also Director of Clinical Research of the Global Obesity Prevention Center at Johns Hopkins and directs its Pilot Studies Core, which evaluates timely, systems-focused proposals worldwide to study such areas as school policies and the built environment’s effect in preventing childhood obesity. Dr. Cheskin’s work is at the intersection of public health and clinical medicine: applying knowledge gained through rigorous clinical investigation to the community and beyond. His work has impacted the problem of obesity through innovative treatment paradigms disseminated through practice (as modeled by the Johns Hopkins Weight Management Center); through application of new treatment paradigms in community-based participatory research, particularly among the underserved in Baltimore, Maryland; and through mentoring of the next generation of public health and clinical scholars dedicated to obesity. He has over 175 peer-reviewed journal publications and has written six books.

Evidence-based Practice Center Presentation

Objective

Given the enormity of obesity as a public health problem, rigorous methodological approaches, including natural experiments, are needed to evaluate the effectiveness of policies and programs to prevent and control obesity. Our objective was to systematically review studies evaluating programs and policies addressing obesity prevention and control in terms of their population-based data sources, use of data linkages, measures reported, study designs, and analytic approaches.

Data Sources

We systematically searched PubMed, CINAHL, PsycINFO, and EconLit from 2000 to August 21, 2017, to identify all U.S. and non-U.S. studies of programs or policies targeting obesity prevention and control in people of all ages and in any setting.

Review Methods

Two independent reviewers screened abstracts and full-text articles. We required articles to address a program, policy, or built environment change and have a defined comparison or unexposed group. We used the Effective Public Health Practice Project (EPHPP) tool to rate studies for their risk of bias.

Results

In all, 295 studies were eligible for inclusion (189 U.S. and 106 non-U.S.); 157 (53%) were natural experiment studies. Studies reported 116 unique primary or secondary shareable data sources, of which 106 (71 U.S. and 35 non-U.S.) data sources met criteria for a data system and 26 of the 71 U.S. data systems were linked with a secondary data source other than the primary data source. Also, 112 studies reported childhood weight measures, 33 had adult weight measures, 152 had physical activity measures, and 149 had dietary measures. Most natural experiment studies were rated as having a “weak” global rating (i.e., high risk of bias), with 64% having a weak rating for handling of withdrawals and dropouts, 25% having a weak rating for study design, 40% having a weak rating for confounding, and 26% having a weak rating for data collection.

Conclusions

We identified a large number of studies and data sources that used a wide variety of outcome measures and analytic methods, often with substantial risk of bias. The findings reinforce the need for methodological and analytic advances that would strengthen efforts to improve obesity prevention and control.


Jamie F. Chriqui, Ph.D., M.H.S.

Jamie F. Chriqui, Ph.D., M.H.S. Jamie F. Chriqui, Ph.D., M.H.S., is a Professor of Health Policy and Administration and Co-Director of the Health Policy Center in the Institute for Health Research and Policy in the School of Public Health at the University of Illinois at Chicago. She has over 27 years’ experience conducting public health policy research, evaluation, and analysis. Her research focuses on obesity and school nutrition-related laws and policies adopted nationwide and their implementation and/or impact on communities, schools, and individual-level outcomes. Topically, her work has emphasized school wellness, zoning and the built environment, and food and nutrition policy including taxation. She served on two Institute of Medicine obesity prevention-related committees; she is an appointed member of the Community Preventive Services Task Force that produces the Community Guide; and she is an advisor for numerous federal, foundation, and non-profit organizations regarding obesity and school policy-related issues, research, and evaluation studies. She holds a B.A. in political science from Barnard College at Columbia University in New York; an M.H.S. in health policy from Johns Hopkins University School of Hygiene and Public Health; and a Ph.D. in policy sciences with a health policy concentration from the University of Maryland, Baltimore County.

Measures of Exposure to Policy and Environment

Background

While many natural experiments in obesity prevention are often conducted within one jurisdiction where a specific natural experiment is occurring such as a new light rail that may increase physical activity, a complete streets project, or zoning and land use policies that explicitly permit food outlets or farmers’ markets in food desert-designated areas, researchers may also examine natural experiments occurring nationally or across multiple jurisdictions. This presentation will briefly highlight key considerations, challenges, and opportunities for measuring policy and environmental “exposures” that may serve as the natural experiment (i.e., the intervention), mediator, and/or outcome measure depending on the research question at hand.

Brief Content Summary

This presentation will focus on the policy and environmental exposures that may affect or are associated with obesity prevention and control. The presentation will (1) briefly define and summarize the primary approaches to measuring policy and environmental exposures for use in studies seeking to assess the impact of policy and/or environmental change-related “natural experiments” across jurisdictions on food and activity-related outcomes; (2) highlight why it is necessary to ensure a “conceptual match” between the measures of policy and/or environmental change-related exposures and the nutrition/diet and activity-related outcomes to which they are being linked; and (3) discuss challenges and opportunities with regards to measuring obesity-related policy and environmental exposures.

Discussion

Studies that seek to evaluate the impact of policy and built environment changes across jurisdictions on obesity-related outcomes need to rely on systematic data collection methods to ensure comparability in what is being measured and, ultimately, evaluated (Brownson et al., 2009; Chriqui et al., 2011; Lytle and Sokol, 2017; McKinnon et al., 2009; Sallis, 2009). When the intervention or natural experiment of interest across jurisdictions is a public policy change, such as a new complete streets policy or nutrition standards governing a new vending machine policy in public places, it is necessary to compile and evaluate the content of the policies to ensure comparable policy content is being measured across jurisdictions. This type of policy surveillance and evaluation is different than simply measuring whether a policy is “on the books” or not; instead, it actually assesses policies across jurisdictions to ensure “apples to apples” in terms of policy provisions or content (Chriqui et al., 2016; Chriqui et al., 2011; Schwartz et al., 2009). Likewise, when measuring the food and physical activity-related built environments, a systematic assessment of the environment across jurisdictions is needed to also ensure that comparable environmental features (e.g., food store presence, food marketing, sidewalks, parks, open space) are being consistently measured (Brownson et al., 2004; Lytle and Sokol, 2017; McKinnon, Reedy, Handy, and Rodgers, 2009; McKinnon, Reedy, Morrissette, et al., 2009; Sallis, 2009). Equally important with studies of policy or environmental exposures is to ensure a “conceptual match” between the policy and/or environmental measures and the food and activity-related outcomes being measured (Ding and Gebel, 2012). Ding and Gebel offer the example of studies that correlate recreational environments with active transportation as being statistically significant even though recreational environments are not an appropriate measure of environmental factors that might influence active travel; the authors consider such a conceptual mismatch to be a Type II error (Ding and Gebel, 2012). Rather, measures of environmental features such as the availability, access to, and density of public transit, bus stops, street and trail network connectivity are better “conceptual matches” for linking with an active travel outcome. This presentation will highlight primary strategies for measuring policy and environmental exposures in a systematic and reliable fashion, will discuss why and how to avoid “conceptual mismatches” in examining the relationship and/or impact of policy and environmental exposures on individual outcomes, and will discuss challenges and opportunities with regards to measurement of obesity-related policy and environmental exposures.

References

Brownson RC, Chang JJ, Eyler AA., et al. Measuring the environment for friendliness toward physical activity: a comparison of the reliability of 3 questionnaires. Am J Public Health. 2004;94(3):473–483.

Brownson RC, Hoehner CM, Day K, Forsyth A, Sallis JF. Measuring the built environment for physical activity: state of the science. Am J Prev Med. 2009;36(4, suppl 1):S99–S123.

Chriqui JF, Leider J, Thrun E, Nicholson L, Slater S. Communities on the move: pedestrian-oriented zoning as a facilitator of adult active travel to work in the United States. Front Public Health. 2016;4:71. doi: 10.3389/fpubh.2016.00071. doi:10.3389/fpubh.2016.00071

Chriqui JF, O’Connor JC, Chaloupka FJ. What gets measured, gets changed: evaluating law and policy for maximum impact. J Law Med Ethics. 2011;39(suppl 1):21–26.

Ding D, Gebel K. Built environment, physical activity, and obesity: what have we learned from reviewing the literature? Health Place. 2012;18(1):100–105. doi:10.1016/j.healthplace.2011.08.021

Lytle LA, Sokol RL. Measures of the food environment: a systematic review of the field, 2007–2015. Health Place. 2017;44:18–34. doi:10.1016/j.healthplace.2016.12.007

McKinnon RA, Reedy J, Handy SL, Rodgers AB. Measuring the food and physical activity environments: shaping the research agenda. Am J Prev Med. 2009;36(4, suppl 1):S81–S85.

McKinnon RA, Reedy J, Morrissette MA, Lytle LA, Yaroch AL. Measures of the food environment: a compilation of the literature, 1990–2007. Am J Prev Med. 2009;36(4, suppl 1):S124–S133.

Sallis JF. Measuring physical activity environments: a brief history. Am J Prev Med. 2009;36(4, suppl 1):S86–S92.

Schwartz MB, Lund AE, Grow HM, et al. A comprehensive coding system to measure the quality of school wellness policies. J Am Diet Assoc. 2009:109(7):1256-1262.


Tomas D. Cook, Ph.D.

Tomas D. Cook, Ph.D. Tomas D. Cook, Ph.D., has a B.A. from Oxford in German and French and a Ph.D. from Stanford University. He was a Professor of Psychology at Northwestern University for 48 years, a Professor of Education and Social Policy for 35 of them, and of Sociology for 28 of them. During most of this time, he was also a Faculty Fellow at the Institute of Policy Research. He has now retired from Northwestern and is a Research Professor at the Trachtenberg School of Public Policy at George Washington University in Washington, DC. He mostly publishes on quasi-experimental design and analysis, having authored several frequently cited books on the topic. For the last 7 years, he has been involved with the conduct of within-study comparisons that judge the efficacy of various quasi-experimental design and analysis techniques in terms of their ability to robustly recreate the same results as a randomized experiment on the same topic and sharing the same treatment group and assessment details.

The Use of Quasi-Experimental Designs in Evaluating Natural Experiments

Two premises of this paper are that no single form of casual design is appropriate for all types of natural experiments and that methods for the design and analysis of lottery-based natural experiments are well known. Thus, this presentation is about empirical justifications for the choice of particular non-experimental methods for evaluating the effects of natural experiments. Specifically, it summarizes the results of within-study comparisons that deliberately contrast effect estimates from a randomized experiment with estimates from some form of a non-experiment when the experiment and non-experiment share the same treatment group, the same outcome measurement, and the same estimand (quantity being estimated). We examine three different kinds of quasi-experiments. Meta-analysis of 15 relevant studies is used to estimate, first, how close the average causal estimates are for experimental and regression-discontinuity designs. Meta-analysis of 11 studies is then used to estimate how close the average causal estimates are for experiments and comparative interrupted time-series designs. In each case, the distribution of study-specific experimental and non-experimental impact differences is also reported. A less formal type of synthesis is used to contrast effect size differences between experiments and the third type of non-experimental design that is characterized by two or more non-equivalent comparison groups and both a pre-test and post-test measure of the study outcome—a design that is sometimes mislabeled as “difference in difference.”


Tamara Dubowitz, Sc.D., M.Sc., S.M.

Tamara Dubowitz, Sc.D., M.Sc., S.M. Tamara Dubowitz, Sc.D., M.Sc., S.M., is a Senior Policy Researcher at the RAND Corporation and faculty member at the Pardee RAND Graduate School. Trained in social epidemiology with concentrations in maternal and child health and public health nutrition, Dr. Dubowitz’s work has focused on the role of neighborhoods, or “place,” in shaping health and health behaviors, especially related to obesity. Dr. Dubowitz either leads or is a co-investigator on multiple National Institutes of Health-funded natural experiments (R01CA149105 “Does a New Supermarket Improve Dietary Behaviors of Low-Income African Americans?”; R01CA164137 “Impact of Greenspace Improvement on Physical Activity in a Low-Income Community”; R01 HL122460 “Neighborhood Change: Impact on Sleep and Obesity-Related Health Disparities”; R01HL131531“Natural Experiment of Neighborhood Revitalization and Cardiometabolic Health”; and R01CA149105 “Urban Revitalization and Long-Term Effects on Diet, Economic, and Health Outcomes”) that evaluate the effect of large neighborhood-level transformations on residents’ food- purchasing behaviors, dietary intake, active transportation, physical activity, sleep, and cardiometabolic outcomes. Dr. Dubowitz’s work and interests center on looking at vulnerable populations and urban America, and her research agenda is focused on measuring and quantifying the impact of the environment on health. She is similarly committed to recognizing the dynamic role of people, places, and policies.

Natural Experiments and Obesity: Balancing Rigor With Practical Realities

There has been a strong call and recognized need for natural experiments in food environment research and its application to obesity. In a natural experiment, the researcher cannot control or withhold the allocation of an intervention to particular neighborhoods or communities of interest, allowing natural variation in allocation to occur and making such a design one of the best alternatives to a randomized controlled study.1 A pre-post design with an intervention and a control group allows for comparison of the average within-person change in the “intervention” group relative to the average within-person change in a “control” group, resulting in calculation of the net effect of a non-randomized intervention. While intervention and control neighborhoods may have baseline differences, pre- and post-measurements on the same persons eliminate potential person-level differences and increase the precision with which we can detect change. In addition, the use of a control group allows for comparison of changes detected in the intervention group against those observed in the control neighborhood, in order to separate the intervention effect from other simultaneous changes and secular trends. Using propensity score adjustments can further strengthen causal inferences.

And yet, neighborhoods are dynamic as are the residents that inhabit neighborhoods. Neighborhood changes or interventions are rarely isolated events. Investments outside of target neighborhoods, unanticipated delays, or changes in either the intervention or control neighborhood can add complexities to an otherwise well-thought-out design.

The Pittsburgh Hill/Homewood Research on Eating, Shopping and Health was the first of multiple natural experiment studies that have allowed our team to tap into a randomly selected cohort of residents and their changing built environments. I will present our methodology, detail the neighborhood investments, and discuss our findings to date on obesity-related outcomes. Our natural experiments followed (and are following) a cohort of households, but we have also been collecting data on the physical environment (parks, stores, streets). In this presentation, we will present how we maintained science while studying an often unpredictable world of people, places, and communities.

Reference
  1. Petticrew M, Cummins S, Ferrell C, et al. Natural experiments: an underused tool for public health? Public Health. 2005;119(9):751–757.

Christina D. Economos, Ph.D.

Christina D. Economos, Ph.D. Christina D. Economos, Ph.D., is a Professor and the New Balance Chair in Childhood Nutrition at the Friedman School of Nutrition Science and Policy and Medical School at Tufts University. She is the Co-Founder and Director of ChildObesity180, an organization that unites leaders from diverse disciplines to find solutions to the childhood obesity epidemic. ChildObesity180 merges the best in nutrition and public health research and practice with the experience of business, government, and non-profit leaders. ChildObesity180 develops, implements, evaluates, and scales high-impact obesity prevention initiatives. ChildObesity180’s work has reached over 11 million children nationwide. She led the Shape Up Somerville study demonstrating that it is possible to reduce excess weight gain in children through multiple leverage points within an entire community. Currently, she leads the Shape Up Under 5 study designed to bring children to kindergarten with health habits that support their growth and learning. Dr. Economos is involved in national obesity and public health activities and has served on four National Academies of Sciences, Engineering, and Medicine committees. Dr. Economos received a B.S. from Boston University, an M.S. in applied physiology and nutrition from Columbia University, and a doctorate in nutritional biochemistry from Tufts University.

Collecting Diet, Activity, and Obesity Measures in Communities: Lessons Learned

Background

Precise and reliable measures of dietary intake, physical activity, and body composition are essential to improve community nutrition programs and health outcomes. While numerous tools are validated and implemented, novel technologies that reduce participant burden and costs and increase accuracy are crucial to assessing and shifting the current trends in diet-related disease. This presentation will explore the tools used to measure dietary intake, physical activity, and body composition in the community setting. Specifically, the strengths and limitations of currently available and nascent tools will be explored to provide objectives for future research.

Brief Content Summary and Discussion

The National Cancer Institute Dietary Assessment Calibration/Validation Register recognizes over 1,000 tools for assessing dietary intake.1 These include biological measures, chemical analysis and duplicate collection, diet history, dietary recall, food frequency questionnaires, food diaries, checklists, and observation. Of these tools, the most valid and reliable are the biological markers such as doubly labeled water. Biological markers are not subject to lapses in memory and interviewer or participant bias, and are strongly correlated with dietary intake.2,3 Biomarkers are not feasible in all circumstances because of the high associated costs, the inability to capture specific dietary patterns, and the variability of results based on participant anatomy and health status.

The future of dietary composition measures blends current practices with emerging technologies. Computer, Internet, mobile phone, scan, and sensor measures are at the forefront of this emerging technology. Research on Internet and computer-based 24-hour recalls shows potential to reduce measurement error and bias, enhance recall, and automate data entry, calculations, and coding.4,5 Mobile phone applications such as WellNavi, Nutricam, and FoodNow capture pictures, voice recording, text, and context of consumption.6–8 These tools are increasingly relevant as photo assessments of foods are highly correlated with food weights.9,10 These technologies provide the same advantages of the computer and Internet-based tools and further reduce researcher and participant burden while increasing adherence.11,12 Successes in mobile phone applications pave the way for smaller wearable cameras and sensors.13–15

While these technologies promise to increase accuracy of dietary data, the costs associated with development make them prohibitive. Costs include software and hardware development and maintenance, technical issues, and training populations that might be unfamiliar with such technology.14,16–18 This provides context as to why development of new 24-hour recall methods and food frequency questionnaires is still common.

Trends in developing and implementing physical activity measures are similar to those of dietary intake. Biological measures are the gold standard, and self-reported questionnaires, diaries or logs, direct observation, accelerometers, pedometers, heart rate monitors, armbands, and FitBits are the most widely utilized tools. While questionnaires and logs are easy to administer, inexpensive, and can provide real-time data, they lack the sensitivity to measure low and moderate activity, and are subject to issues in recall, participant bias, and seasonality.19,20 Accelerometers, pedometers, armbands, and FitBits also offer real-time measurements and can generally distinguish between levels of intensity with limited bias; however, most data are only accurate when participants are walking or running.19,21–25 Heart monitors can augment other sensors by providing data on intensity, but they lack information on context and are confounded by stress and caffeine.

The most promising and innovative technologies offer a combination of the aforementioned tools with cameras, lab technology, video game consoles, mobile phones, and Geographic Positioning Systems (GPS).26,27 The SenseCam used with a self-recall diary and accelerometer provides validated measures of intensity, duration, and context.28 While mobile phone applications have shown mixed results depending on the population and study goals, they offer low respondent burden and their ubiquity is compelling.29–33 The most integrative work is being performed with GPS; this research offers investigations of the social and environmental contexts of physical activity and precise real-time measurements.34–36

Most trials using these technologies assess small populations (n < 45).29–32 Large comprehensive studies of these new technologies will ensure their validity and reliability in diverse populations and determine the future of these practices. As with the dietary intake technologies, these tools have high costs associated with development, maintenance, and social integration, but could increase accuracy and efficiency in reporting data.

Measures of obesity and body composition generally rely on in-person interactions and physical equipment. Dual-energy X-ray absorptiometry scans, air displacement, and underwater weighing are considered to be the most accurate measures of body composition; however, these can be time consuming and expensive. The most commonly used tools for measuring body composition are skinfold thickness, body mass index, and waist circumference. These measures are quick and easy to perform but do not give a clear picture of fat distribution.37

There is substantial room for growth in the technology of body composition instruments and measures. Currently, there are few novel techniques for assessing body composition within a community. Some studies have looked at smart scales and ultrasounds; however, there is limited research on the connection between mobile devices, cameras, and sensors and body composition.38,39

References
  1. Thompson FE, et al. Register of dietary assessment calibration-validation studies: a status report. Am J Clin Nutr. 1997;65(4 suppl):1142S–1147S.
  2. Potischman N. Biologic and methodologic issues for nutritional biomarkers. J Nutr. 2003;133(suppl 3):875S–880S.
  3. Wild CP, et al. A critical evaluation of the application of biomarkers in epidemiological studies on diet and health. Br J Nutr. 2001;86(suppl 1):S37–S53.
  4. Moshfegh AJ, et al. The US Department of Agriculture Automated Multiple-Pass Method reduces bias in the collection of energy intakes. Am J Clin Nutr. 2008;88(2):324–332.
  5. Slimani N, et al. The standardized computerized 24-h dietary recall method EPIC-Soft adapted for pan-European dietary monitoring. Eur J Clin Nutr. 2011;65(suppl 1):S5–S15.
  6. Kikunaga S, et al. The application of a handheld personal digital assistant with camera and mobile phone card (Wellnavi) to the general population in a dietary survey. J Nutr Sci Vitaminol (Tokyo). 2007;53(2):109–116.
  7. Rollo ME, et al. Trial of a mobile phone method for recording dietary intake in adults with type 2 diabetes: evaluation and implications for future applications. J Telemed Telecare. 2011;17(6):318–323.
  8. Pendergast FJ, et al. Evaluation of a smartphone food diary application using objectively measured energy expenditure. Int J Behav Nutr Phys Act. 2017;14(1):30.
  9. Williamson DA, et al. Comparison of digital photography to weighed and visual estimation of portion sizes. J Am Diet Assoc. 2003;103(9):1139–1145.
  10. Martin CK, et al. Measuring food intake with digital photography. J Hum Nutr Diet. 2014;27(suppl 1):72–81.
  11. Sharp DB, Allman-Farinelli M. Feasibility and validity of mobile phones to assess dietary intake. Nutrition. 2014;30(11–12):257–266.
  12. Ngo J, et al. A review of the use of information and communication technologies for dietary assessment. Br J Nutr. 2009;101(suppl 2):S102–S112.
  13. Pettitt C, et al. A pilot study to determine whether using a lightweight, wearable micro-camera improves dietary assessment accuracy and offers information on macronutrients and eating rate. Br J Nutr. 2016;115(1):160–167.
  14. Sun M, et al. A wearable electronic system for objective dietary assessment. J Am Diet Assoc. 2010;110(1):45–47.
  15. Sazonov ES, Fontana JM. A sensor system for automatic detection of food intake through non-invasive monitoring of chewing. IEEE Sens J. 2012;12(5):1340–1348.
  16. Shim JS, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. Epidemiol Health. 2014;36:e2014009.
  17. Long JD, et al. Evidence review of technology and dietary assessment. Worldviews Evid Based Nurs. 2010;7(4):191–204.
  18. Jung HJ, Lee SE, Kim D, et al. Development and feasibility of a web-based program “Diet Evaluation System (DES)” in urban and community nutrition survey in Korea. Korean J Health Promot. 2013;13:107–115.
  19. Sylvia LG, et al. Practical guide to measuring physical activity. J Acad Nutr Diet. 2014;114(2):199–208.
  20. Ainsworth B, et al. The current state of physical activity assessment tools. Prog Cardiovasc Dis. 2015;57(4):387–395.
  21. Diaz KM, et al. Fitbit(R): an accurate and reliable device for wireless physical activity tracking. Int J Cardiol. 2015;185:138–140.
  22. Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of consumer-wearable activity trackers. Int J Behav Nutr Phys Act. 2015;12:159.
  23. Evenson KR, Wen F, Furberg RD. Assessing validity of the Fitbit indicators for U.S. public health surveillance. Am J Prev Med. 2017 Jul 26. Epub ahead of print.
  24. Huang Y, et al. Validity of FitBit, Jawbone UP, Nike+ and other wearable devices for level and stair walking. Gait Posture. 2016;48:36–41.
  25. Price K, et al. Validation of the Fitbit One, Garmin Vivofit and Jawbone UP activity tracker in estimation of energy expenditure during treadmill walking and running. J Med Eng Technol. 2017;41(3):208–215.
  26. Barnett LM, et al. Active gaming as a mechanism to promote physical activity and fundamental movement skill in children. Front Public Health. 2013;1:74.
  27. Chang DC, Piaggi P, Krakoff J. A novel approach to predict 24-hour energy expenditure based on hematologic volumes: development and validation of models comparable to Mifflin-St Jeor and body composition models. J Acad Nutr Diet. 2017;117(8):1177–1187.
  28. O’Connor S, McCaffrey N, Whyte E, Moaran K. The novel use of a SenseCam and accelerometer to validate training load and training information in a self-recall training diary. J Sports Sci. 2016;34(4):303–310.
  29. Toledo MJ, et al. Validation of a smartphone app for the assessment of sedentary and active behaviors. JMIR Mhealth Uhealth. 2017;5(8):e119.
  30. Bort-Roig J, et al. Monitoring sedentary patterns in office employees: validity of an m-health tool (Walk@Work-App) for occupational health. Gac Sanit. 2017 Sep 27. Epub ahead of print.
  31. Rye Hanton C, et al. Mobile phone-based measures of activity, step count, and gait speed: results from a study of older ambulatory adults in a naturalistic setting. JMIR Mhealth Uhealth. 2017;5(10):e104.
  32. Bort-Roig J, et al. Measuring and influencing physical activity with smartphone technology: a systematic review. Sports Med. 2014;44(5):671–686.
  33. Dunton GF, et al. Development of a smartphone application to measure physical activity using sensor-assisted self-report. Front Public Health. 2014;2:12.
  34. Hurvitz PM, et al. Emerging technologies for assessing physical activity behaviors in space and time. Front Public Health. 2014;2:2.
  35. Schipperijn J, et al. Dynamic accuracy of GPS receivers for use in health research: a novel method to assess GPS accuracy in real-world settings. Front Public Health. 2014;2:21.
  36. Ellis K, et al. Identifying active travel behaviors in challenging environments using GPS, accelerometers, and machine learning algorithms. Front Public Health. 2014;2:36.
  37. Wells JC, Fewtrell MS. Measuring body composition. Arch Dis Child. 2006;91(7):612–617.
  38. ChiriTa-Emandi A, et al. A novel method for measuring subcutaneous adipose tissue using ultrasound in children - interobserver consistency. Rom J Morphol Embryol. 2017;58(1):115–123.
  39. Steinberg DM, et al. The efficacy of a daily self-weighing weight loss intervention using smart scales and e-mail. Obesity (Silver Spring). 2013;21(9):1789–1797.

Stephen P. Fortmann, M.D.

Stephen P. Fortmann, M.D. Stephen P. Fortmann, M.D., is an internist and epidemiologist who conducts research on heart disease prevention. Dr. Fortmann received his M.D. from the University of California, San Francisco and postdoctoral training in cardiovascular disease prevention at Stanford University. From 1979 to 2010, Dr. Fortmann was a member of the Stanford University Medical School faculty and Director of the Stanford Prevention Research Center (1998–2010) and the C.F. Rehnborg Professor in Disease Prevention (1999–2010, now Emeritus). In 2010, Dr. Fortmann moved to the Kaiser Permanente Center for Health Research in Portland, Oregon, where he is a Senior Investigator and, since 2017, Senior Director, Science Programs. Dr. Fortmann has conducted both population- and individual-level studies of cardiovascular risk factors and disease rates, smoking cessation, the influence of tobacco marketing on adolescent smoking rates, and exercise and diet change. He has been principal or co-principal investigator on 36 research grants and has published over 210 peer-reviewed articles. Currently, he is studying the health care cost impact of a new light rail line in Portland, cardiovascular disease and diabetes in Asians and Pacific Islanders in Hawaii and California, and the comparative effectiveness of four second-line drugs for treating diabetes, among other studies.

Linking Health System, Environmental, and Contextual Data for Evaluation of Natural Experiments

Background

Overweight and obesity prevalence has increased substantially over the past four decades. The rapid rise in prevalence suggests environmental factors—physical and social—have played a key role. This recognition has stimulated interest in population-level health impacts of transportation and urban design characteristics that promote active travel to increase physical activity (PA) and potentially reduce obesity. Most data on this relationship are cross-sectional; natural experiments provide more rigorous tests of the potential for a causal connection. Several natural experiments have or are examining changes in PA, but none examine changes in health, health care, or costs. One way to address such outcomes is to link electronic health records (EHR) to environmental and contextual data external to the health care system.

Brief Content Summary

The Rails & Health Study in Portland, Oregon, is using EHR data from the northwest region of Kaiser Permanente to evaluate the impact of a new light rail transit line on health and health care cost data; this requires linkage of EHR and environmental data. The Kaiser Permanente Center for Health Research has a long-standing, well-developed research data warehouse (RDW) that includes continuously updated and curated data derived from the clinical EHRs. As an integrated health care system, the RDW includes data on hospital and ambulatory encounters, diagnoses, orders, medication dispensing (pharmacies are in-house), laboratory measures, and imaging procedures. Some hospitalizations occur at non-Kaiser Permanente hospitals for which claims data are available. The Kaiser Permanente EHR, based on Epic®, is also connected to other Epic® EHRs in the region through CareEverywhere. Kaiser Permanente has established links with external data sources, such as state vital statistics records; U.S. census data on socio-economic and demographic variables; Environmental Protection Agency data on air pollution, other toxins, walkability, greenspace, and transit; U.S. Department of Agriculture data on food accessibility; and national market research data on consumer segmentation.1 These data are obtained by using Kaiser Permanente members’ home addresses and are aggregated to the census-tract level for presentation. For the Rails & Health Study, we are collecting more fine-grained data by calculating the built environment measures for a 1-kilometer road network buffer around each participant’s individual address, as well as additional measures such as the pedestrian environment near the home and crime. We are also collecting individual PA data by accelerometry and travel by GPS and travel diary on a subset of participants.

Discussion

Using these data to evaluate a natural experiment presents a number of challenges. The address is key and must be maintained over time (not overwritten when it changes). People move frequently, especially at younger ages. EHR records include the home but not the work address, which is likely important for behavior and is understudied. The EHR is designed to support clinical care and billing, not research; to use these data for research requires considerable curation, which tends to take many years of development. The EHR contains very few patient-reported data as structured variables, including income, education, diet, PA, transit and car use, neighborhood perceptions, depression symptoms, quality of life, social relationships, and food insecurity. Many health care systems lack encounter-level data, especially for hospitalizations, and must rely on claims, which provide limited data for outcome validation and lack many other details.

Available public data also have limitations. They may be available only at the census- tract level, and even block group-level data may be too coarse; air pollution measures are often at even lower density than the census tract; there may be restrictions on availability (including vital statistics in some places); these data are generally not timely.

For proximity-based measures, one must decide where something is. While the address is a reasonable measure of the site of a home, this is not true for schools or for larger worksites. Transit stops are also reasonably represented as points, but how far does their influence extend and what might affect that extent (e.g., pedestrian and bike infrastructure)?

For natural experiments, another challenge is how to find controls. Should they be matched to “intervention” participants at the individual or neighborhood level? What variables are best to use in matching? How best should we define a neighborhood?

Key Recommendations (Draft)
  • More longitudinal, natural experiments to improve causal inference and to measure more distal variables than physical activity; linking EHR to environmental data is a way to address this issue.
  • What defines a neighborhood? How can census and other public data be used to define neighborhoods?
  • How should natural experiments of environmental influences select controls?
Finer grained (spatial and/or temporal) contextual data
  • Detecting neighborhood change via machine learning on Google Street View images.2
  • High-resolution air pollution mapping via Street View cars or sensors.3,4
Detecting changes in life situation
  • Moving
  • Householding
Integration into clinical workflows
  • Improve efficiency of self-reporting to EHR and comparison to population-level survey measures (e.g., census).
  • Prioritize screening outreach based on inferred risk:
    • Food insecurity
    • Childhood lead levels
    • Lifetime exposure to environmental pollutants.
References:
  1. Clift K, Scott L, Johnson M, Gonzalez C. Leveraging geographic information systems in an integrated health care delivery organization. Perm J. 2014;18(2):71–75. doi 10.7812/TPP/13-097 PMC4022561 PMID:24694317
  2. Naik N, Kominers SD, Raskar R, Glaeser EL, Hidalgo CA. Computer vision uncovers predictors of physical urban change. Proc Natl Acad Sci U S A. 2017;114:7571–7576. http://streetchange.media.mit.edu/static/pdf/NaikEtAl_Streetchange_PNAS_2017_LowRes.pdf (PDF - 1.36 MB) . Accessed October 18, 2017.
  3. Apte JS, Messier KP, Gani S, Brauer M, Kirchstetter TW, Lunden MM. High-resolution mapping with Google Street View cars: exploiting big data. Environ Sci Technol. 2017;51:6999–7008. http://pubs.acs.org/doi/abs/10.1021/acs.est.7b00891 . Accessed October 18, 2017.
  4. Vlasits A. A race to develop pollution sensing tech plays out in Oakland. Wired. June 5, 2017. https://www.wired.com/2017/06/race-pollution-sensing-tech-oakland/ . Accessed October 18, 2017.

Christopher L. Fulcher, Ph.D.

Christopher L. Fulcher, Ph.D. Christopher L. Fulcher, Ph.D., serves as the Director of the Center for Applied Research and Engagement Systems at the University of Missouri. Dr. Fulcher and his team integrate GIS, data visualization, community engagement tools, and Internet accessibility to better serve vulnerable and underserved populations. These web-based technologies help organizations and policymakers make more informed decisions about access, equity, and allocation of resources. Dr. Fulcher’s systems-based approach to decision-making enables public and non-profit-sector organizations to effectively address social, economic, environmental, and public health issues using unique engagement systems such as Community Commons (http://www.communitycommons.org). Dr. Fulcher received his B.S. in agricultural engineering at Texas A&M University in 1984 and his M.S. in agricultural economics at Texas A&M in 1985. He received his Ph.D. in agricultural economics at the University of Missouri in 1996. In 2005, he completed his National Library of Medicine Postdoctoral Fellowship in Health Informatics.

Challenges and Opportunities in Using and Sharing Existing Data Resources

The goal of my presentation is to highlight ways people can visualize and interact with secondary data on a platform hosted by CARES at the University of Missouri. Specifically, my presentation will address the following Key Question: “What population-based data sources have been used in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes?”

Background

The CARES team uses and shares existing data resources (over 15,000 publicly available GIS data layers) by developing and implementing mapping, reporting, and engagement systems for social-sector organizations and policymakers. These organizations, including place-based multisector collaboratives, can make more informed decisions about access, equity, and allocation of resources by leveraging these available social determinants of health data, including those data that may relate to the obesity epidemic. Specifically, users can visualize and interact with these data (e.g., socioeconomic, demographic, health, jurisdictional, political, environmental, and infrastructure data) using engagement systems such as Community Commons (http://www.communitycommons.org) .

Brief Content Summary

Community Commons, maintained through a partnership between CARES, the Institute for People, Place and Possibility, and Community Initiatives, was launched in 2011 to serve as an interactive mapping, networking, and learning utility for the broad-based healthy, sustainable, livable communities movement. This national public good website provides easy-to-use, democratized access to collaboration tools, data, maps, reports, and stories that support collective impact for the health of people and places. Community Commons adopts a user-centric design approach that provides people with individual profiles and a coherent navigation structure to access social media, stories, tools, and functionality. The primary audiences include communities (civic leaders, municipalities/agencies, and multisector collaborations), funders (public and private), and intermediary organizations (policy, technical assistance, and evaluation organizations).

There are several distinguishing features of the Commons model. With a user-centric design in place, all users—regardless of their level of access—have a consistent, coherent navigation structure in place and may directly benefit from new tools and functionality generated from other public good grants and contracts. The Commons provides a larger and growing audience (over 55,000 members) ways to share data, maps, stories, policies, and related content that would not otherwise be available via disconnected systems with smaller, content-specific audiences. The traffic generated through Community Commons can be directed to external organization websites through the stories and related content generated via their individual profile or hub. This growing audience base, supported in part by aligned investments, advances the healthy communities movement.

Discussion

Place-based multisector collaboratives have varying resources and competencies for acquiring data and making them available to their communities or regions. Rather than focusing on the limitations of data availability across communities, states, and regions, our center uses a “patchwork quilt” data acquisition approach by integrating data from federal, state, and local agencies, and non-profit organizations. Because data are continuously being created and updated, our center integrates data on an ongoing basis and focuses on how the information can be used for decision support at the community, regional, and national levels. CARES capabilities enable data users to (1) geographically visualize community, regional, and national-level data; (2) integrate new spatial data and overlay these data to conduct location-specific analyses (functionality limited to funded multisector efforts); and (3) generate maps, dynamic reports, and “what if” scenarios that utilize the integrated nature of these information systems.


Rachel Gold, Ph.D., M.P.H.

Rachel Gold, Ph.D., M.P.H. Rachel Gold, Ph.D., M.P.H., is a Health Services Researcher with a joint appointment as an Investigator at the Kaiser Permanente Northwest Center for Health Research and the Lead Research Scientist at OCHIN, Inc. Her research involves developing and implementing diverse health information technology approaches to improving health care quality and outcomes, and reducing health disparities, in the vulnerable populations served by safety net community health centers (CHC). She has been the principal investigator (PI) of several National Heart, Lung, and Blood Institute-funded studies in this area, with a focus on improved cardiovascular care among patients with diabetes and the implementation approaches needed to support related practice changes in CHCs. She is also the PI of the nation’s first study on implementation of social determinants of health data collection in CHCs.

Using Electronic Health Record Tools To Collect, Summarize, and Take Action on Patients’ Social Determinants of Health Data, in the Context of Diabetes Care: Lessons and Challenges

Little is known about how to use electronic health records and other information technology strategies to help primary care teams collect, review, and take action on the social determinants of health (SDH) impacting patients’ obesity-related health, overall and in the context of diabetes care. This talk will present results of a recent pilot study that developed a suite of SDH data tools for use in primary care safety net community health centers. These results will be considered in the context of current initiatives exploring how diverse health care settings can identify and intervene to address patients’ SDH, including an overview of the literature on the impact of SDH screening and interventions, and known barriers to such efforts. It will also discuss the many unanswered research questions related to how SDH information can be integrated into primary care for patients with, or at risk for, obesity, which must be answered to ensure that patients’ social circumstances are considered and addressed by their care teams.


Steven L. Gortmaker, Ph.D.

Steven L. Gortmaker, Ph.D. Steven L. Gortmaker, Ph.D., is Professor of the Practice of Health Sociology at the Harvard T.H. Chan School of Public Health. He directs the Harvard T.H. Chan School of Public Health Prevention Research Center and serves as a senior advisor to the Healthy Eating Research Program of the Robert Wood Johnson Foundation. Current activities include continuing implementation of the school curriculums Planet Health and Eat Well and Keep Moving; the afterschool curriculum co-developed with the YMCA of the USA–Food and Fun; and the Out of School Nutrition and Physical Activity Initiative (OSNAP). Research includes the CHOICES project, funded by the JPB Foundation, which is evaluating the cost-effectiveness, population reach, and impact of more than 40 childhood obesity preventive strategies. Dr. Gortmaker has been an author/co-author of more than 210 published research articles, including the first report in the United States concerning the obesity epidemic among children, the first longitudinal study linking increases in sugar-sweetened beverage intake to increased obesity incidence in youth, the recent four-paper obesity modeling series in the Lancet, and recent cost-effectiveness papers in Health Affairs and Preventive Medicine.

Effectiveness, Costs, and Cost-Effectiveness: Improving Translation of Research to Policy and Practice

Background

Estimates of the effects of programs, policies, and built environments on obesity prevalence are just one important focus for translation of obesity prevention research into practice. Our work emphasizes the use of estimates of effect, but we also utilize estimates of population reach, cost data, and microsimulation modeling to make 10-year projections of population impact, cost-effectiveness, and impact on health equity of different intervention strategies. We thus agree that randomized trials and quasi and natural experimental studies can provide important data. At the same time, our experience is that decision-makers also want to know the cost of different strategies, the population reach (whether a few individuals or a large part of the population is affected), the population impact (e.g., whether it has a small effect on obesity cases or a substantial impact on obesity prevalence), the cost-effectiveness (value for money), what will happen in the future (we use a 10-year time frame for many of our projections), and the implications for health equity. We have found that microsimulation modeling, including careful attention to estimating uncertainty, can provide reasonable answers to all these questions, building on the best available estimates of effect, population reach, cost, and implementation.

Brief Content Summary

We describe CHOICES (Childhood Obesity Intervention Cost Effectiveness Study).1-7 This work includes:

  • Identification of more than 40 programmatic and policy obesity interventions for evaluation
  • Development of logic models for the interventions1
  • Systematic evidence reviews of effect (more than 130,000 peer-reviewed studies); we prioritize evidence of effect from randomized trials, quasi and natural experiments, and studies with body mass index (BMI), BMI z-score, and obesity outcomes1,8
  • Development of realistic growth trajectories of childhood obesity into adulthood4
  • Development of a microsimulation model to project future obesity rates and disparities by race/ethnicity and income, using growth trajectories, U.S. Census, American Community Survey, Behavioral Risk Factor Surveillance System, and National Health and Nutrition Examination Surveys9-12
  • Estimates of the impact of different strategies on prevented cases of obesity over the next decade, health care costs, and cost-effectiveness, including uncertainty intervals.1,2,5

Our development of realistic growth trajectories of BMI throughout the life course builds on the predictability of weight gain/loss. Obesity in childhood and adolescence—and particularly severe obesity—tracks strongly into adulthood. Excess weight gain often continues in adulthood.4

Results from our microsimulation models of policy and programmatic interventions, however, also indicate there are a number of strategies where substantial prevention of obesity prevalence can occur. For some of these strategies, health care cost savings are projected to be more than the interventions cost to implement.1 In these cases, health care cost savings are projected because of an expected slower growth in obesity with intervention, and because health care costs are higher for individuals with obesity. While there are few such cost-saving strategies, many other strategies show good evidence for cost-effectiveness.1,2,5

We find that simulation modeling is an effective method to use in predicting the future impact of different strategies on population health, and to estimate cost-effectiveness, while accounting for the uncertainty in estimates. This approach also provides estimates of the potential impact on health equity.

Discussion

We agree with the importance of establishing the validity of effect estimates from quasi and natural experimental studies, and in developing approaches to strengthen these estimates. While randomized trials are generally preferred, quasi and natural experiments, particularly those with repeated pre- and post-measures, have strong potential for generation of valid effect estimates of policies and programs to prevent obesity.

But better estimates of effect are just one of the critical pieces of data needed to effectively translate research results into practice. We must provide estimates of effect, population reach, cost, cost-effectiveness, and equity as well as realistic models that project this impact into the future if we are to provide decision-makers with the information they need.

Recommendations
  1. Estimates of effect of policy and programmatic interventions are but one important piece of data needed to translate research into practice: we also need estimates of population impact, costs, and cost-effectiveness of the different strategies being considered if we are to provide decision-makers with the data they need.
  2. Because estimates of cost are not often collected in many intervention studies, and because these data are generally not expensive to gather, we recommend their inclusion in funded intervention trials.
  3. Simulation modeling is one useful strategy to use in projecting future estimates of population impact, costs, and cost-effectiveness. There are useful guidelines for cost-effectiveness modeling.13
References
  1. Gortmaker SL, Wang YC, Long MW, et al. Three interventions that reduce childhood obesity are projected to save more than they cost to implement. Health Affairs. 2015;34(11):1932–1939.
  2. Cradock AL, Barrett JL, Kenney EL, et al. Using cost-effectiveness analysis to prioritize policy and programmatic approaches to physical activity promotion and obesity prevention in childhood. Prev Med. 2017;95:S17–S27.
  3. Dietz WH, Gortmaker SL. New strategies to prioritize nutrition, physical activity, and obesity interventions. Am J Prev Med. 2016;51(5):E145–E150.
  4. Ward Z, Long M, Resch S, Giles C, Cradock A, Gortmaker S. Predicting adult impact of childhood obesity – growth trajectory simulation. In press.
  5. Sharifi M, Franz C, Horan CM, et al. Cost-effectiveness of a clinical childhood obesity intervention. Pediatrics. In press.
  6. Long MW, Ward Z, Resch SC, et al. State-level estimates of childhood obesity prevalence in the United States corrected for report bias. Int J Obes. 2016;40(10):1523–1528.
  7. Ward ZJ, Long MW, Resch SC, et al. Redrawing the US obesity landscape: bias-corrected estimates of state-specific adult obesity prevalence. PLoS One. 2016;11(3):e0150735.
  8. Gortmaker SL, Long MW, Resch SC, et al. Cost effectiveness of childhood obesity interventions: evidence and methods for CHOICES. Am J Prev Med. 2015;49(1):102–111.
  9. Gortmaker S. Reversing disparities in obesity: cost effective strategies to promote health equity. Biennial Childhood Obesity Conference; 2017; San Diego, CA.
  10. Gortmaker S. Using cost effectiveness analysis to evaluate the effect of policies on childhood obesity rates and racial/ethnic and income disparities. American Public Health Association; 2016; Denver, CO.
  11. Long M, Hsiao A, Giles C, et al. Reducing racial/ethnic disparities in childhood obesity prevalence by removing SNAP eligibility of sugar-sweetened beverage purchases. American Public Health Association; 2016; Denver, CO.
  12. Barrett J, Cradock A, Ward Z, et al. Impact of the federal Smart Snacks in School regulation on racial/ethnic disparities in childhood obesity and healthcare costs. American Public Health Association; 2016; Denver CO.
  13. Caro JJ, Briggs AH, Siebert U, Kuntz KM, Pract I-SMGR. Modeling Good Research Practices-overview: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Medical Decision Making. 2012;32(5):667–677.

Ross A. Hammond, Ph.D.

Ross A. Hammond, Ph.D. Ross A. Hammond, Ph.D., is a Senior Fellow in Economic Studies at the Brookings Institution, where he is also Director of the Center on Social Dynamics and Policy. His primary area of expertise is modeling complex dynamics in economic, social, and public health systems using methods from complexity science. His current research topics include obesity etiology and prevention, food systems, tobacco control, behavioral epidemiology, health disparities, childhood literacy, crime, corruption, segregation, and decision-making. Dr. Hammond received his B.A. from Williams College and his Ph.D. from the University of Michigan. He has authored numerous scientific articles in prominent journals such as Lancet, JAMA Pediatrics, American Journal of Public Health, PNAS, Evolution, and Journal of Conflict Resolution, and his work has been featured in The Atlantic Monthly, New Scientist, Salon, Scientific American, and major news media.


Christine Hunter, Ph.D., M.A., ABPP

Christine Hunter, Ph.D., M.A., ABPP Christine Hunter, Ph.D., M.A., ABPP, is the Deputy Director for the Office of Behavioral and Social Sciences Research (OBSSR) at the National Institutes of Health. In that role, she supports the OBSSR mission to enhance the impact of health-related behavioral and social sciences research. Dr. Hunter previously served as the Director of Behavioral Research at the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) where she built a strong and vibrant behavioral science research community focused on obesity and diabetes prevention and treatment research. Dr. Hunter obtained her Ph.D. in clinical psychology from The University of Memphis and completed her internship at Wilford Hall Medical Center. She completed a postdoctoral fellowship in clinical health psychology and achieved board certification in clinical health psychology. Prior to joining the NIDDK as a Public Health Service officer, she was an officer in the U.S. Air Force and served in a variety of clinical, research, and policy positions. Dr. Hunter’s research interests include application of rigorous but varied methods and designs in the behavioral and social sciences. She is also interested in implementation science to more rapidly advance the reach, uptake, adaptation, and scale up of effective approaches to improve health into routine settings and care.


Michael Jerrett, Ph.D., M.A.

Michael Jerrett, Ph.D., M.A. Michael Jerrett, Ph.D., M.A., is an internationally recognized expert in geographic information science (GIS) for exposure assessment, environmental epidemiology, and health geography. He is Professor and Chair of the Department of Environmental Health Sciences in the Fielding School of Public Health, University of California, Los Angeles, and a Professor In-Residence in the Division of Environmental Health Sciences, University of California, Berkeley. For the past 22 years, Dr. Jerrett has researched how to characterize population exposures to air pollution and built environment variables, the social distribution of these exposures among different groups (e.g., poor vs. wealthy), and how to assess the health effects from environmental exposures. Over the past 15 years, Dr. Jerrett has also studied how built and natural environments affect health, particularly physical activity and obesity. He has published some of the most widely cited papers in the fields of exposure assessment and environmental epidemiology in leading journals, including The New England Journal of Medicine, The Lancet, Nature, and Proceedings of the National Academy of Science of the United States of America. In 2009, the U.S. National Academy of Science appointed him to the Committee on “Future of Human and Environmental Exposure Science in the 21st Century.” The committee published a report titled Exposure Science in the 21st Century: A Vision and a Strategy. Dr. Jerrett led the chapter covering scientific and technological advances, with a focus on sensors, GIS, and satellite remote sensing. In 2014, 2015, and 2016, Dr. Jerrett was named to the Thomson Reuters List of Highly Cited Researchers, indicating he is in the top 1% of all authors in the fields of environment/ecology in terms of citation by other researchers. Dr. Jerrett earned a B.Sc. in environmental and resource science from Trent University, and an M.A. in political science with accredited specialization in environmental studies and a Ph.D. in geography, both from the University of Toronto.

Linking Environmental and Health Data: National and International Examples and Challenges

Background

Numerous aspects of the built environment can affect physical activity behavior and consequent obesity status. Public transit access, walking and biking infrastructure, and connectivity between different types of land use such as residential and commercial may all contribute to increased physical activity by encouraging utilitarian active travel by foot or bicycle. Improved access to parks and green spaces may also increase leisure-time physical activity, which can also reduce obesity. Many of the existing studies on the built environment are cross-sectional, which leads to questions of reverse causality; specifically, whether active individuals and families self-select into neighborhoods that support their already healthy lifestyles. To overcome this critical limitation, researchers have begun evaluating natural experiments in the built environment that address this self-selection bias by assessing whether infrastructure or policy change may directly affect physical activity behavior and obesity before and after implementation. While these studies have the potential to strengthen causal inference, they also require higher quality environmental data support to assess physical activity and obesity outcomes.

Assessments of physical activity increasingly rely on real-time information from accelerometers and Global Positioning System trackers, collected from stand-alone instruments or cell phones. These momentary, dynamic assessments of physical activity outcomes increase the complexity of linkage to environmental data by necessitating high-resolution estimates of environmental conditions that vary considerably over small areas and short time periods.

As a result, evaluations of natural experiments in the built environment depend crucially on the quality of environmental data that can be linked to health data. Linkages can be divided into two broad classes: those environmental changes related to the natural experiment that become the key exposure of interest, and those that may influence confounding bias, either through time variations that are coincident with the intervention or with spatial control for the contrast control groups.

This presentation will review how to link environmental and health data for evaluating natural experiments in the built environment. With respect to the primary intervention exposure, changes to the environment are often multifaceted; for example, residents moving into “smart growth” communities versus those living in conventional suburbs will encounter numerous facilities such as improved pedestrian infrastructure, shorter distances to key destinations such as schools, and better access to parks and recreation facilities, all of which may influence physical activity (Jerrett et al., 2013). Other changes such as infill development and new commercial land uses may also follow the initial intervention. These environmental changes are often complex and happen incrementally through the period of evaluation. Disentangling these complex spatiotemporal changes requires ongoing surveillance of environmental conditions to understand how the combination of these changes likely affects physical activity behavior and obesity.

Brief Content Summary

Tracking physical activity changes that may result from the intervention represents another important challenge. Existing studies have typically equipped study participants with accelerometers and Global Positioning Systems for limited periods of time before and after the intervention (Almanza et al., 2012; Jerrett et al., 2013) with large potential for bias depending on weather conditions, time of the year, and the limited sampling of the total time budget for a study participant. Some studies have demonstrated the potential of cell phone applications to collect research-grade information on physical activity and geographic position (Donaire et al., 2013; Donaire et al., 2016), but ideally, data should be collected on widely available software that minimizes participant burden. For example, several routine applications on cell phones can be used to monitor physical activity over longer periods of time. One example is the MOVES smartphone app, which assesses trip mode (walk, bike, other transport), geographic position, steps taken, and calories expended. MOVES is currently being used in the Physical Activity Through Sustainable Transport Approaches (PASTA; http://www.pastaproject.eu/home/) project (in conjunction with web-based ongoing surveys in seven European countries and in Los Angeles) to track physical activity in relation to built environment changes that can influence active transport and physical activity.

The PASTA study also will be used to illustrate the value of having ongoing cohorts that measure travel behavior, physical activity, and potentially adverse consequences such as traffic crash and air pollution risks (Götschi et al., 2017; Dons et al., 2015). By enrolling 2,000 participants in seven European cities, the PASTA study has amassed critical information on normal travel behaviors in each city. Such ongoing cohorts can serve as control groups for intervention evaluation or as sample frames for opportunistic assessment of natural experiments.

New developments in the use of public web cameras and private-sector data such as Google omnidirectional Streetview imagery offer promise for improved assessment of these changes resulting from the primary intervention (Hipp et al., 2016; Charreire et al., 2014). The routine collection of video and photo images enhances opportunities to track complex changes in the built environment and potentially physical activity behaviors on an ongoing basis.

Remotely sensed images represent another important advance for linking the built environment to health data. Many images now have incredibly high resolution for assessing built environment features. For example, in assessing green spaces, new images have dramatically increased spatial resolution that are now available at 2 m resolution. The EnviroAtlas developed by the U.S. Environmental Protection Agency now has data on 18 metropolitan areas with rendered aerial photography images that classify land use and cover such as green spaces at 1 m resolution (https://www.epa.gov/enviroatlas). Numerous other useful data on transportation and land use are available nationally, and many more detailed community-level assessments are in development. These fine-scale assessments offer the promise of better characterizing environmental influences on travel and leisure physical activity behavior.

Finally, as another example, the Canadian Urban Environmental Health network (CANUE; http://canue.ca/) is being established with the goal of amassing high-quality environmental data for linkage to numerous health datasets to create huge cohorts for evaluating relationships between the urban environment and health. This network will develop estimates of adverse exposures such as air pollution and noise and positive exposures such as green space and walkability. CANUE will develop the environmental estimates at a fine scale for each Canadian postal code, which is essentially the midpoint of a city block. The ongoing linkages with large cohorts such as the Canadian Census Health and Environment Cohort, which now has more than 16 million adults enrolled with linkages to tax files and health data offers huge potential for selecting samples for natural experiment evaluation.

Discussion

In summary, numerous novel sources of environmental and outcome data are becoming available, particularly from routine surveillance such as embedded camera sensors, crowd-sourced data, remote sensing, and data compilations supported by national and regional governments. Many of these sources offer the potential to improve substantially the research on the physical activity and obesity-reduction benefits of changes to the built environment and related policies.

Key Recommendations
  • Support large compilations of fine-resolution environmental data such as EnviroAtlas and other similar projects like CANUE to produce ongoing and readily available assessments of environmental conditions likely to influence physical activity and obesity.
  • Support and encourage stronger linkages to private-sector data suppliers like Google to enhance ongoing evaluations of environmental conditions, particularly in areas around the natural environment interventions.
  • Encourage the use of more publicly available data such as web camera photography to maximize use of existing resources for built environment research.
  • Support systematic development of cell phone applications that can measure physical activity and other environmental conditions, while systematically validating the information from these applications against current gold standard tools and methods.
  • Work with NASA and other remote-sensing product developers to establish standardized, high-resolution data for the assessment of built and natural environment features.
  • Develop methods for dealing with the big data that will come from numerous cell phone, embedded, and remote sensors.
References

Almanza E, Jerrett M, Dunton G, Seto E, Pentz MA. A study of community design, greenness, and physical activity in children using satellite, GPS and accelerometer data. Health Place. 2012;18(1):46–54.

Charreire H, Mackenbach JD, Ouasti M, et al. Using remote sensing to define environmental characteristics related to physical activity and dietary behaviours: a systematic review (the SPOTLIGHT project). Health Place. 2014 Jan;25:1–9. doi: 10.1016/j.healthplace.2013.09.017. Epub 2013 Oct 23. Review. PubMed PMID: 24211730.

Donaire-Gonzalez D, de Nazelle A, Seto E, Mendez M, Nieuwenhuijsen MJ, Jerrett M. Comparison of physical activity measures using mobile phone-based CalFit and Actigraph. J Med Internet Res. 2013;15:e111.

Donaire-Gonzalez D, Valentín A, de Nazelle A, et al. Benefits of mobile phone technology for personal environmental monitoring. J Med Internet Res. 2016;10:e126.

Dons E, Götschi T, Nieuwenhuijsen M, de Nazelle A, et al. Physical activity through sustainable transport approaches (PASTA): protocol for a multi-centre, longitudinal study. BMC Public Health. 2015;15:1126.

Götschi T, de Nazelle A, Brand C, Gerike R; PASTA Consortium. Towards a comprehensive conceptual framework of active travel behavior: a review and synthesis of published frameworks. Curr Environ Health Rep. 2017 Jul 13. doi: 10.1007/s40572-017-0149-9. Epub ahead of print. Review. PubMed PMID: 28707281; PubMed Central PMCID: PMC5591356.

Hipp JA, Manteiga A, Burgess A, Stylianou A, Pless RB. Webcams, crowdsourcing, and enhanced crosswalks: developing a novel method to analyze active transportation. Front Public Health. 2016;4:97. doi:10.3389/fpubh.2016.00097.

Jerrett M, Almanza E, Davies M, et al. 2013. Smart growth community design and physical activity in children. Am J Prev Med. 2013;45:386–392.


Hadi Kharrazi, M.D., Ph.D., M.H.I.

Hadi Kharrazi, M.D., Ph.D., M.H.I. Hadi Kharrazi, M.D., Ph.D., M.H.I., is a core faculty member of Health Policy and Management at the Johns Hopkins Bloomberg School of Public Health with a joint appointment at the Johns Hopkins School of Medicine. He is the Research Director of the Johns Hopkins Center for Population Health IT and serves on multiple national advisory boards and steering committees including the Public Health Informatics Working Group Executive Committee of the American Medical Informatics Association, the Steering Committee of the AcademyHealth’s Health IT Interest Group, and the U.S. Department of Health and Human Services Office of the National Coordinator’s Measurement Community of Practice. Dr. Kharrazi’s primary research interest is in population health informatics. His research focuses on the use of new data sources such as electronic health records (EHR) and advanced informatics/predictive methods such as deep learning in identifying high-risk subpopulations. Results of his research are often used in operational settings to better align clinical and social interventions and improve population outcomes while containing cost. Dr. Kharrazi is a senior clinical informatician with specialization in EHR platforms, health information exchange, and clinical decision support systems (CDSS). Dr. Kharrazi’s long-term research interest is in contextualizing CDSS in PHI platforms to be utilized at different health IT levels of managed care such as EHR platforms or consumer health informatics solutions. In line with this contextualization and funded through a National Library of Medicine grant, he has modified and regenerated electronic quality measures based on PHI-derived CDSS, and applied it to large and multi-institutional clinical datasets to develop a regional real-time population metrics dashboard.

Evidence-based Practice Center Presentation

Objective

Given the enormity of obesity as a public health problem, rigorous methodological approaches, including natural experiments, are needed to evaluate the effectiveness of policies and programs to prevent and control obesity. Our objective was to systematically review studies evaluating programs and policies addressing obesity prevention and control in terms of their population-based data sources, use of data linkages, measures reported, study designs, and analytic approaches.

Data Sources

We systematically searched PubMed, CINAHL, PsycINFO, and EconLit from 2000 to August 21, 2017, to identify all U.S. and non-U.S. studies of programs or policies targeting obesity prevention and control in people of all ages and in any setting.

Review Methods

Two independent reviewers screened abstracts and full-text articles. We required articles to address a program, policy, or built environment change and have a defined comparison or unexposed group. We used the Effective Public Health Practice Project (EPHPP) tool to rate studies for their risk of bias.

Results

In all, 295 studies were eligible for inclusion (189 U.S. and 106 non-U.S.); 157 (53%) were natural experiment studies. Studies reported 116 unique primary or secondary shareable data sources, of which 106 (71 U.S. and 35 non-U.S.) data sources met criteria for a data system and 26 of the 71 U.S. data systems were linked with a secondary data source other than the primary data source. Also, 112 studies reported childhood weight measures, 33 had adult weight measures, 152 had physical activity measures, and 149 had dietary measures. Most natural experiment studies were rated as having a “weak” global rating (i.e., high risk of bias), with 64% having a weak rating for handling of withdrawals and dropouts, 25% having a weak rating for study design, 40% having a weak rating for confounding, and 26% having a weak rating for data collection.

Conclusions

We identified a large number of studies and data sources that used a wide variety of outcome measures and analytic methods, often with substantial risk of bias. The findings reinforce the need for methodological and analytic advances that would strengthen efforts to improve obesity prevention and control.


Emily A. Knapp, M.H.S.

Emily A. Knapp, M.H.S. Emily A. Knapp, M.H.S., is a doctoral candidate in epidemiology and a predoctoral trainee with the National Institute of Diabetes, Digestive, and Kidney Diseases (NIDDK)-funded Clinical Research and Epidemiology in Diabetes and Endocrinology T32 training program at the Johns Hopkins Bloomberg School of Public Health. Her main research interests are place-based and policy determinants of obesity, with an emphasis on strengthening measurement of these risk factors. Her dissertation investigates the role of community economic insecurity on children’s weight-gain trajectories. Her past projects have explored measurement of the obesogenic built environment, the effect of neighborhood-level socioeconomic status on diabetes, and novel measures of financial stability. She previously received her M.H.S. in epidemiology from the Johns Hopkins Bloomberg School of Public Health.

Evidence-based Practice Center Presentation

Objective

Given the enormity of obesity as a public health problem, rigorous methodological approaches, including natural experiments, are needed to evaluate the effectiveness of policies and programs to prevent and control obesity. Our objective was to systematically review studies evaluating programs and policies addressing obesity prevention and control in terms of their population-based data sources, use of data linkages, measures reported, study designs, and analytic approaches.

Data Sources

We systematically searched PubMed, CINAHL, PsycINFO, and EconLit from 2000 to August 21, 2017, to identify all U.S. and non-U.S. studies of programs or policies targeting obesity prevention and control in people of all ages and in any setting.

Review Methods

Two independent reviewers screened abstracts and full-text articles. We required articles to address a program, policy, or built environment change and have a defined comparison or unexposed group. We used the Effective Public Health Practice Project (EPHPP) tool to rate studies for their risk of bias.

Results

In all, 295 studies were eligible for inclusion (189 U.S. and 106 non-U.S.); 157 (53%) were natural experiment studies. Studies reported 116 unique primary or secondary shareable data sources, of which 106 (71 U.S. and 35 non-U.S.) data sources met criteria for a data system and 26 of the 71 U.S. data systems were linked with a secondary data source other than the primary data source. Also, 112 studies reported childhood weight measures, 33 had adult weight measures, 152 had physical activity measures, and 149 had dietary measures. Most natural experiment studies were rated as having a “weak” global rating (i.e., high risk of bias), with 64% having a weak rating for handling of withdrawals and dropouts, 25% having a weak rating for study design, 40% having a weak rating for confounding, and 26% having a weak rating for data collection.

Conclusions

We identified a large number of studies and data sources that used a wide variety of outcome measures and analytic methods, often with substantial risk of bias. The findings reinforce the need for methodological and analytic advances that would strengthen efforts to improve obesity prevention and control.


Robin McKinnon, Ph.D., M.P.A.

Robin McKinnon, Ph.D., M.P.A. Robin McKinnon, Ph.D., M.P.A., is a Senior Advisor for Nutrition Policy at the U.S. Food and Drug Administration (FDA) Center for Food Safety and Applied Nutrition (CFSAN). Dr. McKinnon works to advance nutrition-related activities across CFSAN. Prior to joining the FDA, Dr. McKinnon was a Health Policy Specialist at the National Cancer Institute (NCI), National Institutes of Health. At NCI, Dr. McKinnon’s primary responsibilities were to lead initiatives to advance policy-relevant research on diet, obesity, and physical activity, and to manage a grant portfolio focused on policy and environmental interventions to improve population nutrition and physical activity behavior and weight outcomes. Dr. McKinnon has a Ph.D. in public policy and administration from the George Washington University and an M.P.A. from Harvard University.


Adetokunbo “Toks” Omishakin, M.U.R.P.

Adetokunbo �Toks� Omishakin, M.U.R.P. Adetokunbo “Toks” Omishakin, M.U.R.P., was appointed Assistant Commissioner and Chief of Environment and Planning at the Tennessee Department of Transportation (TDOT) in October of 2011. He is responsible for the bureau’s administrative and project budget that exceeds $300 million annually. He leads the activities of the divisions of Environmental Services, Long-Range Planning, and Multimodal Transportation Resources. In 2014, he was promoted to Deputy Commissioner at TDOT. Prior to joining TDOT, he served as the Director of Healthy Living Initiatives in the Office of Mayor Karl Dean in Nashville, Tennessee. There he led efforts to develop Metro Nashville’s Complete Streets Policy and helped establish a more balanced approach to transportation planning and design for the city. He also spearheaded the creation of two bicycle sharing programs (Nashville BCycle and Nashville Green Bikes) for the city. Mr. Omishakin has been a speaker and presenter at several national and international conferences, and his work has been published in the American Journal of Preventive Medicine and profiled in The Wall Street Journal, HBO documentaries, and Newsweek magazine. He was appointed by Governor Bill Haslam to the Board of the Tennessee Tombigbee Waterway Authority in 2012 and the Governor’s Rural Development Task Force in 2015. He also serves on the Board of Directors at America Walks and the Civil Engineering and Environment Advisory Board at Tennessee Tech University, and is Co-Chair of the Tennessee Regions Roundtable Network. He is an active member of the American Planning Association and the Institute of Transportation Engineers. He holds a Master of Urban and Regional Planning with concentrations in Transportation Planning and Urban Design from Jackson State University, and a Bachelor of Science in Engineering Technology from Mississippi Valley State University.


Lisa M. Powell, Ph.D.

Lisa M. Powell, Ph.D. Lisa M. Powell, Ph.D., is a Distinguished Professor and Director in the Division of Health Policy and Administration in the School of Public Health and Director of the Illinois Prevention Research Center in the Institute for Health Research and Policy at the University of Illinois at Chicago. Dr. Powell has extensive experience as an applied micro-economist in the empirical analysis of the effects of public policy on a series of behavioral outcomes. Much of her current research is on assessing the importance of economic and environmental factors (such as food prices; sugar-sweetened beverage [SSB] taxes; access to food stores, restaurants, and physical activity facilities; and exposure to advertising on television for food and beverage products) on food consumption and physical activity behaviors and as determinants of obesity, including related disparities. Her work has made substantial contributions to the evidence base for policymakers in the areas of SSB taxes and child-directed marketing. Dr. Powell’s research has been funded by Bloomberg Philanthropies, the Centers for Disease Control and Prevention, the National Institutes of Health, the Robert Wood Johnson Foundation, and the U.S. Department of Agriculture, and she serves on a number of national and international advisory committees.

Linking Economic and Behavioral Data for Evaluation of Policy and Retail Environments

Background

Limited access to food retail environments that offer healthy food at affordable prices and exposure to food and beverage marketing may play significant roles in impacting consumption patterns, diets, and risk of obesity (Story et al., 2008; Glanz et al., 2012). Further, differential environmental contexts have been identified in low-income and minority neighborhoods, which in turn, may contribute to racial disparities in obesity and related health outcomes (Larson et al., 2009). Improving access to healthy, affordable foods in underserved neighborhoods and reducing exposure to marketing for unhealthy food products, particularly those that are child directed, are major policy foci recommended by the Institute of Medicine and are part of the Centers for Disease Control and Prevention’s key Winnable Battles. Linking economic, environmental, and policy data to individual-level data provides an opportunity to generate an evidence base that can inform effective policy development, but such work is also subject to a number of challenges.

Brief Content Summary

In the absence of randomized control trials or quasi-experimental design studies to evaluate economic and environmental factors that can either support or deter certain behaviors, researchers often need to be able to link economic, environmental, and policy data to individual-level data using geographic or other contextual (i.e., school or school district) identifiers to generate an evidence base that can inform effective public health policy development. This presentation will provide an overview of the key challenges/limitations to undertaking such studies and highlight a number of important issues to take into consideration. Questions and issues with respect to various datasets and related research designs that will be addressed include:

  • Economic/contextual/environmental data:
    • What type of economic/contextual data are available?
    • What is available publically versus commercially?
    • What is the quality of such data?
    • What geocode level are they available at?
    • What are the frequency of measures and issues of seasonality?
    • What about audit tools for real-time data collection?
  • Individual-level data:
    • What individual-level datasets are available with geographic/school identifiers?
    • What level of geocode is available in publicly available datasets versus those available under confidential user agreements?
    • Longitudinal versus cross-sectional datasets? How do we establish causality?
    • Trade-offs with data that on the one hand provide the most proximate geocodes (i.e., addresses) such as medical records, but on the other hand, lack key measures on sociodemographic controls, particularly income.
    • Endogeneity of self-reported economic or contextual measures.
  • Census data:
    • Importance in terms of providing additional layers of important context.
Discussion

There are numerous challenges to undertaking studies that rely on linkages between economic/contextual and individual datasets that need to be carefully considered (Fleischhacker et al., 2013; Lebel et al., 2017). Primary data collection offers an alternative, but is costly and must be done in real time (e.g., The NEMS Tools; Bridging the Gap tools). Additional considerations are that even when longitudinal data are available, they do not account for time-varying unobserved heterogeneity, and medical records usually lack important control variables (Zenk et al., 2017). Quasi-experimental designs that evaluate public health policy initiatives can help mitigate some of these challenges. But these studies also require careful research designs including comparison sites and baseline data collection, preferably with multiple waves.

Key Recommendations
  • Use validation studies to inform choices of retail commercial datasets to improve validity.
  • Use Census data to adjust measures of access.
  • Use Census data to help mitigate bias due to omitted variable bias.
  • If possible, undertake primary data collection of store retail environments using validated tools.
  • Test sensitivity of impacts based on proximity of fixed geocode measures.
  • Test sensitivity of impacts based on adjustable geographic buffers.
  • Use longitudinal data where possible; at a minimum, with stacked cross-sectional models, use geographic fixed effects to account for unobservables.
  • Use quasi-experimental study designs when possible to evaluate public health policies.
References

Bridging the Gap Community Data Tools. http://www.bridgingthegapresearch.org/research/community_data/ . Accessed October 2017.

Fleischhacker SE, Evenson KR, Sharkey J, Pitts SBJ, Rodriguez DA. Validity of secondary retail food outlet data: a systematic review. Am J Prev Med. 2013;45(4):462–473.

Glanz K, Bader MDM, Iyer S. Retail grocery store marketing strategies and obesity: an integrative review. Am J Prev Med. 2012;42(5):503–512

Larson, NI, Story MT, Nelson MC. Neighborhood environments: disparities in access to healthy foods in the U.S. Am J Prev Med. 2009;36(1):74–81

Lebel A, Daepp MIG, Block JP, et al. Quantifying the foodscape: a systematic review and meta-analysis of the validity of commercially available business data. PLOS One. 2017. https://doi.org/10.1371/journal.pone.0174417 . Accessed October 2017.

Story M, Kaphingst KM, Robinson-O’Brien R, Glanz K. Creating healthy food and eating environments: policy and environmental approaches. Annu Rev Public Health. 2008;29:253–272.

The NEMS Tools. http://www.med.upenn.edu/nems/measures.shtml . Accessed October 2017.

Zenk SN, Tarlov E, Wing C, et al. Geographic accessibility of food outlets not associated with body mass index change among veterans, 2009–14. Health Affairs. 2017;36(8):1433–1442.


Lorrene Ritchie, Ph.D., R.D.

Lorrene Ritchie, Ph.D., R.D. Lorrene Ritchie, Ph.D., R.D., has devoted her 20-year career to synthesizing and conducting research to inform nutrition programs and policy. The goal of her work is the prevention of food insecurity, obesity, and chronic disease, and the promotion of health and wellness, with an emphasis on the youngest and most vulnerable populations and the federal nutrition assistance programs. She recently was a member of the Institute of Medicine Committee on Evaluating Progress of Obesity Prevention Efforts and on the Robert Wood Johnson Foundation Expert Panel to develop Feeding Guidelines for Infants and Toddlers. Current research projects include evaluation of the relationship of school-level programs and policies on student dietary intakes and weight status; the impact of nutrition policy and the Child and Adult Care Food Program in child care in California; the relationship of community programs and policies with child nutrition and weight status in California and nationwide; the impact of Women, Infants, and Children nutrition education on child feeding practices and how the feeding of young children age 0 to 5 years has evolved over time; and investigating how food insecurity relates to measures of dietary intake and health in children and college-age adults. Dr. Ritchie has an M.S. and Ph.D. in nutritional sciences from the University of California, Berkeley and is also a Registered Dietitian.

Assessing Community Programs, Policies, and Practices Related to Obesity Prevention

Background

Despite plateauing in the rate of increase, the prevalence of child obesity in the United States is at an all-time high (Ogden et al., 2014). Multi-sector, multi-component interventions are recommended to improve child nutrition and physical activity (IOM, 2012). While several multifaceted community interventions have shown success (Hoelscher et al., 2013; Bleich et al., 2013; Economos et al., 2013; Brennan et al., 2014), the types or combinations of community strategies most effective for preventing childhood obesity is unclear.

Brief Content Summary

Lessons learned will be summarized from several studies assessing community programs, policies, and practices targeting pediatric obesity prevention.

  1. The concept of “population dose” emerged from several ongoing evaluations of Kaiser Permanente’s Community Health Initiative (CHI) community-based interventions conducted in several U.S. states. Using a combination of key informant interviews and tracking of intervention progress, the population dose concept provides a way to describe and compare the relative strength of different community strategies as well as characterize combinations of efforts (Cheadle et al., 2012; Schwartz et al., 2015). Population dose is a function of two metrics: reach and strength. Reach involves capturing the number of people in the community likely to be exposed to a program or policy. Strength involves estimating the degree to which people are likely to change a health behavior as a result of being exposed to a community program or policy given the length of exposure and type of strategy. Relatively higher population dose occurs across a community when more people are reached more often for longer periods of time with high-impact strategies across multiple sectors. For CHI, dose estimates were calculated by grouping strategies into clusters by target behaviors. Then for each strategy in a behavioral outcome cluster, reach and strength were estimated. The doses of all strategies in a cluster targeting one behavioral outcome were added together to estimate overall population dose.
  2. The National Institutes of Health-funded Healthy Communities Study (HCS) involved collecting data in 2013–2015 on community programs and policies in 130 U.S. communities (John et al., 2015). Communities were selected through a combination of probability-based and purposive sampling to ensure inclusion of a diverse representation of children by oversampling low-income and ethnic minority communities. Up to 81 children in kindergarten through eighth grade were recruited in each community, for a total study sample of 5,138 participants. Up to 14 key informants from each community were identified by a snowball technique and interviewed on community efforts implemented over the prior 10 years. Key informant interviews were supplemented by content analysis of documents to identify and describe instances of community programs and policies. A protocol was used to further characterize each community program or policy by factors that allowed for the estimation of intensity (i.e., reach, duration, strategy), building on the population dose concepts as well as several additional scoring metrics to enable a better understanding of the community efforts (e.g., goal, sector, behavioral objective).
  3. The aim of the Robert Wood Johnson-funded California Healthy Kids Study (CHKS), conducted in 2015–2016, was to identify the combination of programs, policies, and practices at the school, school district, and community levels associated with positive body mass index (BMI) change. The study included 30 low-income schools (at least 50% of the student population eligible for free or reduced-price lunch) with 10-year improvements in BMI based on state-mandated FitnessGram® data and 30 comparison low-income schools where BMI had not improved. Observational and survey data were collected from each school on current school nutrition and physical activity environments. These data were augmented with key informant interview data collected to assess the school nutrition and physical activity environment over the 2005–2016 study period, using the HCS protocol.
Discussion
  1. The population dose concept has been used to compare different combinations of programs and policies implemented by numerous communities. Since 2005, when the first three Kaiser Permanente CHI sites were begun in Colorado, Kaiser Permanente has implemented CHI in six of eight regions and nearly 60 communities. As of 2015, a total of 730 program, policy, and environmental strategies have been implemented, reaching a total of 715,000 individuals. Fifty-one percent of the strategies have resulted in policy or environmental changes, a key CHI design principle given evidence that such strategies may be more sustainable over the long term. Another 18% of strategies focused on community capacity building. All of the observed population health changes related to the presence of strong interventions (high dose) took place in schools, as opposed to community settings, and most school changes were in physical activity.
  2. A total of 9,681 community programs and policies were documented across the 130 HCS communities. Most community programs and policies were ongoing in duration, had low reach (i.e., 1–5% of the population), and used moderate-strength strategies. Overall intensity and related features of programs and policies showed variation across communities and increases over time. More community programs and policies implemented in communities addressed physical activity than nutrition (58% vs. 27%). The most frequent sector in which efforts were implemented was schools, and the vast majority of efforts targeted children directly.
  3. A total of 916 key informant interviews or surveys were conducted in CHKS. Of these, 60 were for middle school nutrition, 60 were for middle school physical activity, 142 were for elementary feeder school nutrition, and 236 were for elementary feeder school physical activity. In addition, 161 key informant interviews were conducted with early childhood-based organizations, 74 with community-based organizations, 52 with health care-based organizations, and 131 with government-based organizations. Data were summarized into a total of over 100 independent variables. Rather than relying on a priori scoring of the variables into a smaller number of factors, a data adaptive approach was used to identify the combination of factors that were most associated with student BMI.

The findings from these observational studies provide important information regarding what communities are doing to address pediatric obesity. They suggest common approaches already occurring in many communities that can inform future efforts to address childhood obesity.

Key Recommendations
  1. For comprehensive assessment of what communities are doing to prevent pediatric obesity, a combination of interview, survey, document review, secondary data, and site observation is recommended.
  2. Multiple ways of expressing data on community efforts may be necessary as different expressions may be related to different outcomes.
  3. Data adaptive approaches for characterizing community efforts should be further explored to minimize assumptions used in scoring.
  4. Prospective collection of data to comprehensively characterize community efforts, while more labor-intensive, is preferred over retrospective data collection.
References

Bleich SN, Segal J, Wu Y, Wilson R, Wang Y. Systematic review of community-based childhood obesity prevention studies.Pediatrics. 2013;132:e201–e210.

Brennan LK, Brownson RC, Orleans CT. Childhood obesity policy research and practice: evidence for policy and environmental strategies. Am J Prev Med. 2014;46:e1–e16.

Cheadle A, Schwartz PM, Rauzon S, et al. Using the concept of “population dose” in planning and evaluation community-level obesity prevention initiatives. Am J Eval. 2012;34:71–84.

Economos CD, Hyatt RR, Must A, et al. Shape Up Somerville two-year results: a community-based environmental change intervention sustains weight reduction in children. Prev Med. 2013;57:322–327.

Hoelscher DM, Kirk S, Ritchie L, Cunningham-Sabo L. Position of the Academy of Nutrition and Dietetics: interventions for the prevention and treatment of pediatric overweight and obesity. J Acad Nutr Diet.2013;113:1375–1394.

Institute of Medicine. Accelerating Progress in Obesity Prevention: Solving the Weight of the Nation. Washington DC: The National Academies Press; 2012.

John LV, Gregoriou M, Pate RR, et al. Operational implementation of the Healthy Communities Study. Am J Prev Med. 2015;49:631–635.

Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of childhood and adult obesity in the United States, 2011–2012. JAMA. 2014;311:806–814.

Schwartz P, Rauzon S, Cheadle S. Dose Matters: An Approach to Strengthening Community Health Strategies to Achieve Greater Impact. National Academy of Medicine Discussion Paper. August 26, 2015. https://nam.edu/dose-matters-an-approach-to-strengthening-community-health-strategies-to-achieve-greater-impact-2/ . Accessed November 7, 2017.


James F. Sallis, Ph.D.

James F. Sallis, Ph.D. James F. Sallis, Ph.D., is Distinguished Professor Emeritus in the Department of Family Medicine and Public Health at the University of California, San Diego and Professorial Fellow at Australian Catholic University in Melbourne. His doctorate in clinical psychology is from Memphis State University (now University of Memphis). His primary research interests are promoting physical activity and providing evidence to guide policy and environmental strategies to improve physical activity, sedentary behavior, nutrition, and obesity. His health improvement programs have been studied and used in health care settings, schools, universities, and companies. From 2001 to 2016, he was director of Active Living Research, which managed a $27 million research budget and supported 250 projects. He is an author of over 600 scientific publications, on the editorial boards of several journals, and one of the world’s most cited authors in the social sciences. Thomson-Reuters identified him as one of the world’s most creative scientific minds in 2014–2016. He is a member of the U.S. National Academy of Medicine, was named a fellow in four scientific societies, and received a Lifetime Achievement Award from the President’s Council on Fitness, Sports, and Nutrition. He is Past-President of the Society of Behavioral Medicine.

Measures of Co-Benefits: Considerations Based on Research Question and Population

Background

In obesity research, it is common for a measure of obesity or body composition to be the primary outcome, and often the only outcome. There is a strong rationale for including additional outcomes, especially in evaluations of natural experiments that can have broad audiences. Natural experiments can be considered examples of “strategic science” (Brownell and Roberto, 2015) or studies intended to inform or influence the actions of practitioners and policymakers. Studies indicate that such decision-makers value the type of “real-world” evidence provided by evaluating natural experiments, which are closely related to decisions they have made or are considering (Sallis et al., 2016). Thus, the types of outcomes collected should be relevant to the needs and interests of all users of the research.

Even though health outcomes, such as obesity, are understandably of greatest interest to health researchers, other target audiences want additional information, especially if they work in “non-health” sectors such as industry, city planning, transportation, education, parks and recreation, or are elected officials with broad responsibilities. Additional outcomes can be thought of as “co-benefits” of obesity control interventions, but stakeholders in other sectors may consider them primary outcomes. Assessing co-benefits has the potential to enhance the impacts of obesity interventions on policy and practice, especially if such results are communicated effectively to decision-makers.

Brief Content Summary

Primary outcomes of obesity-related natural experiments typically are eating behaviors, physical activity, sedentary behaviors, and self-weighing. Domains of co-benefits can include physical health, mental health, social health, school or job performance, traffic injuries, crime and security, environmental sustainability, and economics (Sallis et al., 2015). Equity of outcomes can also be considered a co-benefit of natural experiments. Selection of co-benefit measures is complicated by the multiple dimensions or aspects of each domain, population, and audience that can be considered. The co-benefits that can be expected depend on the nature, setting, and target population of the natural experiment.

Measures of health-related outcomes are usually well developed, and there are resources to support measurement selection, such as the National Institutes of Health’s Patient-Reported Outcomes Measurement System (PROMIS) measures (Ader, 2007) and the National Collaborative on Childhood Obesity Research’s measurement and surveillance system registries (McKinnon et al., 2012) and user guides (http://www.nccor.org/nccor-tools/mruserguides/). Selecting measures in “non-health” domains is likely to benefit from, or even require, collaboration with experts in other sectors. Obtaining input from “end users” of the results, such as practitioners and policymakers, is useful to ensure the measured variables are meaningful to those audiences.

Education and business professionals are knowledgeable about indicators of school and job performance. Transportation and public health experts in injury control can help identify measures of traffic injuries. Criminologists have expertise in measuring crime and perceptions of crime. Environmental scientists, transportation researchers, and environmental health experts understand measures of pollution and greenhouse gas emissions. Economists are needed to provide guidance related to the goals and methods of economic evaluation. There are public health professionals and advocates who have expertise in measuring health equity. Developing a plan to evaluate co-benefits of natural experiments is an interdisciplinary activity. Engaging partners with a diversity of expertise has the advantage of facilitating communication of results to decision-makers in the multiple sectors who will be interested in the natural experiment and have responsibility for making decisions that will affect how lessons from natural experiments will affect policy and practice in the future.

Discussion

At this point, it is not possible to provide a standard list of measures of co-benefits of obesity-related natural experiments. But it is possible to recommend key questions that can be used to guide the process of developing a co-benefit measurement plan.

  • What co-benefits beyond primary obesity-related outcomes can reasonably be expected from this natural experiment?
  • Which disciplines, sectors, agencies, and organizations have expertise in the co-benefit domains?
  • Which government agencies, industries, and professional organizations are involved in practice and policy in each of the targeted domains of co-benefits?
  • Within each co-benefit domain, which specific variables are likely to change, and which variables are of most interest to practitioners and policymakers?
  • For each variable, what measures (1) have good evidence of psychometric quality, (2) are relevant to the context and study population, (3) are relevant to end users of results, and (4) are feasible for implementation and communication?
Key Recommendations
  • Commission a review of obesity-related natural experiments to document the evaluation of co-benefits.
  • Conduct qualitative research with experts in each domain or discipline of co-benefits about the most important variables in their areas, recommended measures of those variables, barriers to and facilitators of their engagement in obesity-related research, and approaches to engaging professionals in their disciplines in obesity-related research.
  • Conduct qualitative research with likely “end users” of natural experiment evaluation results who are involved in policy and practice across relevant sectors. Obtain input on their use of evidence in decision-making, the types of co-benefits that are most meaningful to them, and effective methods for communicating results.
  • Convene an interdisciplinary working group to develop a consensus on best practices in assessing co-benefits of obesity-related natural experiments.
References

Ader DN. Developing the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45(5):S1–S2.

Brownell KD, Roberto CA. Strategic science with policy impact.The Lancet. 2015;385(9986):2445–2446.

McKinnon RA, Reedy J, Berrigan D, Krebs-Smith SM. The National Collaborative on Childhood Obesity Research Catalogue of Surveillance Systems and Measures Registry. Am J Prev Med. 2012;42(4):433–435.

Sallis JF, Bull F, Burdett R, et al. Use of science to guide city planning policy and practice: how to achieve healthy and sustainable future cities. The Lancet. 2016;388(10062):2936–2947.

Sallis JF, Spoon C, Cavil N, et al. Co-benefits of designing communities for active living: an exploration of literature. Int J Behav Nutr Phys Act. 2015;12(1):30. https://ijbnpa.biomedcentral.com/articles/10.1186/s12966-015-0188-2 . Accessed November 7, 2017.


Glenn E. Schneider, M.P.H.

Glenn E. Schneider, M.P.H. Glenn E. Schneider, M.P.H., is a passionate, strategic, and skilled public health leader with extensive experience managing successful campaigns that improve lives. He leads a program team that oversees diverse initiatives, including Howard County’s Unsweetened and Sugar Free Kids Maryland, which have successfully advocated for policies that make healthy food and drinks more widely available in parks, schools, government offices, and child care facilities. Sugary drink sales have declined by 20 percent in Howard County due to these campaigns. Before joining the Foundation, Mr. Schneider was a consultant, organizer, executive director, and policy director in the government and non-profit sectors. His work resulted in 35 new state and local laws and regulations across the nation that increased access to health care, improved healthy food access, raised tobacco prices, created smoke-free public places, and cut youth access to tobacco. Mr. Schneider is an adjunct professor at several universities and is a nationally recognized speaker and trainer. His work is featured in the book, The DeMarco Factor: Transforming Public Will into Political Power. Mr. Schneider has a Master of Public Health from the University of Pittsburgh, where he received the school’s highest honor, the Distinguished Graduate Award, in 2002.


Eva Tseng, M.D., M.P.H.

Eva Tseng, M.D., M.P.H. Eva Tseng, M.D., M.P.H., is an Assistant Professor and clinician-investigator in the Division of General Internal Medicine at Johns Hopkins. She attended medical school at Tufts University and obtained a combined M.D./M.P.H. degree. She then completed her residency in internal medicine at Jefferson University Hospital in 2014. Most recently, Dr. Tseng finished the 3-year General Internal Medicine Research Fellowship at Johns Hopkins. Her research is focused on improving health and quality of life by reducing the burden of diabetes and other chronic conditions through prevention. She has had the opportunity to work on several studies and systematic reviews including the recent Agency for Healthcare Research and Quality-funded systematic review and meta-analysis of the comparative effectiveness and safety of diabetes medications. As a primary care doctor, she recognizes the burden of diabetes and related complications among her clinic patients and the missed opportunity for primary prevention. Therefore, her current focus has been on developing and implementing a systematic approach to screening, diagnosing, and managing patients at high risk of diabetes.

Evidence-based Practice Center Presentation

Objective

Given the enormity of obesity as a public health problem, rigorous methodological approaches, including natural experiments, are needed to evaluate the effectiveness of policies and programs to prevent and control obesity. Our objective was to systematically review studies evaluating programs and policies addressing obesity prevention and control in terms of their population-based data sources, use of data linkages, measures reported, study designs, and analytic approaches.

Data Sources

We systematically searched PubMed, CINAHL, PsycINFO, and EconLit from 2000 to August 21, 2017, to identify all U.S. and non-U.S. studies of programs or policies targeting obesity prevention and control in people of all ages and in any setting.

Review Methods

Two independent reviewers screened abstracts and full-text articles. We required articles to address a program, policy, or built environment change and have a defined comparison or unexposed group. We used the Effective Public Health Practice Project (EPHPP) tool to rate studies for their risk of bias.

Results

In all, 295 studies were eligible for inclusion (189 U.S. and 106 non-U.S.); 157 (53%) were natural experiment studies. Studies reported 116 unique primary or secondary shareable data sources, of which 106 (71 U.S. and 35 non-U.S.) data sources met criteria for a data system and 26 of the 71 U.S. data systems were linked with a secondary data source other than the primary data source. Also, 112 studies reported childhood weight measures, 33 had adult weight measures, 152 had physical activity measures, and 149 had dietary measures. Most natural experiment studies were rated as having a “weak” global rating (i.e., high risk of bias), with 64% having a weak rating for handling of withdrawals and dropouts, 25% having a weak rating for study design, 40% having a weak rating for confounding, and 26% having a weak rating for data collection.

Conclusions

We identified a large number of studies and data sources that used a wide variety of outcome measures and analytic methods, often with substantial risk of bias. The findings reinforce the need for methodological and analytic advances that would strengthen efforts to improve obesity prevention and control.


Andrew Turner, M.S.

Andrew Turner, M.S. Andrew Turner, M.S., is the Director and Chief Technology Officer of the Esri R&D in Washington, DC, developing new technology for open data, civic technology, and geospatial web collaboration. Mr. Turner’s work focuses on cross-domain collaboration and democratizing the map-making process—creating open tools for cartography and analysis. In 2006, he published the popular and widely regarded book Introduction to Neogeography, which led an industry shift of the geographic information systems market. He is a well-respected speaker, author, advocate, and engineer for crowd-sourced geospatial technology and a successful entrepreneur who has grown and exited two companies through acquisition. Mr. Turner’s team is developing new technologies for government and citizen collaboration to build communities. This includes the global ArcGIS Hub for open access to authoritative data, as well as open-source and interactive tools and applications to build and share insights and solutions. Mr. Turner is an active member in many organizations developing and supporting open standards such as the OpenStreetMap, Open Geospatial Consortium, Open Web Foundation, OSGeo, and World Wide Web Consortium. He is also the co-founder of CrisisCommons, a global community of volunteers leveraging technology to assist in building solutions for disaster response, recovery, and rebuilding.

Integration of Spatial and Other Data via the Private Sector, Crowd-Sourcing, and Open-Data Initiatives

Background

Evaluating the complex relationships between multi-thematic data can yield valuable insights and identify opportunities. Data that include spatial attributes provide a common context for correlating information between these different themes and sources. Federal and local data such as socioeconomic demographics, environment, and infrastructure are just a few spatial data that can dramatically improve the methods for evaluating health trends and impacts. These data are increasingly made available as open data, meaning they can be freely accessed and reused by researchers, private industry, researchers, other governments, and any public constituent.

Esri works extensively across many governments in the United States and internationally to make their data open, accessible, and usable with analytical tools. Through this work, we have discerned and shared common practices for data gathering, dissemination, and reuse that improve scientific research, policy development, commercial operations, and municipal planning.

Recently, the increased usage of mobile, network-connected sensing technologies such as consumer smartphones has demonstrated the potential for crowd-sourcing to provide more granular, dynamic, and longitudinal data to support authoritative spatial data. Esri has developed mobile survey applications, stream-processing analysis engines, and new analytic techniques that can integrate crowd-sourced and dynamic data with health-related research and experiments.

Brief Content Summary

Spatial data are commonly used to measure the related connections between population and socioeconomic demographics to health.

According to the Centers for Disease Control and Prevention, “Education is one of the strongest predictors of health; the more schooling people have, the better their health is likely to be.1 Additionally, prevalence of poverty, access to health care, and housing all have strong correlation to differences in health.

Esri supports governments sharing geographic data that support statistical and spatial analysis to identify important trends and relationship to actions to reduce obesity and improve health outcomes. These data are available for the public, businesses, and practitioners to incorporate into their workflows and inform their own policies.

Citizens are increasingly leveraged as new measure inputs to fill data gaps as well as understand intent and actions related to health issues. This crowd-sourcing can augment traditional data sources and provide more granular, timely, and longitudinal information for further research.

Reference
  1. Freudenberg N. Reframing school dropout as a public health issue. Prev Chronic Dis. 2007;4:A107. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2099272/. Accessed November 12, 2017.

Contact Us