Continued development in the field of measuring the food environment would benefit from attention to several areas including: (1) considering the level of variability obtained in food environmental measures, (2) moving beyond observational data and increasing the evaluation of longitudinal relationships and change over time, (3) increasing attention paid to the expected associations between environmental measures and outcomes, and (4) continued efforts to promote use of common measures where possible.

Considering the Level of Variability Obtained in Food Environmental Measures

Variability is an important consideration in designing and powering studies. The degree of variability between people helps determine the sample size needed to test our hypotheses. Another type of variability is the amount of fluctuation in the factor being measured within an individual. As an example, in the field of diet assessment researchers often use 24-hour dietary recalls to to assess “usual intake” of some food or nutrient in order to make associations between dietary intake and disease risk. To estimate the number of people needed in a study to detect group level differences in some nutrient level, they need to know how variable that nutrient intake is between people, or the “inter-individual variability.” But if they want to know how dietary intake is related to a health outcome, they also need to consider how dietary intake changes from day to day for an individual, or the “intra-individual variability.” For example, if they want to understand how one’s caloric intake is related to their weight, they need to collect about 3-4 days of dietary recalls. But if they want to know how one’s intake of Vitamin A is related to their cancer risk, they need to collect in excess of 20 days of recalls to get a sense of usual intake of Vitamin A-rich foods. The difference in the number of recalls needed is because calorie intake stays relatively stable day to day while consumption of foods that are rich in vitamin A varies greatly day to day. How many recalls is “enough” is based on an assessment of the amount of intra-individual variation in the dietary factor.

Variability should also be considered when trying to understand the relationship between environmental factors and population health outcomes. Both the physical and social environments change with some regularity, and, depending on one’s research or practice question, that variability may be very important to consider. A single environmental scan of the physical food environment may not adequately capture what that environment usually looks like any more that a single dietary recall represents what an individual usually eats. As an example, the choices available at a farmers market may change day to day and week to week (representing intra-environmental variability). Therefore, a single assessment of a farmers market may not be related to individuals’ intake of fruits and vegetables. Multiple assessments of products and foods available at the market would be necessary to approximate "usual" exposure. The field has not yet begun to estimate intra-environmental variation of the food environment; therefore, the question of "how many assessments of the environment are enough?" cannot be answered at this time.

Moving Beyond Observation Data and Increasing the Evaluation of Longitudinal Relationships and Change Over Time

The examination of the food environment has been primarily observational since work in the field began. Multiple reviews of the food environment literature document that the preponderance of the research is cross-sectional.^{2,13,14,67,68} Although a great deal has been learned about characteristics of the food environment and how they are related to the health of populations, it is impossible to draw causal inference from these studies. Observational studies are complicated by the inter-relatedness of multiple factors in the environment that may be predisposing a population to poor health outcomes. The myriad factors that represent one’s food environment and dietary choices may be important omitted covariates that refute assumptions related to causality.^29,69 Consider the person-centered, social, and physical environments in our simple conceptual model presented in this Guide or a much more complicated model, such as a social-ecological model.⁹ Any of the factors in these models could be important covariates to consider when one is examining the relationship between one element of the food environment and an outcome such as obesity risk. In addition, it is likely that an interaction between the physical, social, and person-centered environment exists that cannot be fully understood with cross-sectional data.²⁴

In addition, public health researchers and practitioners study the food environment, in part, to understand the kinds of interventions, including programs and policies, that would be effective in reducing health risk from the food environment. Therefore, the ability of food environment measures to examine cross-sectional relationships between the food environment and a given outcome is not enough; it is also important that measures are sensitive and specific enough to detect change in the environment and that the change in environment mediates the change in dietary intake and/or the health outcome of interest.²⁴ Additional work is needed studying food measures longitudinally both to assess causality and improve our ability to detect change overtime.⁷⁰

Increasing the Attention Paid to the Expected Associations Between Environmental Measures and Outcomes

More work is needed to help researchers and practitioners understand the level of associations to expect when evaluating psychometric properties related to environmental measures. In one of the first reviews of the food environment literature conducted by McKinnon et al.¹² covering 1990 to 2007, of the 137 articles reviewed, only 13.1 percent reported on any psychometric properties of the measurement tools described and only 5.8 percent of the articles reported on any measure of validity and one-quarter of those reported on face validity only. An update of that review including 432 articles from 2007 to 2015 found some improvement; 25.9 percent of the articles reported on reliability and 28.2 percent reported on validity.¹³ However, of the articles that reported on a tool’s validity, only 3.2 percent reported on construct validity.

Psychometric properties are used to help us evaluate the quality of food environment measures, and for some types of psychometric tests guidelines are available to suggest the degree of associations expected in a quality measure. For example, when assessing internal consistency of a scale of items (a measure of reliability), convention tells us that a Cronbach’s alpha of 0.70 or greater indicates acceptable reliability.⁷¹ But for other psychometric tests of the food environment, there is little guidance on what level of association should be expected to signify a strong measure.

Specifically, while construct validity is very important to assess because it evaluates the relationship between an environmental factor and a health outcome, there is no guidance on the level of association one might expect between factors representing the food environment and a health outcome of interest. As examples: What level of association might be expected between a measure used to assess the availability of sugar-sweetened beverages in the home and the consumption of calories from refined sugar by children in the home? What level of association might be expected from a measure that is used to assess the number of full-service grocery stores in a neighborhood and childhood obesity prevalence rates in the same census track? Beyond examining the level of association as an indictor of the quality of the environmental measure, there is also need to realize that a low association between the environmental factor and a health outcome of interest could be attributed to a host of other issues including: (1) too much measurement error in the dependent or outcome variable; (2) too much intra-environmental variation in the environmental factor being assessed; (3) covariance with other factors in the models that masks associations; or (4) the importance of the relationship in some communities but not in others. A great deal of work is needed to understand construct validity as it relates to measures used to assess the food environment.

Time and resources must be committed to develop and test the quality of the measures that are used to assess the food environment. Without this essential step, it is difficult to have confidence in the associations between environmental factors and health outcomes. In addition, this step is essential before tools should be widely adopted by the larger scientific or practice community. Without data to show that the measures or methods are reliable and valid across communities and neighborhoods, limited resources may be poorly used and false conclusions can be made.