Food Environment
MEASURES REGISTRY USER GUIDE
8. Case Studies
SECTION
7
Collecting and Reporting Data
As described in Section 6, it is important to consider data collection and reporting methods as measures are selected and also throughout the entire process of planning a food environment measurement project. This section discusses data collection, analysis, and reporting considerations for measures across the physical, social, and person-centered food environments.
Data Collection Mode and Sampling
Physical environment data for use in GIS may be available as existing secondary data sources, for example, lists of food retailers available from Dun and Bradstreet or local- or state-level government agencies (e.g., alcohol law enforcement for off-premise alcohol sales as a proxy for many food retailers), licensing, or tax revenue lists. Other data describing the physical environment (e.g., food availability and marketing of foods at stores or restaurants), or social or person-centered environment will require that data be collected. Data collection mode (e.g., secondary data collection, observational assessment through paper or mobile device, phone versus face-to-face interview) and sampling method (e.g., convenience sample versus random sample) are important considerations as measures are selected.
With regard to sample, it is important to consider the original research question or project purpose. It could be that a convenience sample of stores, schools, or people is sufficient to answer the question. Alternatively, a more sophisticated generalizable sample, generated by a statistician, may be necessary to meet the project objectives. In the case of evaluating an intervention (e.g., a program or policy change), study design is also important. If the project attempts to evaluate a natural experiment, such as the introduction of a full-service grocery store into an underserved area, finding a comparison community to evaluate differences between groups will help the investigator understand the impact of the change. If it is a more planned intervention, collecting pre- and post-project data and paying attention to sample size and randomization are important considerations. Assessing the social environment and the person-centered environment requires a different type of approach, as assessors will need to think carefully about what stakeholder groups will need to be involved and how to sample the groups.
Choice of data collection mode will likely be based on project resources. Some secondary GIS data are available for free; others must be purchased. For primary data collection, paper and pencil data collection tools or electronic mobile devices have costs and benefits. One disadvantage of paper and pencil data collection tools is the need for entering data later into a spreadsheet; mobile devices mitigate this need but involve upfront programming and testing. Telephone data collection may require recording instruments or access to quiet meeting spaces. In-person data collection at stores, schools, or other venues will require data collectors to travel by car or on foot to the locations of interest, incurring additional expenses and time related to travel.
Some projects involve sending data collectors out into the community to collect data, often using environmental scans (e.g., store or restaurant audits, records, or logs) that document the physical environment. Careful planning is required to have community-level data collection campaigns go smoothly. Before the start of data collection, each data collector should be prepared with a list of venues that are assigned to them. Sometimes, in the case of store or restaurant observations, the data collection protocol has data collectors work from a prescribed list; other times, venues are added to a list in the field; still other times, both techniques are used. Whatever the strategy, it should be made clear in advance. Data collection in stores or restaurants does not often require advanced scheduling because it is a public venue, but it is still essential that the data collector introduce themselves and ask permission to conduct the assessment upon arrival. For schools, or other closed venues, data collection visits must be coordinated and scheduled in advance with school administrators, parents, youth, or other relevant stakeholders. It is best to schedule data collection at a time that is most convenient for the venue, rather than the data collector. As with any research study, the length of the data collection visit should be kept as short as is feasible to reduce the burden and inconvenience imposed on the community.
Phone interviews can also be used as a data collection mode, for example, to measure the social food environment (e.g., parent feeding practices or school policies) or the person-centered environment. When collecting data using a phone interview, the project leader must consider important logistical protocols such as how many calls will be made to attempt to complete an interview, how scheduling of calls will be coordinated among project team members, data collection venue (e.g., school), and potential respondents. It is also important to conduct quality checks on interviews to ensure the interviewer is collecting data according to the intention of the selected measure. Recording of several early-round interviews, playback to another trained interviewer, and exchanging feedback, is a helpful strategy in this regard. Additional protocols with regard to obtaining consent from interview participants, and assurance of confidentiality of responses must also be developed.
Many of these phone interview considerations also apply to data collection using a mailed survey. Steps should be taken in advance to develop a data collection protocol that will ensure the highest response rate possible.
Data Collectors
The process of selecting a measure helps the investigator or practitioner become familiar with the types of data that will be collected. A next step is to consider who, specifically, will be collecting the data, and to be certain that the experience of the data collector matches that required to administer the measure. Data collectors may be professional researchers, directors of a practice-based project, or could be students, youth or young adults, or community volunteers. Each of these groups may have different levels of familiarity with the selected measures or data collection in general; experience level should guide data collector training procedures. Thorough training of data collectors is essential to collecting quality data. If a data collector is not familiar with the items within the tool, and the most accurate way to record responses, reliability will suffer. It is also important to consider the amount and kinds of interaction each prospective data collector will have in study locations or with study participants. Data collectors who conduct face-to-face or phone interviews must be trained to collect data in a way that minimizes bias. Assessing objective elements of the physical environment typically involves the least amount of interaction with individuals, particularly if the physical environment is a public space such as a store or restaurant.
Data Collection Time Period
The data collection time period will be related to the project purpose or research question. For physical environment data collection in stores or restaurants, data collection time points should be limited to hours when the venues are less busy. If the venue is very busy at the first data collection attempt, a good protocol is to return at a different time. Other physical environment venues such as schools or preschools may operate seasonally, with different schedules at different seasons. For example, data collectors in school cafeterias may want to avoid end-of-year testing or plan around breaks. In any project it is important to build in time to ensure data quality. Quality checks may include measuring inter-rater reliability on an observational assessment and retraining data collectors in areas with low reliability, or reviewing recorded interviews to check for completeness and evidence of non-bias. Ensuring data quality is an important step in building a good quality food environment measurement project.
Data Analysis and Reporting
Incoming food environment measure data could be in a variety of forms: electronic data files/spreadsheets, paper and pencil surveys that must be recorded, or quantitative or qualitative data from interviews or questionnaires. Analyses will be informed by the project purpose or research questions. At the end of the project, what is it that the researcher or practitioner would like to know, very specifically? To what extent do the chosen measures help answer that question? This list of knowledge points can guide the data analyses. Before data collection, it may be helpful to identify a use for an item contained within the data collection instrument. If no use for each item is identified, one may reconsider why the item is being used. With quantitative data, it is helpful to start with univariate statistics for the variables of interest that are present on the data collection instrument (e.g., proportion of surveyed stores with fresh fruit available). The next step is to move to bivariate and multivariate statistics to understand relationships between variables of interest (e.g., availability of fresh fruit and neighborhood by demographics). Consider charts, infographics, tables, or maps to report findings; often free or low-cost software tools can be used to create compelling graphics (e.g., https://piktochart.com). It may be helpful to identify an individual data analyst or team in advance of the project, as well as back up support. Matching the kinds of data analysis required for the measures with the skills of the current team is important for project success. If special analytic skills are needed to assess, manipulate, and interpret the data collected, it is important to plan for those in advance.