Physical Activity Environment
MEASURES REGISTRY USER GUIDE
SECTION
6
Selecting Measures
This section describes the various purposes for using built environment measures, how users can identify and refine their purpose, and key selection considerations for each project purpose. This section also suggests processes for using and searching for measures within the Measures Registry.
Defining Project Purpose
Physical activity environment projects can have a wide variety of purposes and objectives, as highlighted in the Behavioral Epidemiology Framework presented in Figure 4. The figure distinguishes five project purposes: Basic, Health Outcomes, Surveillance, Theory and Correlates, and Intervention. The majority of projects in the practice domain fall under Surveillance and Intervention Research. Environmental research fits within each of the five levels, but has less of a role in Basic Research. As outlined in Section 2, physical activity environment research is guided by ecological models that suggest that physical activity is influenced by factors from multiple levels.8 Although many study purposes are multi-level, this section specifically focuses on selecting measures at the environmental level.
Figure 4: Behavioral Epidemiology Framework
Source: Welk, 2002. The figure was adapted to feature physical activity environment measures.122
Once the study purpose is refined using the Behavioral Epidemiology Framework, clearly defined project objectives can be identified to facilitate selecting appropriate environmental assessment tools. Objectives should be SMART: Specific, Measureable, Achievable, Realistic, and Time-bound.123 The primary objectives of conducting environmental assessments include:
- To conduct periodic measures for public health planning purposes and assessing trends over time (i.e., surveillance).
- To observe associations between environmental attributes and physical activity, sedentary behavior, or health (i.e., observational/correlates study)
- To conduct intervention research or evaluation involving one or more of the following:
- To educate and/or engage community members in advocacy around environments and health (i.e., community engagement)
- To identify areas and/or attributes in need of improvement (i.e., needs assessment); can also include assessment of disparities in physical activity environmental attributes
- To evaluate whether and how environmental attributes change over time or as the result of an intervention (i.e., evaluation)
Whether grounded in research or practice, most projects fall into one or more of these three purposes. An example of a multiple-purpose project is when an investigator may conduct a physical activity intervention and wants to use built environment assessment to 1) engage participants in advocacy efforts, and 2) evaluate whether environmental improvements occur. Correlates research also can be combined with health outcomes research, for example when physical activity is investigated as a mediator of the link between the built environment and health outcomes.
Considerations in Selecting Measures
Once the project purpose is defined using SMART objectives, multiple factors should be considered in selecting the most appropriate tool(s) for environmental assessment. These factors include the setting of interest, comprehensiveness, reliability and validity, relevance to population being studied, resources and expertise required, and flexibility and adaptability to local needs.
Setting of Interest
Selecting the setting of interest is often evident once the project objectives are finalized. A researcher should select the setting(s) that affect the population being studied or the setting in which the intervention is taking place. In youth, this often involves homes, schools, neighborhoods, parks, and recreation settings. Because it is often not feasible to assess all of these settings, settings should be prioritized based on their relevance (e.g., where youth spend a large amount of time), availability of partners (e.g., where partners work and/or have influence), and/or potential for intervention. However, because it is uncommon for a single measure to cover multiple settings, multiple measures may be needed to adequately account for the variance in youth’s overall physical activity.
A practitioner or researcher should select the settings that have potential for being affected by policy or environmental changes. For example, a needs assessment could be conducted on streetscape characteristics around schools seeking or receiving Safe Routes to School funding.124 When using environmental assessment for community engagement, the involvement of community members could inform which setting(s) to target.
Comprehensiveness
Comprehensiveness is an examination of whether a measure or set of measures sufficiently assesses the aspects of the physical activity environment that need to be assessed. For example, if the aim of a project is to examine changes in the neighborhood built environment following the installation of a new bike/pedestrian trail, measures of the quality of as well as the access to (e.g., pedestrian features linking the trail to existing neighborhood) the bike/pedestrian trail could be examined. Comprehensiveness of measures includes the number of settings as well as the number of attributes or constructs within each setting being selected. Comprehensiveness should be balanced with project goals, feasibility, resources required, and measurement properties (e.g., reliability and validity). However, comprehensiveness is not necessarily the equivalent of length. Long measures can include multiple inter-related items representing single constructs (e.g., multi-item scales) and brief measures can cover multiple constructs using one to two items per construct.
Reliability and Validity
Having evidence of acceptable measurement properties is important for both research- and practice-based projects. Tools with poor reliability and/or validity limit the ability to detect differences among groups, changes over time, or associations between environmental attributes and health outcomes. Reliability and validity are less important when environmental assessment is used for community engagement, but if the results of the assessments will be used for a needs assessment or to target environmental changes, it is important that the tool have acceptable measurement properties. Visit the “Validity” and “Reliability” tabs for each measure in the Measures Registry, where results are presented when available.
Relevance to the Population Being Studied
It is important to note that reliability and validity can differ across geographic areas and participant populations, the latter being particularly relevant for self-report measures. A tool validated in urban areas or with a predominantly white non-Hispanic sample may not be reliable or valid for use in other geographic areas, cultures, or samples. Similarly, a measure’s utility is specific to the purpose of the project. A tool may have acceptable validity for observational research but have poor sensitivity for detecting environmental changes over time, often because the response categories are very broad or there are no items to assess the planned intervention. Thus, users should consider whether the tool was designed and evaluated in a similar geographic area and/or population and for a similar purpose as the user intends. A majority of the measures in the Measures Registry have not been tested in multiple geographic areas and/or populations, particularly diverse populations in terms of race/ethnicity and socioeconomic status.
Rural and international environments are two key areas where environmental assessment tools diverge, particularly those related to community design. A strength of the NCCOR Measures Registry is that rural measures are separated from urban/metropolitan measures so they can be easily identified. When applying a questionnaire or audit tool to another country, adaptations are often needed, and GIS-derived data and relevant variables can vary drastically across countries125 due to regional and historical differences in community design.
Resources and Expertise Required
The practicality of using any environmental assessment tool comes down to its complexity and cost for data collection and management. The “How To Use” tab in the Measures Registry provides useful information about resources needed to administer the measure, when available. GIS expertise is specialized and costly, and not all GIS technicians will have expertise in the variables related to physical activity. Accessing GIS data from multiple sources and deriving variables are complex and time-consuming activities. However, GIS-based methods are advantageous in large samples of individuals from few geographic regions because the cost is primarily based on the number of jurisdictions and variables rather than number of people. Audits, on the other hand, carry a per-person or per-environment (e.g., school, neighborhood) cost, making them less feasible for use in large studies. Audits require expertise in training in observational methods, ongoing attention to quality control, and management of complex data. The primary limitation to questionnaires is participant burden. Knowledge and careful consideration of the study participants and the time required to complete all study questionnaires will inform the selection of appropriate questionnaires. Data management demands are least complex with questionnaires.
Complexity and cost also vary within each method of assessment. Data collection, training, and scoring guides greatly reduce the complexity and cost of using GIS, audit, and questionnaire tools. The quality and comprehensiveness of the guides should be assessed before a measure is selected. Measure developers may or may not be able to answer questions or provide assistance, so thorough documentation is highly desirable. Smartphone and tablet applications can significantly reduce data management burden with audits and questionnaires as long as they have been fully vetted.
Flexibility and Adaptability
It is important to choose a flexible measure in situations where the tool may need to be adapted or tailored to specific populations or settings. New items in audit tools and questionnaires should be pilot tested for inter-rater or test-retest reliability when possible. Tools that use smartphone or tablet applications or require complex scoring algorithms may be less flexible because of difficulties in making software or scoring modifications. GIS-based measures can be flexible; for example, a variable can be computed multiple ways, and the options will be determined by the datasets available.
Suggested Process for Using the Measures Registry
The following sections outline the main steps that users can follow to use the Measures Registry to select the most appropriate measures for a designated purpose.
Searching and Filtering Results
The Measures Registry can be accessed from the NCCOR home page, under the “Tools” tab. The physical activity environment measures in the Measures Registry are organized by “Measure Type” (i.e., measurement method), “Ages” covered, and “Context” (i.e., urban or rural). The check boxes within each category can be used to narrow the results field. Once the user selects the desired measure type and context, a list of measures fitting those criteria is displayed. When a measure has been evaluated (i.e., tested for reliability or validity), the Registry will most likely include the measurement development or evaluation publication. If multiple measurement evaluation publications exist for a given measure, the Registry will typically include each of the publications. If a measure development or evaluation publication does not exist, the Registry will include the first paper published using the measure, typically a physical activity correlates study. Many of the publications in the Registry are not measure development or evaluation studies.
The Measures Registry does not include a setting code specific to physical activity environments to organize studies. When a user is interested in a particular setting, it is recommended to use the search function. Entering keywords such as “home environment” or “school” into the search box results in a list of publications that included those keywords in the title or abstract. The search function can be used in combination with the category check boxes to further refine the search, such as limiting the search to a specific setting and measurement method.
Navigating the Information Tabs Within Each Publication
Clicking on a publication’s title will open a link with more detailed information about the measurement tool. It is recommended that each tab be viewed in detail while keeping in mind the selection considerations outlined above. The “At A Glance” tab includes helpful information when available, such as the length, constructs covered, and how to obtain the measure. The “Study Design” tab reports the characteristics of the sample used to develop and evaluate the measure, so users can consider whether the tool is appropriate for the population they intend to study. The “How To Use” tab includes information on how the tool is administered and whether data collection and/or analysis protocols exist. In circumstances where the Measures Registry does not include a link to the measure or protocols, the user should contact the authors of the study. Finally, the “Validity” and “Reliability” tabs include specific results from the publication on the tool’s measurement properties. If a tool has multiple publications in the Measures Registry, the user should view the tabs for each publication.