Case Study 1: Evaluating a School-Based Intervention on its Ability to Positively Influence the School Food Environment

Background

A project team is planning a school-based obesity prevention intervention attempting to change à la carte offerings in middle school cafeterias. The primary outcome of the study is foods sold in the cafeteria using sales data from cash register receipts. Twenty-four schools in a metropolitan area have been recruited to participate in the study.

For their primary outcome, the team has already verified that the schools will be able and willing to provide sales data that can detail the food items purchased on a daily basis. The investigators also want to be able to assess foods and beverages available in the schools before and after the intervention period as process data. They want to be able to verify that the intervention was delivered as planned and that healthy foods being offered à la carte increased in the intervention schools.

Considerations

The first thing the team needs to consider is how they want to document what is available on the à la carte line. They can: (1) ask the cook manager for purchase orders from vendors; (2) ask the cook manager to list all of the à la carte items available; (3) have team members document all of the items available on à la carte using an inventory; or (4) have team members document what is available using a checklist of types of foods.

As they make these decisions they also need to consider the following:

• How many data collection periods will there be?
• What resources are available for data collection, cleaning, and analysis?
• What is the team’s relationship with the school staff? Are staff at all of the schools involved willing and able to provide the data required?
• What level of information is needed for the study? Is it sufficient to know basic information, such as the proportion of healthy foods to less healthy food types available à la carte or is more detailed information needed, such as average calories or grams of fat available from foods à la carte?

As part of their formative assessment, the team discovers that purchase orders from vendors are not available and that not all schools involved are willing and able to follow a study measurement protocol. Therefore, the team realizes that it needs to collect the data. Formative assessment also shows that some of the schools have dozens of à la carte items and that the items change frequently. Therefore, the team decides to look for a valid and reliable data collection tool that will be completed by the study team to document the available à la carte items.

Measure Selection

The team enters “School” as a search term on the Measures Registry and find more than 100 matches. Within that search, they find reference to the TACOS study²¹ that was an intervention attempting to positively affect change in the foods offered on à la carte lines in 24 secondary schools. The dependent variable in the TACOS study was the use of sales data using cash register information. This study design and primary outcome matched the team’s objectives well so they chose to use the protocol from the TACOS study to collect data for the primary outcome.

TACOs used a comprehensive inventory to document all of the foods available in the à la carte line.⁴⁹ Trained study staff went into the 24 schools before randomization and documented each à la carte item available. Information collected on all foods available for sale in à la carte areas included brand name, package size, serving size, and grams of fat per serving. Teams of two or three TACOS staff members met with kitchen managers at each school to review and verify the à la carte food list. However, TACOS staff made return visits to the school and also follow-up telephone calls to food service staff and food manufacturer representatives to gather details about foods offered and their nutritional information. The list of individual food products was grouped into 24 categories (such as chips and crackers, candy/candy bars, pizza, and fruits and vegetables) based on foods similar in nutrients of interest or foods that composed a large share of à la carte sales. The foods available were entered into a nutrition software package (Nutrition Data System) that provided details on macro and micro nutrient content of the foods. The researchers considered this option, liking the detail available on both the foods and nutrients available in the school à la carte lines. However, they are concerned that they do not have the staff, time or other resources to collect the data at the schools or to do the required data entry, cleaning, and analysis.

The team continues looking on the Measures Registry and finds an article by Hearst et al.⁵⁰ that used a similar inventory system to collect data on à la carte items in middle schools as part of the TREC IDEA study.⁵¹ This group had found the complete inventory approach too burdensome to collect in the high schools in their study and developed a simple checklist that included a list of 20 categories of foods based on the CDC’s School Health Policy and Practice Survey.¹⁶ The group conducted a validation study in 38 schools to determine whether the healthfulness rating between the inventory approach and the checklist approach would similarly rank schools.

To determine the healthfulness of à la carte offerings in the middle schools using the inventory method, each food and beverage item on the inventory was classified as not meeting (score = 0) or meeting (score = 1) IOM criteria.⁵² IOM criteria include (1) food servings less than 35 percent of calories from fat, (2) food servings equal to or less than 200 calories per serving for food, (3) a serving size of less than 4 ounces of 100 percent fruit juice for middle school students, and (4) water without additives or carbonation. A total score representing the proportion of foods and beverages offered that met the IOM criteria was created for each school.

Based on this information and the consideration for the resources available in their study, the team decides to use the TACOs data collection method for their primary outcome and the IDEA checklist for their documentation of foods available on the à la carte lines.

Case Study 2: Evaluating a Family-Based Intervention on its Ability to Reduce BMI-Z Scores in Children with Obesity

Background

A project team is planning to evaluate the effectiveness of a family-based obesity treatment intervention for children ages 8 to 10 years using a randomized controlled trial with 100 families. Their primary outcome is change in child’s BMI-z score and the secondary outcomes are child-level caloric intake and Healthy Eating Index (based on four 24-hour recalls). Interventionists will work with families randomized to the intervention condition to help change the foods available in the home and on fostering positive parenting practices and attitudes around child eating behavior.

Considerations

The team has experience in collecting anthropometric data to assess BMI-z score and in collecting and analyzing 24-hour recall data. They are looking to the Measures Registry for resources to assess foods available in the home and to assess parenting practices and attitudes related to children’s eating behavior. These data will be used to characterize the obesogenic nature of the homes as well as to assess change in the home environment related to the intervention.

To guide their selection of measures, the team asks:

• What are the specific environmental targets that the intervention is attempting to change?
• Will the intervention target all foods in the home or just selected foods (for example, increasing fruits and vegetables or eliminating soft drinks in the home environment)?
• Are there valid and reliable existing parenting practice and attitude scales for children ages 8 to 10?
• Who will collect the data? What resources are available for cleaning and analyzing the data?

Measure Selection

The team begins the search by selecting “food environment” and entering “home food inventory” as a search option. Of the options available, several are immediately eliminated because they are for the wrong age group (i.e., infants, preschool) or population (i.e., WIC participants or Spanish-speaking/Somali populations). The investigators decide that they are interested in a more comprehensive picture of the home food environment and therefore eliminate the inventories that were designed to assess only fruits and vegetables or only packaged foods using UPS codes. Of the options left, one (Home Food Inventory [HFI]),³⁵ meets some important criteria: (1) a wide range of foods in the home were assessed; (2) the measure could be completed by parents; (3) information on how to construct an obesogenic index from the HFI was delineated; (4) inter-rater reliability had been tested and shown to be good; (5) criterion validity had been tested using research staff as the gold standard and shown to be very good; and (6) construct validity had been tested and shown to be acceptable, including a significant association between the constructed obesogenic home food availability score and parent and child’s energy intake. In addition, the instrument was available as a download on the NCCOR Measures Registry site.

Next, the team looks for an appropriate instrument to assess parental practices and attitudes related to child’s eating behavior. They include children ages 6 to 11 years and enter “parenting practices” as a search term. Several good options emerge:

• Larios et al.⁵⁴ reported on a bilingual (Spanish and English) survey administered to Latina mothers about parenting strategies for eating and activity. The constructs assessed included limit-setting, monitoring, discipline, control, and reinforcement. Reliability related to internal consistency was assessed and found to be good, and both types of criterion-related validity (i.e., predictive and concurrent validity) were assessed and found to be good. In particular, correlations with child’s BMI-z score were in the expected direction for all five of the constructs. The questions used were available in the article as published.
• Gattshall et al.⁵⁵ included two scales assessing parental role modeling around healthy eating and parental policy on healthy eating that were developed for and tested with overweight children ages 8 to 12. These scales showed good reliability as well as construct validity; parental role modeling and parental policies were related to child and parent eating habits.
• The Child Feeding Questionnaire (CFQ)¹⁹ was another good option and included seven scales assessing the following constructs related to child eating behavior and the family food-related environment: perceived responsibility of parents; parents’ perceptions of their own weight during the life course; parents’ perceptions of their child’s weight; parents’ concern about their child’s weight; food restriction in the home; parent practices related to pressuring their child to eat; and parental monitoring of their child’s eating behavior. The scales were tested in three different samples of parents, including Hispanic mothers and fathers. Internal consistency of the scales was shown to be good and construct validity linking scores on the CFQ and child weight status was confirmed.

Because the CFQ included the broadest interpretation of parental food practices and attitudes; had been developed, tested and used in similar age samples; and was found to have good psychometric properties, the investigators opted to use the CFQ.

Case Study 3: Improving Healthy Eating Behaviors in Independent Neighborhood Restaurants

Background

A large city health department is working with the local restaurant association to improve healthy eating behaviors within independent neighborhood restaurants. Their goal is to prevent obesity and chronic disease among city residents and promote economic development. The project involves baseline data collection of the availability and prices of healthy options, an intervention to support restaurant owners as they revise their menus, and repeated data collection at the end of the two-year project. Their goal is to identify change in availability and pricing over time and changes in menu item sales.

Considerations

The project partners are interested in working with restaurants to increase healthy food offerings at prices that encourage consumption.

After recruiting independent restaurants that serve populations who are most affected by diet-related chronic diseases, they must train health department and restaurant association staff to collect data about menu offerings (e.g., types of food, serving size, price per serving), contextual factors in the restaurant that may influence decision making (e.g., presence of menu labeling), and analysis of a sample of sales records from before and after the intervention.

Measure Selection

A study team leader visits the NCCOR Measures Registry to identify existing measures that can be used verbatim or adapted for the study. To narrow the choices, the team leader selects the “Food Environment” domain, the “Environmental Observation” measure type, and the “Metro/Urban” context.

The team leader scans the measure names on the list of nearly 100 matches for words that are most relevant to the study purpose (e.g., restaurant, menu, and price). Based on these additional criteria, the team leader clicks “Compare” on the eight most relevant measures. They consider the Food Price Comparison (FPC);⁵⁶ Food Price Surveys (FPS);⁵⁷ Healthy Food Availability and Pricing Checklist (HFAPC);⁵⁸ Marketing and Availability of Healthy Options in Restaurants (MAHOR);⁵⁹ Menu Checklist on Healthy Choice Cues (MCHCC);⁶⁰ Nutrition Environment Measures Survey – Restaurant (NEMS-R);³² Price and Availability Indices of Healthy Food (PAIHF);⁶¹ and Restaurant Physical Environment Profile (RPEP).⁵⁹

All eight candidate measures have known validity and reliability, which is important to every project, but only four of the candidate measures make the complete instrument available. Therefore, measures without available instruments are ruled out (FPC, FPS, MCHCC, and PAIHF).

The team leader reviews the four remaining options (HFAPC, MAHOR, NEMS-R, and RPEP) with project partners. Given that the NEMS-R has been widely used, offers a free training, and has demonstrated reliability, it is chosen for this project. However, given limitations in established construct validity, project leaders decide to structure their work so that they can contribute to the field by testing for evidence of construct validity in the relationships between, for example, the sum (price) of individual items compared to a combo meal, prices of healthy entrees compared to regular ones, presence of charge for a shared entrée, or price for smaller portion compared to regular portion and hypothesized sales of “healthy” versus “unhealthy” options.

Case Study 4: Implementing a Farmers’ Market-based Obesity Treatment Program to Change Purchase and Eating Behaviors for Women and Children Enrolled in Women, Infants and Children and SNAP Programs

Background

A state Department of Health and Human Services is partnering with the statewide Farmers Market Coalition to implement an obesity treatment intervention to change purchasing and eating behaviors for women and children receiving benefits from the Women, Infants, and Children (WIC) or SNAP programs. The intervention consists of classroom-based training, including preparing fruits and vegetable dishes from canned and frozen foods; marketing and promotion of fruits and vegetables in stores; home visits; and check-ins with a nutritionist. The primary outcome for the year-long project is parent and child BMI. Secondary outcomes are changes in home food environment and fruit and vegetable intakes.

Considerations

One important aim of the project is to alter the home food environment as a result of the intervention. The home food environment will be measured both by the project participants (the WIC/SNAP benefit recipient who lives in the home) and the project administrators.

A team leader navigates to the NCCOR Measures Registry to identify existing measures that can be used verbatim or adapted for the study. To narrow the choices, the team leader selects the “Food Environment” domain, the “Environmental Observation” measure type, and adds the word “home” to the "Search" box to yield more than a dozen choices. The team leader scans the measure names on the list for words and phrases that appear most relevant to the study.

Measure Selection

Most of the measures that are listed in the results from this search do not seem relevant to the home environment given their titles: Assessment of Worksite Canteen Lunches;⁶² Availability of Nutrition Information from Chain Restaurants;⁶³ Food Desert Identification;⁵⁸ Food Price Comparison;⁵⁶ Healthy Food Availability and Pricing Checklist;⁵⁸ Healthy Food Pricing for 5 to 16 Year Olds;⁶⁴ Marketing and Availability of Health Options in Restaurants;⁵⁹ Price, Availability, and Variety of Fruit and Vegetables;⁶⁵ Restaurant Physical Environment Profile.⁵⁹

However, one measure looks promising, Exhaustive Home Food Inventory for WIC Participant Households.⁶⁶ According to the registry, this Food Inventory has objective measures of food quality for all foods in the home. It needs to be administered by project or research staff, through direct in-person observation. Training is required to complete the measures; however, the time to training and time to administer the measure is not reported. The “How to Use” tab for the Exhaustive Home Food Inventory includes information about the data collection protocol. It is administered by collecting information through Scanned Universal Product Codes (UPC), which are then transferred to a laptop computer, and linked to a reference database. Given the close match to this project, the team opts to use this measure.